Method, apparatus, medium and device for processing track data in multimedia resources

Abstract

Embodiments of the present application provide a method, apparatus, medium, and device for processing track data in a multimedia resource. The method for processing track data in the multimedia resource includes: receiving a signaling file corresponding to the multimedia resource, where the signaling file contains descriptors respectively corresponding to multiple track data of the multimedia resource, and the dependency identifier contained in the descriptor corresponding to the main bitstream track data points to the descriptor corresponding to the knowledge image track data; parsing the signaling file, and determining the dependency relationship between the main bitstream track data and the knowledge image track data according to the dependency identifier; and sequentially obtaining the knowledge image track data and the main bitstream track data from the data source side according to the dependency relationship. The technical solution of the embodiments of the present application can obtain the association relationship between each track data in advance according to the signaling file, avoiding unnecessary delay caused by temporarily obtaining the knowledge image track data.

Description

Technical Field

[0001] This application relates to the field of computer and communication technologies. Specifically, it relates to a method, device, medium, and equipment for processing track data in multimedia resources. Background Art

[0002] In the AVS3 video coding and decoding technology, in order to improve the video compression efficiency, the concept of knowledge images is proposed. That is, when encoding a video sequence, a main bitstream and a library bitstream can be generated, and the image frames in the main bitstream can refer to the image frames in the library bitstream during decoding. However, the data receiver may not know in advance the relationship between the main bitstream and the library bitstream. If the data receiver has not requested or decoded the corresponding knowledge image track before, it needs to temporarily request or decode the knowledge image track, which will bring unnecessary delays and affect the user experience. Summary of the Invention

[0003] Embodiments of this application provide a method, device, medium, and equipment for processing track data in multimedia resources, which can, at least to a certain extent, obtain the association relationship between each track data in advance according to the signaling file, avoiding the unnecessary delay caused by temporarily obtaining the knowledge image track data.

[0004] Other features and advantages of this application will become apparent through the following detailed description, or will be learned in part through the practice of this application.

[0005] According to one aspect of the embodiments of this application, a method for processing track data in multimedia resources is provided, including: receiving a signaling file corresponding to the multimedia resource, where the signaling file contains descriptors corresponding to multiple track data of the multimedia resource, the multiple track data including main bitstream track data corresponding to the main bitstream and knowledge image track data corresponding to the library bitstream, and the dependency identifier contained in the descriptor corresponding to the main bitstream track data points to the descriptor corresponding to the knowledge image track data; parsing the signaling file, and determining the dependency relationship between the main bitstream track data and the knowledge image track data according to the dependency identifier; and sequentially obtaining the knowledge image track data and the main bitstream track data from the data source side according to the dependency relationship.

[0006] According to one aspect of the embodiments of the present application, a method for processing track data in a multimedia resource is provided, including: generating a signaling file corresponding to the multimedia resource, where the signaling file contains descriptors respectively corresponding to multiple track data of the multimedia resource, the multiple track data including main bitstream track data corresponding to the main bitstream and knowledge image track data corresponding to the knowledge image bitstream, and the dependency identifier included in the descriptor corresponding to the main bitstream track data points to the descriptor corresponding to the knowledge image track data; sending the signaling file to a data receiver, so that the data party determines the dependency relationship between the main bitstream track data and the knowledge image track data according to the dependency identifier in the signaling file, and sequentially obtains the knowledge image track data and the main bitstream track data from the data source side according to the dependency relationship.

[0007] According to one aspect of the embodiments of the present application, a device for processing track data in a multimedia resource is provided, including: a receiving unit configured to receive a signaling file corresponding to the multimedia resource, where the signaling file contains descriptors respectively corresponding to multiple track data of the multimedia resource, the multiple track data including main bitstream track data corresponding to the main bitstream and knowledge image track data corresponding to the knowledge image bitstream, and the dependency identifier included in the descriptor corresponding to the main bitstream track data points to the descriptor corresponding to the knowledge image track data; an analysis unit configured to analyze the signaling file and determine the dependency relationship between the main bitstream track data and the knowledge image track data according to the dependency identifier; an obtaining unit configured to sequentially obtain the knowledge image track data and the main bitstream track data from the data source side according to the dependency relationship.

[0008] In some embodiments of the present application, based on the foregoing solution, the descriptor corresponding to the knowledge image track data contains first element information, and the first element information is used to indicate that the descriptor containing the first element information is the descriptor corresponding to the knowledge image track data.

[0009] In some embodiments of the present application, based on the foregoing solution, at least two knowledge image track data are included in the multiple track data, and the descriptor corresponding to each knowledge image track data contains second element information, and the second element information is used to indicate the track group where the knowledge image track data is located.

[0010] In some embodiments of the present application, based on the foregoing solution, the descriptor corresponding to each knowledge image track data contains third element information, and the value of the third element information is used to indicate whether the knowledge image track data is relied on by multiple main bitstream track data.

[0011] In some embodiments of the present application, based on the foregoing solution, if there is target knowledge image track data among the at least two knowledge image track data that is relied on by multiple primary bitstream track data, the descriptor corresponding to the target knowledge image track data further includes fourth element information, and the fourth element information is used to indicate the frame rate of the specified primary bitstream track data among the multiple primary bitstream track data.

[0012] In some embodiments of the present application, based on the foregoing solution, the descriptor corresponding to each knowledge image track data further includes a sample index identifier, and the sample index identifier is used to indicate the sample index number range in the primary bitstream track data for indexing the knowledge image track data.

[0013] In some embodiments of the present application, based on the foregoing solution, the sample index identifier includes fifth element information and sixth element information, the value of the fifth element information indicates the minimum sample index number in the primary bitstream track data for indexing the knowledge image track data, and the value of the sixth element information indicates the maximum sample index number in the primary bitstream track data for indexing the knowledge image track data.

[0014] In some embodiments of the present application, based on the foregoing solution, the primary bitstream track data includes an index identifier, and the index identifier is used to indicate the knowledge image track data relied on by the primary bitstream track data or to indicate the knowledge image track group relied on by the primary bitstream track data.

[0015] In some embodiments of the present application, based on the foregoing solution, the primary bitstream track data includes a track reference type data box, and the track reference type data box includes a reference type field, and the reference type field is used to represent the index identifier.

[0016] In some embodiments of the present application, based on the foregoing solution, the primary bitstream track data includes a track reference data box, and the track reference data box includes the track reference type data box.

[0017] In some embodiments of the present application, based on the foregoing solution, the multiple track data includes at least two knowledge image track data, and each knowledge image track data includes a track group identifier, and the track group identifier is used to indicate the track group where the knowledge image track data is located.

[0018] In some embodiments of the present application, based on the foregoing solution, each piece of knowledge image track data further includes first field information for indicating whether the knowledge image track data is relied on by multiple primary bitstream track data; if the first field information indicates that the knowledge image track data is relied on by 1 primary bitstream track data, the knowledge image track data further includes a field indicating the minimum sample index number in the primary bitstream track data for indexing the knowledge image track data, and a field indicating the maximum sample index number in the primary bitstream track data for indexing the knowledge image track data.

[0019] In some embodiments of the present application, based on the foregoing solution, if the first field information indicates that the knowledge image track data is relied on by multiple primary bitstream track data, the knowledge image track data further includes fields respectively indicating the minimum sample index number in each primary bitstream track data for indexing the knowledge image track data, fields respectively indicating the maximum sample index number in the primary bitstream track data for indexing the knowledge image track data, and a field for indicating the frame rate of each primary bitstream track data among the multiple primary bitstream track data.

[0020] In some embodiments of the present application, based on the foregoing solution, each piece of knowledge image track data further includes first field information for indicating whether the knowledge image track data is relied on by multiple primary bitstream track data; if the first field information indicates that the knowledge image track data is relied on by 1 primary bitstream track data, the knowledge image track data further includes a field indicating the number of sample groups in the primary bitstream track data for indexing the knowledge image track data, and a field indicating the sample group index number in the primary bitstream track data for indexing the knowledge image track data.

[0021] In some embodiments of the present application, based on the foregoing solution, if the first field information indicates that the knowledge image track data is relied on by multiple primary bitstream track data, the knowledge image track data further includes fields respectively indicating the number of sample groups in the primary bitstream track data for indexing the knowledge image track data, fields respectively indicating the sample group index number in the primary bitstream track data for indexing the knowledge image track data, and a field for indicating the frame rate of each primary bitstream track data among the multiple primary bitstream track data.

[0022] In some embodiments of the present application, based on the foregoing solution, the processing device for track data in the multimedia resource further includes: a decoding unit configured to determine the decoding order according to the dependency relationship; and perform decoding processing on the knowledge image track data and the primary bitstream track data in sequence according to the decoding order to obtain the multimedia resource.

[0023] In some embodiments of the present application, based on the foregoing solution, the decoding unit is configured to: decode the main bitstream track data; when the sample index number interval of the knowledge image track data that needs to be referred to in the decoded main bitstream track data is obtained, determine the knowledge image track data that needs to be referred to from multiple knowledge image track data according to the sample index number interval; and decode the knowledge image track data that needs to be referred to.

[0024] According to one aspect of the embodiments of the present application, there is provided an apparatus for processing track data in a multimedia resource, including: a generating unit configured to generate a signaling file corresponding to the multimedia resource, where the signaling file includes descriptors respectively corresponding to multiple track data of the multimedia resource, the multiple track data including main bitstream track data corresponding to the main bitstream and knowledge image track data corresponding to the knowledge image bitstream, and a dependency identifier included in the descriptor corresponding to the main bitstream track data points to the descriptor corresponding to the knowledge image track data; a sending unit configured to send the signaling file to a data receiver, so that the data party determines the dependency relationship between the main bitstream track data and the knowledge image track data according to the dependency identifier in the signaling file, and sequentially obtains the knowledge image track data and the main bitstream track data from the data source side according to the dependency relationship.

[0025] In some embodiments of the present application, based on the foregoing solution, the generating unit is further configured to: before generating the signaling file corresponding to the multimedia resource, generate the main bitstream track data corresponding to the main bitstream and the knowledge image track data corresponding to the knowledge image bitstream, where an index identifier is included in the main bitstream track data, and the index identifier is used to indicate the knowledge image track data on which the main bitstream track data depends.

[0026] According to one aspect of the embodiments of the present application, there is provided a computer-readable medium, on which a computer program is stored, and when the computer program is executed by a processor, the method for processing track data in a multimedia resource as described in the above embodiments is implemented.

[0027] According to one aspect of the embodiments of the present application, there is provided an electronic device, including: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method for processing track data in a multimedia resource as described in the above embodiments.

[0028] According to one aspect of the embodiments of the present application, a computer program product or a computer program is provided. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method for processing track data in multimedia resources provided in the above various alternative embodiments.

[0029] In the technical solutions provided in some embodiments of the present application, by receiving a signaling file corresponding to a multimedia resource, determining the dependency relationship between the main bitstream track data and the knowledge image track data according to the descriptors respectively corresponding to multiple track data included in the signaling file and the dependency identifiers included in the descriptor corresponding to the main bitstream track data, and then sequentially obtaining the knowledge image track data and the main bitstream track data from the data source side according to the dependency relationship, the data receiver can obtain the association relationship between each track data in advance according to the signaling file, so as to decide whether to obtain the knowledge image track data and request what kind of knowledge image track data. On the premise of ensuring reasonable allocation of network and CPU resources, it avoids unnecessary delays caused by temporarily obtaining the knowledge image track data, which is beneficial to improving the user experience.

[0030] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. Brief Description of the Drawings

[0031] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts. In the drawings:

[0032] Figure 1 A schematic diagram showing an exemplary system architecture to which the technical solutions of the embodiments of the present application can be applied;

[0033] Figure 2 A schematic diagram showing the placement of a video encoding device and a video decoding device in a streaming system;

[0034] Figure 3 A basic flowchart showing a video encoder;

[0035] Figure 4 A flowchart showing the overall transmission of a video file according to an embodiment of the present application;

[0036] Figure 5Shows a schematic diagram of encoding a video sequence to generate a main bitstream and a knowledge image bitstream;

[0037] Figure 6 Shows a flowchart of a method for processing track data in a multimedia resource according to an embodiment of the present application;

[0038] Figure 7 Shows a flowchart of a method for processing track data in a multimedia resource according to an embodiment of the present application;

[0039] Figure 8 Shows a flowchart of a method for processing track data in a multimedia resource according to an embodiment of the present application;

[0040] Figure 9 Shows a block diagram of an apparatus for processing track data in a multimedia resource according to an embodiment of the present application;

[0041] Figure 10 Shows a block diagram of an apparatus for processing track data in a multimedia resource according to an embodiment of the present application;

[0042] Figure 11 Shows a schematic structural diagram of a computer system of an electronic device suitable for implementing the embodiments of the present application. Detailed implementation manners

[0043] Now, example embodiments will be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more complete and comprehensive, and will fully convey the concept of the example embodiments to those skilled in the art.

[0044] In addition, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present application. However, those skilled in the art will realize that the technical solutions of the present application can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the present application.

[0045] The block diagrams shown in the drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0046] The flowcharts shown in the accompanying drawings are merely illustrative and not necessarily include all contents and operations / steps, nor are they necessarily executed in the described order. For example, some operations / steps can be decomposed, while some operations / steps can be combined or partially combined. Therefore, the actual execution order may change according to the actual situation.

[0047] It should be noted that: "a plurality of" mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0048] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solution of the embodiment of the present application can be applied is shown.

[0049] As Figure 1 shown, the system architecture 100 includes a plurality of terminal devices, and the terminal devices can communicate with each other through, for example, the network 150. For example, the system architecture 100 can include a first terminal device 110 and a second terminal device 120 interconnected through the network 150. In Figure 1 the embodiment, the first terminal device 110 and the second terminal device 120 perform unidirectional data transmission.

[0050] For example, the first terminal device 110 can encode video data (such as a video picture stream collected by the terminal device 110) for transmission to the second terminal device 120 through the network 150. The encoded video data is transmitted in the form of one or more encoded video bitstreams. The second terminal device 120 can receive the encoded video data from the network 150, decode the encoded video data to restore the video data, and display the video pictures according to the restored video data.

[0051] In an embodiment of the present application, the system architecture 100 can include a third terminal device 130 and a fourth terminal device 140 that perform bidirectional transmission of encoded video data. The bidirectional transmission can occur, for example, during a video conference. For bidirectional data transmission, each of the third terminal device 130 and the fourth terminal device 140 can encode video data (such as a video picture stream collected by the terminal device) for transmission to the other terminal device among the third terminal device 130 and the fourth terminal device 140 through the network 150. Each of the third terminal device 130 and the fourth terminal device 140 can also receive the encoded video data transmitted by the other terminal device among the third terminal device 130 and the fourth terminal device 140, and can decode the encoded video data to restore the video data, and can display the video pictures on an accessible display device according to the restored video data.

[0052] In Figure 1 the embodiment, the first terminal device 110, the second terminal device 120, the third terminal device 130, and the fourth terminal device 140 may be a server, a personal computer, and a smart phone, but the principles disclosed in this application are not limited thereto. The embodiments disclosed in this application are applicable to laptop computers, tablet computers, media players, and / or dedicated video conferencing devices. The network 150 represents any number of networks for transmitting encoded video data between the first terminal device 110, the second terminal device 120, the third terminal device 130, and the fourth terminal device 140, including, for example, wired and / or wireless communication networks. The communication network 150 may exchange data in circuit-switched and / or packet-switched channels. The network may include a telecommunications network, a local area network, a wide area network, and / or the Internet. For the purposes of this application, unless otherwise explained below, the architecture and topology of the network 150 may be immaterial to the operations disclosed in this application.

[0053] In one embodiment of this application, Figure 2 shows the placement of a video encoding device and a video decoding device in a streaming environment. The subject matter disclosed in this application is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV (television), storing compressed video on digital media including CD, DVD, memory stick, etc.

[0054] The streaming system may include an acquisition subsystem 213, and the acquisition subsystem 213 may include a video source 201 such as a digital camera. The video source creates an uncompressed video picture stream 202. In the embodiment, the video picture stream 202 includes samples taken by the digital camera. Compared with the encoded video data 204 (or encoded video bitstream 204), the video picture stream 202 is depicted as a thick line to emphasize the high data volume of the video picture stream. The video picture stream 202 may be processed by an electronic device 220, and the electronic device 220 includes a video encoding device 203 coupled to the video source 201. The video encoding device 203 may include hardware, software, or a combination of hardware and software to implement or carry out aspects of the disclosed subject matter described in more detail below. Compared with the video picture stream 202, the encoded video data 204 (or encoded video bitstream 204) is depicted as a thin line to emphasize the lower data volume of the encoded video data 204 (or encoded video bitstream 204), which may be stored on the streaming server 205 for future use. One or more streaming client subsystems, such as Figure 2The client subsystems 206 and 208 therein can access the streaming server 205 to retrieve copies 207 and 209 of the encoded video data 204. The client subsystem 206 can include, for example, the video decoding device 210 in the electronic device 230. The video decoding device 210 decodes the incoming copy 207 of the encoded video data and generates an output video picture stream 211 that can be presented on a display 212 (such as a display screen) or another rendering device. In some streaming systems, the encoded video data 204, video data 207, and video data 209 (such as video bitstreams) can be encoded according to certain video coding / compression standards. Examples of such standards include ITU-T H.265. In an embodiment, a video coding standard under development is informally referred to as Versatile Video Coding (VVC), and this application can be used in the context of the VVC standard.

[0055] It should be noted that the electronic devices 220 and 230 may include other components not shown in the figure. For example, the electronic device 220 may include a video decoding device, and the electronic device 230 may further include a video encoding device.

[0056] In an embodiment of the present application, taking the international video coding standards HEVC (High Efficiency Video Coding), VVC (Versatile Video Coding), and the Chinese national video coding standard AVS as examples, after inputting a video frame image, according to a block size, the video frame image is divided into a number of non-overlapping processing units, and each processing unit will perform similar compression operations. This processing unit is called a CTU (Coding Tree Unit), or an LCU (Largest Coding Unit). The CTU can be further divided more finely to obtain one or more basic coding units CU, and the CU is the most basic element in a coding link. The following introduces some concepts when encoding the CU:

[0057] Predictive Coding: Predictive coding includes methods such as intra-frame prediction and inter-frame prediction. After the original video signal is predicted by the selected reconstructed video signal, a residual video signal is obtained. The encoding end needs to decide which predictive coding mode to select for the current CU and inform the decoding end. Among them, intra-frame prediction means that the predicted signal comes from the area that has been encoded and reconstructed within the same image; inter-frame prediction means that the predicted signal comes from other images (referred to as reference images) that have been encoded and are different from the current image.

[0058] Transform & Quantization: After the residual video signal undergoes transform operations such as DFT (Discrete Fourier Transform) and DCT (Discrete Cosine Transform), the signal is converted into the transform domain, and the resulting coefficients are called transform coefficients. The transform coefficients are further subjected to a lossy quantization operation, losing some information, which makes the quantized signal conducive to compressed representation. In some video coding standards, there may be more than one transform method to choose from. Therefore, the encoder also needs to select one of the transform methods for the current CU and inform the decoder. The fineness of quantization is usually determined by the Quantization Parameter (QP). A larger QP value means that coefficients within a larger value range will be quantized to the same output, usually resulting in greater distortion and a lower bitrate. Conversely, a smaller QP value means that coefficients within a smaller value range will be quantized to the same output, usually resulting in less distortion and a corresponding higher bitrate.

[0059] Entropy Coding or Statistical Coding: The quantized transform domain signal will be statistically compressed encoded according to the frequencies of each value, and finally, a binary (0 or 1) compressed bitstream is output. At the same time, other information generated during encoding, such as the selected coding mode, motion vector data, etc., also needs to be entropy encoded to reduce the bitrate. Statistical coding is a lossless coding method that can effectively reduce the bitrate required to represent the same signal. Common statistical coding methods include Variable Length Coding (VLC) or Context-Adaptive Binary Arithmetic Coding (CABAC).

[0060] Loop Filtering: The signal that has been transformed and quantized will undergo operations such as inverse quantization, inverse transformation, and prediction compensation to obtain the reconstructed image. Compared with the original image, due to the impact of quantization, some information in the reconstructed image is different from the original image, that is, the reconstructed image will produce distortion. Therefore, filtering operations can be performed on the reconstructed image, such as deblocking filter (abbreviated as DB), SAO (Sample Adaptive Offset), or ALF (Adaptive Loop Filter) and other filters, which can effectively reduce the degree of distortion caused by quantization. Since these filtered reconstructed images will be used as references for subsequent encoded images to predict future image signals, the above filtering operations are also called loop filtering, that is, filtering operations within the encoding loop.

[0061] In an embodiment of the present application, Figure 3 A basic flowchart of a video encoder is shown, and in this process, intra-frame prediction is taken as an example for illustration. Among them, the original image signal s k [x, y] and the predicted image signal perform a difference operation to obtain the residual signal u k [x, y], and the residual signal u k [x, y] undergoes transformation and quantization processing to obtain quantization coefficients. On the one hand, the quantization coefficients are entropy-coded to obtain the encoded bitstream, and on the other hand, they are inverse-quantized and inverse-transformed to obtain the reconstructed residual signal u' k [x, y], the predicted image signal and the reconstructed residual signal u' k [x, y] are superimposed to generate the image signal The image signal is input to the intra-frame mode decision module and the intra-frame prediction module for intra-frame prediction processing on the one hand, and on the other hand, the reconstructed image signal s' k [x, y] is output through loop filtering. The reconstructed image signal s' k [x, y] can be used as a reference image for the next frame for motion estimation and motion compensation prediction. Then, based on the result of motion compensation prediction s' r [x + m x , y + m y and the intra-frame prediction result the predicted image signal for the next frame is obtained and the above process is continued until the encoding is completed.

[0062] Based on the above encoding process, at the decoding end, for each CU, after obtaining the compressed bitstream (i.e., the bitstream), entropy decoding is performed to obtain various mode information and quantization coefficients. Then, the quantization coefficients are processed through inverse quantization and inverse transformation to obtain the residual signal. On the other hand, according to the known encoding mode information, the prediction signal corresponding to this CU can be obtained. Then, after adding the residual signal and the prediction signal, the reconstructed signal can be obtained. The reconstructed signal is further processed through operations such as loop filtering to generate the final output signal.

[0063] Briefly speaking, the overall transmission process of a video file is as Figure 4 shown. The video file is obtained through video capture, and then after video encoding and video file encapsulation processing, the video file is transmitted to the receiving party. After the receiving party receives the video file, it performs decapsulation on the video file. After decapsulation, time-frequency decoding processing is performed, and finally the decoded video is presented.

[0064] In the AVS3 video coding and decoding technology, in order to improve the video compression efficiency, the concept of knowledge image is proposed. As Figure 5 shown, when encoding a video sequence, a main bitstream and a library bitstream can be generated, and when decoding the image frames in the main bitstream, the image frames in the library bitstream can be referred to.

[0065] Meanwhile, in related technologies, the signaling of the segment dependency descriptor is also proposed. Specifically, an EssentialProperty element with an @schemeIdUri attribute of "urn:avs:ims:2018:ds" (dependent segment) represents a segment dependency descriptor. At least one segment dependency descriptor is specified at the presentation layer, and should not be specified at the MPD (media presentation description) layer and the adaptation set layer. The segment dependency descriptor indicates that each segment in each representation has a non-temporal dependency relationship with other segments (which may be segments in the same representation or segments in different representations). The identifier (URL or indicator) of the other segments on which it depends and the image number in the segment for the compression layer should be included in this descriptor.

[0066] Although the related technologies indicate the information of the samples of the knowledge images referred to by certain samples in the main bitstream, and also give the dependency relationships between the segment levels at the signaling level. However, these sample-level dependencies and associations can only be obtained by the decoder when it parses specific sample segments. If the data receiver has not requested or decoded the corresponding knowledge image track before, it needs to temporarily request or decode the knowledge image track, which will bring unnecessary delays and affect the user experience.

[0067] Therefore, the technical solution of the embodiment of the present application proposes a new processing solution for track data in multimedia resources, enabling the data receiver to obtain the association relationships between various track data in advance according to the signaling file, so as to decide whether to obtain the knowledge image track data and what kind of knowledge image track data to request. On the premise of ensuring reasonable allocation of network and CPU resources, it avoids the unnecessary delays caused by temporarily obtaining the knowledge image track data, which is beneficial to improving the user experience.

[0068] The following details the implementation details of the technical solution of the embodiment of the present application:

[0069] Figure 6 The flowchart of the method for processing track data in multimedia resources according to an embodiment of the present application is shown. The method for processing track data in multimedia resources can be executed by a media playback device, which can be a smart phone, a tablet computer, etc. Referring to Figure 6 As shown, the method for processing track data in multimedia resources at least includes steps S610 to S630, which are introduced in detail as follows:

[0070] In step S610, a signaling file corresponding to the multimedia resource is received. The signaling file contains descriptors corresponding to multiple track data of the multimedia resource. The multiple track data include main bitstream track data corresponding to the main bitstream and knowledge image track data corresponding to the knowledge image bitstream. The dependency item identifier contained in the descriptor corresponding to the main bitstream track data points to the descriptor corresponding to the knowledge image track data.

[0071] It should be noted that: The multimedia resource contains specific media resource data, such as the specific content of the introduction video of item A (video pictures, introduction audio, etc.). The signaling file corresponding to the multimedia resource can be a DASH (Dynamic Adaptive Streaming over HTTP) signaling file.

[0072] Optionally, among the multiple track data of the multimedia resource, there can be one knowledge image track data or multiple knowledge image track data.

[0073] In an embodiment of the present application, the descriptor corresponding to the knowledge image track data may include first element information, and the first element information is used to indicate that the descriptor containing the first element information is the descriptor corresponding to the knowledge image track data.

[0074] In an embodiment of the present application, if at least two knowledge image track data are included in multiple track data of a multimedia resource, then the descriptor corresponding to each knowledge image track data may include second element information, and the second element information is used to indicate the track group where the knowledge image track data is located.

[0075] In an embodiment of the present application, if at least two knowledge image track data are included in multiple track data of a multimedia resource, then the descriptor corresponding to each knowledge image track data includes third element information, and the value of the third element information is used to indicate whether the knowledge image track data is relied on by multiple primary bitstream track data. For example, if the value of the third element information is 1, it means that the knowledge image track data is relied on by multiple primary bitstream tracks; if the value of the third element information is 0, it means that the knowledge image track data is relied on by 1 primary bitstream track.

[0076] In an embodiment of the present application, if at least two knowledge image track data are included in multiple track data of a multimedia resource, and there is target knowledge image track data that is relied on by multiple primary bitstream track data, then the descriptor corresponding to the target knowledge image track data further includes fourth element information, and the fourth element information is used to indicate the frame rate of the specified primary bitstream track data among the multiple primary bitstream track data. Optionally, the specified primary bitstream track data may be the multiple primary bitstream track data, or may also be some of the primary bitstream track data.

[0077] In an embodiment of the present application, if at least two knowledge image track data are included in multiple track data of a multimedia resource, then the descriptor corresponding to each knowledge image track data further includes a sample index identifier, and the sample index identifier is used to indicate the sample index number range in the primary bitstream track data for indexing the knowledge image track data. Optionally, the sample index identifier includes fifth element information and sixth element information, the value of the fifth element information indicates the minimum sample index number in the primary bitstream track data for indexing the knowledge image track data, and the value of the sixth element information indicates the maximum sample index number in the primary bitstream track data for indexing the knowledge image track data.

[0078] In step S620, parse the signaling file, and determine the dependency relationship between the primary bitstream track data and the knowledge image track data according to the dependency identifier.

[0079] In step S630, according to the dependency relationship between the main stream track data and the knowledge image track data, the knowledge image track data and the main stream track data are sequentially acquired from the data source side.

[0080] In an embodiment of the present application, since there is a dependency relationship between the main stream track data and the knowledge image track data, after the knowledge image track data is acquired, the main stream track data is acquired from the data source side.

[0081] Optionally, if there are multiple knowledge image track data, the knowledge image track data that the main stream track data needs to refer to first can be acquired first, and then during the process of decoding the main stream track data, if it is decoded to a position where other knowledge image track data needs to be referred to, other knowledge image track data can be acquired. Of course, it is also possible to perform decoding processing after the main stream track data and all knowledge image track data are acquired.

[0082] In an embodiment of the present application, the main stream track data may include an index identifier, and the index identifier is used to indicate the knowledge image track data on which the main stream track data depends or the knowledge image track group on which the main stream track data depends.

[0083] Optionally, the main stream track data includes a track reference type data box, and the track reference type data box includes a reference type field, and the reference type field is used to represent the index identifier. Based on this, the value of the reference type field is used to indicate the knowledge image track data on which the main stream track data depends or the knowledge image track group on which it depends.

[0084] In an embodiment of the present application, the main stream track data may include a track reference data box. In this case, the track reference data box includes the track reference type data box.

[0085] In an embodiment of the present application, among the multiple track data of the multimedia resource, there may be at least two knowledge image track data, and each knowledge image track data includes a track group identifier, and the track group identifier is used to indicate the track group where the knowledge image track data is located.

[0086] In an embodiment of the present application, after the data receiver sequentially acquires the knowledge image track data and the main stream track data from the data source side according to the dependency relationship, it can determine the decoding order according to the dependency relationship; then according to the determined decoding order, the knowledge image track data and the main stream track data are sequentially decoded to obtain the multimedia resource.

[0087] In an embodiment of the present application, the main bitstream track data may be decoded first. When decoding to the sample index number interval in the main bitstream track data that needs to refer to the knowledge image track data, according to this sample index number interval, the knowledge image track data to be referred to is determined from multiple knowledge image track data, and then the knowledge image track data to be referred to is decoded. Optionally, the decoding process may be performed after obtaining the main bitstream track data and all the knowledge image track data; or the knowledge image track data that the main bitstream track data needs to refer to first may be obtained first, and then during the process of decoding the main bitstream track data, other knowledge image track data may be obtained when decoding to the position where other knowledge image track data needs to be referred to.

[0088] In an embodiment of the present application, if the media resource includes at least two knowledge image track data, then each knowledge image track data further includes first field information for indicating whether the knowledge image track data is relied on by multiple main bitstream track data. Optionally, the first field information may be, for example, multi_main_bitstream. If the value of multi_main_bitstream is 1, it indicates that the knowledge image track data is relied on by multiple main bitstream track data; if the value of multi_main_bitstream is 0, it indicates that the knowledge image track data is relied on by 1 main bitstream track data.

[0089] In an embodiment of the present application, if the first field information indicates that the knowledge image track data is relied on by 1 main bitstream track data, then the knowledge image track data further includes a field for indicating the minimum sample index number in the main bitstream track data for indexing the knowledge image track data, and a field for indicating the maximum sample index number in the main bitstream track data for indexing the knowledge image track data. Optionally, the field for indicating the minimum sample index number may be sample_number_min, and the field for indicating the maximum sample index number may be sample_number_max. Among them, the segment between the minimum sample index number and the maximum sample index number in the main bitstream track data needs to rely on the knowledge image track data.

[0090] In an embodiment of the present application, if the first field information indicates that the knowledge image track data depends on multiple primary bitstream track data, the knowledge image track data further includes fields respectively indicating the minimum sample index number used to index the knowledge image track data in each primary bitstream track data, fields respectively indicating the maximum sample index number used to index the knowledge image track data in the primary bitstream track data, and a field used to indicate the frame rate of each primary bitstream track data among the multiple primary bitstream track data. Similarly, the field indicating the minimum sample index number may be sample_number_min, and the field indicating the maximum sample index number may be sample_number_max.

[0091] In an embodiment of the present application, if the first field information indicates that the knowledge image track data depends on 1 primary bitstream track data, the knowledge image track data further includes a field indicating the number of sample groups used to index the knowledge image track data in the primary bitstream track data, and a field indicating the sample group index number used to index the knowledge image track data in the primary bitstream track data. Optionally, the field indicating the number of sample groups may be num_sample_groups, and the field indicating the sample group index number may be group_description_index.

[0092] In an embodiment of the present application, if the first field information indicates that the knowledge image track data depends on multiple primary bitstream track data, the knowledge image track data further includes fields respectively indicating the number of sample groups used to index the knowledge image track data in the primary bitstream track data, fields respectively indicating the sample group index number used to index the knowledge image track data in the primary bitstream track data, and a field used to indicate the frame rate of each primary bitstream track data among the multiple primary bitstream track data. Similarly, the field indicating the number of sample groups may be num_sample_groups, and the field indicating the sample group index number may be group_description_index.

[0093] Figure 6 The technical solution of the embodiment of the present application is described from the perspective of the receiver of the media resource. The following combines Figure 7 To further illustrate the implementation details of the embodiment of the present application from the data source side:

[0094] Figure 7 The flowchart of the processing method of the track data in the multimedia resource according to an embodiment of the present application is shown. The processing method of the track data in the multimedia resource may be executed by a media generation device, and the media generation device may be a smart phone, a tablet computer, etc. Refer to Figure 7As shown, the method for processing track data in the multimedia resource at least includes steps S710 to S720, which are introduced in detail as follows:

[0095] In step S710, a signaling file corresponding to the multimedia resource is generated. The signaling file contains descriptors respectively corresponding to multiple track data of the multimedia resource. The multiple track data include main bitstream track data corresponding to the main bitstream and knowledge image track data corresponding to the knowledge image bitstream. The dependency identifier contained in the descriptor corresponding to the main bitstream track data points to the descriptor corresponding to the knowledge image track data.

[0096] It should be noted that: The multimedia resource contains specific media resource data, such as the specific content of the introduction video of item A (video pictures, introduction audio, etc.). The signaling file corresponding to the multimedia resource can be a DASH signaling file.

[0097] Optionally, among the multiple track data of the multimedia resource, there can be one knowledge image track data or multiple knowledge image track data.

[0098] In an embodiment of the present application, the descriptor corresponding to the knowledge image track data may contain first element information, and the first element information is used to indicate that the descriptor containing the first element information is the descriptor corresponding to the knowledge image track data.

[0099] In an embodiment of the present application, if there are at least two knowledge image track data among the multiple track data of the multimedia resource, then the descriptors corresponding to the respective knowledge image track data may contain second element information, and the second element information is used to indicate the track group where the knowledge image track data is located.

[0100] In an embodiment of the present application, if there are at least two knowledge image track data among the multiple track data of the multimedia resource, then the descriptors corresponding to the respective knowledge image track data contain third element information, and the value of the third element information is used to indicate whether the knowledge image track data is relied on by multiple main bitstream track data. For example, if the value of the third element information is 1, it means that the knowledge image track data is relied on by multiple main bitstream tracks; if the value of the third element information is 0, it means that the knowledge image track data is relied on by 1 main bitstream track.

[0101] In an embodiment of the present application, if there are at least two knowledge image track data in the multiple track data of the multimedia resource, and there is target knowledge image track data that is relied on by multiple primary bitstream track data, then the descriptor corresponding to the target knowledge image track data further includes fourth element information, and the fourth element information is used to indicate the frame rate of the specified primary bitstream track data among the multiple primary bitstream track data. Optionally, the specified primary bitstream track data may be the multiple primary bitstream track data, or may also be some of the primary bitstream track data.

[0102] In an embodiment of the present application, if there are at least two knowledge image track data in the multiple track data of the multimedia resource, then the descriptor corresponding to each knowledge image track data further includes a sample index identifier, and the sample index identifier is used to indicate the sample index number range in the primary bitstream track data for indexing the knowledge image track data. Optionally, the sample index identifier includes fifth element information and sixth element information, the value of the fifth element information indicates the minimum sample index number in the primary bitstream track data for indexing the knowledge image track data, and the value of the sixth element information indicates the maximum sample index number in the primary bitstream track data for indexing the knowledge image track data.

[0103] In step S720, the signaling file is sent to the data receiver, so that the data party determines the dependency relationship between the primary bitstream track data and the knowledge image track data according to the dependency identifier in the signaling file, and sequentially obtains the knowledge image track data and the primary bitstream track data from the data source side according to the dependency relationship.

[0104] In an embodiment of the present application, before generating the signaling file corresponding to the multimedia resource, the data source side may also generate primary bitstream track data corresponding to the primary bitstream and knowledge image track data corresponding to the knowledge image bitstream. The primary bitstream track data includes an index identifier, and the index identifier is used to indicate the knowledge image track data relied on by the primary bitstream track data or the knowledge image track group relied on by the primary bitstream track data.

[0105] Optionally, the primary bitstream track data includes a track reference type data box, and the track reference type data box includes a reference type field, and the reference type field is used to represent the index identifier. Based on this, the value of the reference type field is used to indicate the knowledge image track data relied on by the primary bitstream track data or the knowledge image track group relied on.

[0106] In an embodiment of the present application, the primary bitstream track data may include a track reference data box. In this case, the track reference data box includes the track reference type data box.

[0107] In one embodiment of the present application, at least two knowledge image track data may be included in the multiple track data of the multimedia resource, and each knowledge image track data includes a track group identifier, which is used to indicate the track group where the knowledge image track data is located.

[0108] Optionally, other relevant content of the knowledge image track group data may refer to the technical solutions of the foregoing embodiments and will not be elaborated herein.

[0109] The technical solutions of the embodiments of the present application have been described above from the perspective of the data receiver and the data source side. The following Figure 8 further elaborates on the implementation details of the embodiments of the present application as a whole:

[0110] As Figure 8 shown, taking the server as the data source side and the client as the data receiver as an example, it may specifically include the following steps S801 to S807:

[0111] In step S801, the server generates a bitstream.

[0112] In one embodiment of the present application, the server may generate a main bitstream and one or more knowledge image bitstreams during the video encoding process.

[0113] In step S802, the server encapsulates and generates track data.

[0114] In one embodiment of the present application, during the video file encapsulation process, the server may encapsulate the main bitstream into a separate file track, and each knowledge image bitstream into a separate file track, and associate the main bitstream track and the knowledge image track through the index relationship between the tracks according to the reference relationship between the main bitstream and the knowledge image bitstream during decoding. If the main bitstream track needs to refer to multiple knowledge image tracks, then these knowledge image tracks can be associated through a track group, and different knowledge image tracks can be distinguished within the track group through sample index range information, description information, etc.

[0115] Optionally, one main bitstream track may be associated with one knowledge image track or one knowledge image track group. Multiple main bitstream tracks (generally multiple tracks of the same content with different frame rates) may also be associated with the same knowledge image track or knowledge image track group.

[0116] In step S803, the server generates DASH signaling.

[0117] In an embodiment of the present application, in the signaling generation stage, the server can specially mark the media resources corresponding to the knowledge image bitstream and indicate the dependency relationship between the main bitstream media resources and the knowledge image bitstream media resources. If the main bitstream media resources need to refer to multiple knowledge image media resources, these knowledge image media resources can be associated with each other and distinguished by sample index range information, description information, etc.

[0118] In step S804, the server sends a DASH signaling to the client.

[0119] In step S805, the client requests a media file from the server according to the DASH signaling.

[0120] In an embodiment of the present application, the client determines whether the required media resources depend on the media resources corresponding to the knowledge image bitstream according to the signaling file. If so, it preferentially requests the media resources corresponding to the knowledge image bitstream. If it depends on multiple media resources corresponding to the knowledge image bitstream, it requests the corresponding media resources according to the sample index range information to which the currently presented frame belongs.

[0121] In step S806, the server transmits the media file to the client.

[0122] In step S807, the client unpacks the media file and presents the corresponding media resources.

[0123] In an embodiment of the present application, after the client requests the corresponding media resources, it can preferentially decode the track data corresponding to the knowledge image bitstream according to the index relationship between the file tracks. If there is a knowledge image track group, that is, the track data corresponding to the main bitstream depends on the track data corresponding to multiple knowledge image bitstreams, it decodes the corresponding track data according to the sample index range information to which the currently presented frame belongs.

[0124] To implement the technical solutions of the foregoing embodiments, some descriptive field information is added in the embodiments of the present application. The following takes the form of extending the ISOBMFF data box and DASH MPD signaling as an example to define relevant fields to support the AVS3 knowledge image technology, specifically as follows:

[0125] 1. Define the index relationship between the knowledge image track and the main bitstream track:

[0126] In an embodiment of the present application, the primary bitstream track can be indexed to the knowledge image track on which its decoding depends through the track index data box. The corresponding TrackReferenceTypeBoxes (track reference type data box) should be added to the TrackReferenceBox (track reference data box) of the primary bitstream track. The knowledge image track or knowledge image track group indexed by the current primary bitstream track is indicated by track_IDs in the TrackReferenceTypeBoxes data box.

[0127] Specifically, the index between the primary bitstream track and the knowledge image track is identified by the corresponding reference_type (reference type) index type in the TrackReferenceTypeBoxes. The definition of this type field is as follows:

[0128] 'a3lr': The indexed track is the knowledge image track corresponding to the current track.

[0129] Optionally, a primary bitstream track can be indexed to a knowledge image track or a knowledge image track group through 'a3lr'; multiple primary bitstream tracks can be indexed to a knowledge image track or a knowledge image track group through 'a3lr'.

[0130] 2. Define the knowledge image track group:

[0131] In an embodiment of the present application, if a primary bitstream track needs to refer to multiple knowledge image tracks, then these knowledge image tracks should be associated through a knowledge image track group. Optionally, one definition of the knowledge image track group is as follows:

[0132]

[0133]

[0134] In the above definition, the knowledge image track group is obtained by extending the track group data box and is identified by the 'a3lg' track group type. Among all the tracks containing the TrackGroupTypeBox of the 'a3lg' type, the tracks with the same group ID belong to the same track group. The semantics of each field in the Avs3LibraryGroupBox are as follows:

[0135] multi_main_bitstream indicates whether the knowledge image track is referred to by multiple primary bitstream tracks. The value of this field being 1 means that the knowledge image track is referred to by multiple primary bitstream tracks; the value of this field being 0 means that the knowledge image track is referred to by only one primary bitstream track. Optionally, the default value of this field is 0.

[0136] sample_number_min indicates the minimum sample index number that indexes the current knowledge image track in the main bitstream track or the main bitstream track with a specific frame rate.

[0137] sample_number_max indicates the maximum sample index number that indexes the current knowledge image track in the main bitstream track or the main bitstream track with a specific frame rate.

[0138] frame_rate: When this knowledge image track is referenced by multiple main bitstream tracks, it indicates the frame rate of a certain track among the multiple main bitstream tracks.

[0139] track_description is a string terminated by a null character, indicating the description information of this knowledge image track. Optionally, sample group information can also be used to distinguish different tracks in the same knowledge image track group. Specifically, another definition of the knowledge image track group is as follows:

[0140]

[0141]

[0142] In the above definition, the knowledge image track group is obtained by extending the track group data box and is identified by the 'a3lg' track group type. Among all the tracks containing the TrackGroupTypeBox of the 'a3lg' type, the tracks with the same group ID belong to the same track group. The semantics of each field in the Avs3LibraryGroupBox are as follows:

[0143] multi_main_bitstream indicates whether this knowledge image track is referenced by multiple main bitstream tracks. When the value of this field is 1, it means this knowledge image track is referenced by multiple main bitstream tracks; when the value of this field is 0, it means this knowledge image track is only referenced by one main bitstream track. Optionally, the default value of this field is 0.

[0144] num_sample_groups indicates the number of LibrarySampleGroupEntry sample groups that index the current knowledge image track in the main bitstream track or the main bitstream track with a specific frame rate.

[0145] group_description_index indicates the index number of the LibrarySampleGroupEntry sample group that indexes the current knowledge image track in the main bitstream track or the main bitstream track with a specific frame rate.

[0146] frame_rate: When this knowledge image track is referenced by multiple main bitstream tracks, it indicates the frame rate of a certain track among the multiple main bitstream tracks.

[0147] The track_description is a null-terminated string indicating the description information of the knowledge graph track.

[0148] 3. In the DASH signaling extension, a knowledge graph descriptor is defined:

[0149] In an embodiment of the present application, the knowledge graph descriptor Avs3Library is a SupplementalProperty element, and its @schemeIdUri attribute is "urn:avs:ims:2018:av3l". This descriptor can exist at the adaptation set level or the representation level. When this descriptor exists at the adaptation set level, it describes all the representations within that adaptation set; when this descriptor exists at the representation level, it describes the corresponding representation. The Avs3Library descriptor indicates the relevant properties of the knowledge graph representation, and the specific properties are shown in Table 1 below:

[0150]

[0151] Table 1

[0152] In Table 1, "0…N" in the "Use" column represents the number (specifically an integer), O represents Optional, and CM represents Conditional Mandatory. "xs" in the "Datatype" column represents the short int type.

[0153] In a specific example, assume that there is media content A and media content B on the server side, and the server encodes them respectively to generate bitstreams. Specifically, for media content A, a main bitstream StreamA and a knowledge graph bitstream StreamAL are generated; for media content B, a main bitstream StreamB is generated.

[0154] After generating the bitstreams, the server encapsulates StreamA as TrackA (Track A), encapsulates StreamAL as TrackAL, and uses a TrackReferenceTypeBox of type 'a3lr' in TrackA to index to TrackAL.

[0155] In addition, the server encapsulates StreamB as TrackB. Since there is no corresponding knowledge image track for TrackB, the TrackB does not need to include a TrackReferenceTypeBox of the 'a3lr' type.

[0156] After encapsulation, for TrackA and TrackAL, the server describes them as 1 representation each (i.e., RA and RAL). The @dependencyId (dependency identifier) attribute of RA should point to RAL, indicating that the consumption of RA depends on RAL, and RAL needs to be described with an Avs3Library descriptor. For TrackB, the server describes it as 1 representation (i.e., RB) without special extension.

[0157] After describing the track data, the server generates DASH signaling based on this and sends the signaling file to the client.

[0158] After receiving the signaling file, the client can determine the dependency relationship between descriptors according to the signaling file. For example, RA depends on RAL and RAL is a knowledge image media resource. Suppose client 1 needs to request the media resource corresponding to RA, and client 2 needs to request the media resource corresponding to RB. Then client 1 needs to first request the media resource corresponding to RAL from the server and then request the media resource corresponding to RA from the server. And client 2 can directly request the media resource corresponding to RB.

[0159] After receiving the media resource corresponding to RAL and the media resource corresponding to RA, client 1 preferentially decodes the media resource corresponding to RAL and then decodes the media resource corresponding to RA. And after receiving the media resource corresponding to RB, client 2 can directly decode the media resource corresponding to RB.

[0160] In the above example, the media content contains a main bitstream and a knowledge image bitstream. The following takes the media content containing a main bitstream and multiple knowledge image bitstreams as an example for further illustration:

[0161] In a specific example of this application, assume that there is media content A on the server, and the server encodes it to generate a main bitstream StreamA and knowledge image bitstreams StreamAL1 and StreamAL2.

[0162] After generating the bitstream, the server encapsulates StreamA as TrackA, StreamAL1 as TrackAL1, and StreamAL2 as TrackAL2. At the same time, TrackAL1 and TrackAL2 are associated as an 'a3lg'-type track group, and the parameters are as follows:

[0163] TrackAL1: {group_id = 100; sample_number_min = 0; sample_number_max = 100}

[0164] TrackAL2: {group_id = 100; sample_number_min = 101; sample_number_max = 200}

[0165] At this time, in TrackA, the 'a3lr'-type TrackReferenceTypeBox is used to index to the corresponding track group (indexed by group_id, and in this example, group_id is 100).

[0166] After encapsulation, for TrackA, TrackAL1, and TrackAL2, the server describes them as 1 representation each (i.e., RA, RAL1, and RAL2). The @dependencyId (dependency identifier) attribute of RA should point to RAL1 and RAL2, indicating that the consumption of RA depends on RAL1 and RAL2. And RAL1 and RAL2 need to be described with an Avs3Library descriptor, specifically as follows:

[0167] RAL1: {group_id = 100; sample_number_min = 0; sample_number_max = 100}

[0168] RAL2: {group_id = 100; sample_number_min = 101; sample_number_max = 200}

[0169] After describing the track data, the server generates DASH signaling based on this and sends the signaling file to the client.

[0170] After receiving the signaling file, the client can determine the dependency relationships between descriptors based on the signaling file. For example, RA depends on RAL1 and RAL2, and RAL1 and RAL2 are knowledge image media resources. RAL1 corresponds to the samples at the front in RA. Suppose client 1 needs to request the media resources corresponding to RA. Then client 1 needs to first request the media resources corresponding to RAL1 and RA from the server. When client 1 consumes RA close to the 101st sample, it requests the media resources corresponding to RAL2 from the server again.

[0171] The technical solution of the above embodiment of the present application aims at the feature of knowledge images in the AVS3 codec standard, and proposes a method for encapsulation and transmission signaling indication at the file track level. Through the technical solution of the embodiment of the present application, the knowledge image track and the main bitstream track can be flexibly associated at the file track level, and this association relationship is indicated by signaling. In the data transmission stage, the client can decide whether to request the knowledge image track and which knowledge image track to request based on this information. Similarly, in the decoding stage, the client can decide the decoding order of different tracks based on this information, and finally reasonably allocate network and CPU resources.

[0172] The device embodiments of the present application are introduced below, which can be used to execute the method for processing track data in multimedia resources in the above embodiments of the present application. For the details not disclosed in the device embodiments of the present application, please refer to the embodiments of the method for processing track data in multimedia resources above of the present application.

[0173] Figure 9 The block diagram of the device for processing track data in multimedia resources according to an embodiment of the present application is shown. The device for processing track data in multimedia resources can be set in a media playback device, and the media playback device can be a smart phone, a tablet computer, etc.

[0174] Refer to Figure 9 As shown, the device 900 for processing track data in multimedia resources according to an embodiment of the present application includes: a receiving unit 902, an analyzing unit 904, and an obtaining unit 906.

[0175] Among them, a receiving unit 902 is configured to receive a signaling file corresponding to a multimedia resource, where the signaling file includes descriptors respectively corresponding to multiple track data of the multimedia resource, the multiple track data includes main bitstream track data corresponding to a main bitstream and knowledge image track data corresponding to a knowledge image bitstream, and a dependency identifier included in the descriptor corresponding to the main bitstream track data points to the descriptor corresponding to the knowledge image track data; a parsing unit 904 is configured to parse the signaling file and determine a dependency relationship between the main bitstream track data and the knowledge image track data according to the dependency identifier; an obtaining unit 906 is configured to sequentially obtain the knowledge image track data and the main bitstream track data from a data source side according to the dependency relationship.

[0176] In some embodiments of the present application, based on the foregoing solution, the descriptor corresponding to the knowledge image track data includes first element information, and the first element information is used to indicate that the descriptor including the first element information is the descriptor corresponding to the knowledge image track data.

[0177] In some embodiments of the present application, based on the foregoing solution, at least two knowledge image track data are included in the multiple track data, and second element information is included in the descriptors respectively corresponding to the knowledge image track data, and the second element information is used to indicate a track group where the knowledge image track data is located.

[0178] In some embodiments of the present application, based on the foregoing solution, third element information is included in the descriptors respectively corresponding to the knowledge image track data, and a value of the third element information is used to indicate whether the knowledge image track data is relied on by multiple main bitstream track data.

[0179] In some embodiments of the present application, based on the foregoing solution, if there is target knowledge image track data relied on by multiple main bitstream track data among the at least two knowledge image track data, fourth element information is further included in the descriptor corresponding to the target knowledge image track data, and the fourth element information is used to indicate a frame rate of a specified main bitstream track data among the multiple main bitstream track data.

[0180] In some embodiments of the present application, based on the foregoing solution, a sample index identifier is further included in the descriptors respectively corresponding to the knowledge image track data, and the sample index identifier is used to indicate a sample index number interval in the main bitstream track data for indexing the knowledge image track data.

[0181] In some embodiments of the present application, based on the foregoing solution, the sample index identifier includes fifth element information and sixth element information. The value of the fifth element information indicates the minimum sample index number in the main bitstream track data for indexing the knowledge image track data, and the value of the sixth element information indicates the maximum sample index number in the main bitstream track data for indexing the knowledge image track data.

[0182] In some embodiments of the present application, based on the foregoing solution, the main bitstream track data contains an index identifier, which is used to indicate the knowledge image track data on which the main bitstream track data depends or the knowledge image track group on which the main bitstream track data depends.

[0183] In some embodiments of the present application, based on the foregoing solution, the main bitstream track data contains a track reference type data box, and the track reference type data box contains a reference type field, and the reference type field is used to represent the index identifier.

[0184] In some embodiments of the present application, based on the foregoing solution, the main bitstream track data contains a track reference data box, and the track reference data box contains the track reference type data box.

[0185] In some embodiments of the present application, based on the foregoing solution, at least two knowledge image track data are included in the multiple track data, and each knowledge image track data contains a track group identifier, and the track group identifier is used to indicate the track group where the knowledge image track data is located.

[0186] In some embodiments of the present application, based on the foregoing solution, each knowledge image track data further contains first field information for indicating whether the knowledge image track data is depended on by multiple main bitstream track data; if the first field information indicates that the knowledge image track data is depended on by 1 main bitstream track data, the knowledge image track data further contains a field indicating the minimum sample index number in the main bitstream track data for indexing the knowledge image track data, and a field indicating the maximum sample index number in the main bitstream track data for indexing the knowledge image track data.

[0187] In some embodiments of the present application, based on the foregoing solution, if the first field information indicates that the knowledge image track data is depended on by multiple main bitstream track data, the knowledge image track data further contains fields respectively indicating the minimum sample index number in each main bitstream track data for indexing the knowledge image track data, fields respectively indicating the maximum sample index number in the main bitstream track data for indexing the knowledge image track data, and a field for indicating the frame rate of each main bitstream track data in the multiple main bitstream track data.

[0188] In some embodiments of the present application, based on the foregoing solution, each piece of knowledge image track data further includes first field information for indicating whether the knowledge image track data is dependent on multiple main bitstream track data; if the first field information indicates that the knowledge image track data is dependent on one main bitstream track data, the knowledge image track data further includes a field indicating the number of sample groups in the main bitstream track data for indexing the knowledge image track data, and a field indicating the sample group index number in the main bitstream track data for indexing the knowledge image track data.

[0189] In some embodiments of the present application, based on the foregoing solution, if the first field information indicates that the knowledge image track data is dependent on multiple main bitstream track data, the knowledge image track data further includes fields respectively indicating the number of sample groups in the main bitstream track data for indexing the knowledge image track data, fields respectively indicating the sample group index numbers in the main bitstream track data for indexing the knowledge image track data, and a field for indicating the frame rate of each main bitstream track data among the multiple main bitstream track data.

[0190] In some embodiments of the present application, based on the foregoing solution, the processing device 900 for track data in the multimedia resource further includes: a decoding unit configured to determine a decoding order according to the dependency relationship; and sequentially perform decoding processing on the knowledge image track data and the main bitstream track data according to the decoding order to obtain the multimedia resource.

[0191] In some embodiments of the present application, based on the foregoing solution, the decoding unit is configured to: decode the main bitstream track data; when decoding to the sample index number interval in the main bitstream track data that needs to refer to the knowledge image track data, determine the knowledge image track data to be referred to from multiple pieces of knowledge image track data according to the sample index number interval; and decode the knowledge image track data to be referred to.

[0192] Figure 10 The block diagram of the processing device for track data in a multimedia resource according to an embodiment of the present application is shown. The processing device for track data in the multimedia resource can be provided in a media generation device, and the media generation device can be a smart phone, a tablet computer, etc.

[0193] Refer to Figure 10 As shown, the processing device 1000 for track data in a multimedia resource according to an embodiment of the present application includes: a generating unit 1002 and a sending unit 1004.

[0194] Among them, the generating unit 1002 is configured to generate a signaling file corresponding to the multimedia resource. The signaling file contains descriptors corresponding to multiple track data of the multimedia resource. The multiple track data include main bitstream track data corresponding to the main bitstream and knowledge image track data corresponding to the knowledge image bitstream. The dependency identifier included in the descriptor corresponding to the main bitstream track data points to the descriptor corresponding to the knowledge image track data. The sending unit 1004 is configured to send the signaling file to the data receiver, so that the data party determines the dependency relationship between the main bitstream track data and the knowledge image track data according to the dependency identifier in the signaling file, and sequentially obtains the knowledge image track data and the main bitstream track data from the data source side according to the dependency relationship.

[0195] In some embodiments of the present application, based on the foregoing solution, the generating unit 1002 is further configured to: before generating the signaling file corresponding to the multimedia resource, generate the main bitstream track data corresponding to the main bitstream and the knowledge image track data corresponding to the knowledge image bitstream. The main bitstream track data contains an index identifier, and the index identifier is used to indicate the knowledge image track data on which the main bitstream track data depends.

[0196] Figure 11 The structural schematic diagram of the computer system of the electronic device suitable for implementing the embodiments of the present application is shown.

[0197] It should be noted that Figure 11 The shown computer system 1100 of the electronic device is only an example, and should not bring any limitation to the functions and usage scopes of the embodiments of the present application.

[0198] As Figure 11 shown, the computer system 1100 includes a central processing unit (CPU) 1101, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1102 or the program loaded from the storage section 1108 into the random access memory (RAM) 1103, such as executing the method described in the above embodiments. In the RAM 1103, various programs and data required for system operation are also stored. The CPU 1101, ROM 1102, and RAM 1103 are connected to each other through a bus 1104. The input / output (I / O) interface 1105 is also connected to the bus 1104.

[0199] The following components are connected to the I / O interface 1105: an input section 1106 including a keyboard, a mouse, etc.; an output section 1107 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 1108 including a hard disk, etc.; and a communication section 1109 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1109 performs communication processing via a network such as the Internet. The drive 1110 is also connected to the I / O interface 1105 as needed. A removable medium 1111 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is installed on the drive 1110 as needed so that a computer program read therefrom is installed into the storage section 1108 as needed.

[0200] Specifically, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product including a computer program carried on a computer-readable medium, the computer program including a computer program for performing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 1109, and / or installed from the removable medium 1111. When the computer program is executed by a central processing unit (CPU) 1101, various functions defined in the system of the present application are executed.

[0201] It should be noted that the computer-readable medium shown in the embodiments of the present application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. In the present application, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable computer program. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0202] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. Among them, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code, and the above module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0203] The units involved in the embodiments described in this application can be implemented in software or in hardware, and the described units can also be provided in a processor. Among them, the names of these units do not constitute a limitation to the unit itself in certain cases.

[0204] As another aspect, this application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the methods described in the above embodiments.

[0205] It should be noted that although several modules or units of a device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of the two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0206] From the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented in software, or in a manner of software combined with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (such as a personal computer, a server, a touch terminal, or a network device, etc.) to execute the methods according to the embodiments of this application.

[0207] After considering the specification and practicing the embodiments disclosed herein, those skilled in the art will readily conceive of other embodiments of this application. This application is intended to cover any variations, uses, or adaptations of this application, which follow the general principles of this application and include known common general knowledge or conventional technical means in the technical field not disclosed in this application.

[0208] It should be understood that this application is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is only limited by the appended claims.

Claims

1. A method for processing track data in a multimedia resource, characterized in that Including: Receiving a signaling file corresponding to a multimedia resource, where the signaling file contains descriptors respectively corresponding to multiple track data of the multimedia resource. The multiple track data includes main bitstream track data corresponding to a main bitstream and knowledge image track data corresponding to a knowledge image bitstream. A dependency identifier included in the descriptor corresponding to the main bitstream track data points to the descriptor corresponding to the knowledge image track data; wherein, a sample index identifier is further included in each descriptor corresponding to the knowledge image track data, and the sample index identifier is used to indicate a sample index number interval in the main bitstream track data for indexing the knowledge image track data. Parsing the signaling file, and determining a dependency relationship between the main bitstream track data and the knowledge image track data according to the dependency identifier. Successively obtaining the knowledge image track data and the main bitstream track data from a data source side according to the dependency relationship. Wherein, successively obtaining the knowledge image track data and the main bitstream track data from the data source side according to the dependency relationship includes: According to the dependency relationship, obtaining a sample index identifier from a descriptor corresponding to the knowledge image track data on which the main bitstream track data depends, and determining a sample index number interval to be referred to by the main bitstream track data according to the sample index identifier. Obtaining the knowledge image track data indicated by the sample index number interval from the data source side, and obtaining the main bitstream track data.

2. The method for processing track data in multimedia resources according to claim 1, wherein A first element information is included in the descriptor corresponding to the knowledge image track data, and the first element information is used to indicate that the descriptor including the first element information is the descriptor corresponding to the knowledge image track data.

3. The method for processing track data in a multimedia resource according to claim 1, characterized in that, At least two knowledge image track data are included in the multiple track data, and a second element information is included in each descriptor corresponding to the knowledge image track data, and the second element information is used to indicate a track group where the knowledge image track data is located.

4. The method for processing track data in a multimedia resource according to claim 3, wherein A third element information is included in each descriptor corresponding to the knowledge image track data, and a value of the third element information is used to indicate whether the knowledge image track data is depended on by multiple main bitstream track data.

5. The method for processing track data in a multimedia resource according to claim 4, wherein If there is a target knowledge image track data among the at least two knowledge image track data that is depended on by multiple main bitstream track data, a fourth element information is further included in the descriptor corresponding to the target knowledge image track data, and the fourth element information is used to indicate a frame rate of a specified main bitstream track data among the multiple main bitstream track data.

6. The method for processing track data in a multimedia resource according to claim 1, characterized in that, The sample index identifier includes a fifth element information and a sixth element information. A value of the fifth element information indicates a minimum sample index number in the main bitstream track data for indexing the knowledge image track data, and a value of the sixth element information indicates a maximum sample index number in the main bitstream track data for indexing the knowledge image track data.

7. The method for processing track data in a multimedia resource according to claim 1, wherein An index identifier is included in the main bitstream track data, and the index identifier is used to indicate the knowledge image track data on which the main bitstream track data depends or to indicate a knowledge image track group on which the main bitstream track data depends.

8. The method for processing track data in a multimedia resource according to claim 7, characterized in that, The master bitstream track data contains a track reference type data box, and the track reference type data box contains a reference type field for indicating the index identifier.

9. The method for processing track data in a multimedia resource according to claim 8, wherein The master bitstream track data contains a track reference data box, and the track reference data box contains the track reference type data box.

10. The method for processing track data in a multimedia resource according to claim 1, wherein The multiple track data contains at least two knowledge image track data, and each knowledge image track data contains a track group identifier for indicating the track group where the knowledge image track data is located.

11. The method for processing track data in a multimedia resource according to claim 10, wherein Each knowledge image track data further contains first field information for indicating whether the knowledge image track data is relied on by multiple master bitstream track data; If the first field information indicates that the knowledge image track data is relied on by one master bitstream track data, the knowledge image track data further contains a field for indicating the minimum sample index number in the master bitstream track data for indexing the knowledge image track data, and a field for indicating the maximum sample index number in the master bitstream track data for indexing the knowledge image track data; and / or If the first field information indicates that the knowledge image track data is relied on by one master bitstream track data, the knowledge image track data further contains a field for indicating the number of sample groups in the master bitstream track data for indexing the knowledge image track data, and a field for indicating the sample group index number in the master bitstream track data for indexing the knowledge image track data.

12. The method for processing track data in a multimedia resource according to claim 11, wherein If the first field information indicates that the knowledge image track data is relied on by multiple master bitstream track data, the knowledge image track data further contains fields respectively indicating the minimum sample index number in each master bitstream track data for indexing the knowledge image track data, fields respectively indicating the maximum sample index number in the master bitstream track data for indexing the knowledge image track data, and a field for indicating the frame rate of each master bitstream track data in the multiple master bitstream track data; and / or If the first field information indicates that the knowledge image track data is relied on by multiple master bitstream track data, the knowledge image track data further contains fields respectively indicating the number of sample groups in the master bitstream track data for indexing the knowledge image track data, fields respectively indicating the sample group index number in the master bitstream track data for indexing the knowledge image track data, and a field for indicating the frame rate of each master bitstream track data in the multiple master bitstream track data.

13. The method for processing track data in a multimedia resource according to any one of claims 1 to 12, characterized in that, The processing method further includes: Determining a decoding order according to the dependency relationship; Sequentially performing decoding processing on the knowledge image track data and the master bitstream track data according to the decoding order to obtain the multimedia resource.

14. The method for processing track data in a multimedia resource according to claim 13, wherein Sequentially performing decoding processing on the knowledge image track data and the master bitstream track data according to the decoding order includes: Decoding the master bitstream track data; When decoding to the sample index number range in the master bitstream track data that needs to refer to the knowledge image track data, determining the knowledge image track data to be referred to from the multiple knowledge image track data according to the sample index number range; Decode the knowledge image track data to be referenced.

15. A method for processing track data in a multimedia resource, characterized in that, Including: Generate a signaling file corresponding to the multimedia resource, where the signaling file contains descriptors corresponding to multiple track data of the multimedia resource. The multiple track data includes main bitstream track data corresponding to the main bitstream and knowledge image track data corresponding to the knowledge image bitstream. The dependency identifier included in the descriptor corresponding to the main bitstream track data points to the descriptor corresponding to the knowledge image track data; wherein, the descriptor corresponding to the knowledge image track data further includes a sample index identifier, and the sample index identifier is used to indicate the sample index number interval in the main bitstream track data for indexing the knowledge image track data. Send the signaling file to the data receiver, so that the data receiver determines the dependency relationship between the main bitstream track data and the knowledge image track data according to the dependency identifier in the signaling file, and obtains the sample index identifier from the descriptor corresponding to the knowledge image track data on which the main bitstream track data depends according to the dependency relationship, determines the sample index number interval to be referenced by the main bitstream track data according to the sample index identifier, and obtains the knowledge image track data indicated by the sample index number interval from the data source side, and obtains the main bitstream track data.

16. The method for processing track data in a multimedia resource according to claim 15, wherein Before generating the signaling file corresponding to the multimedia resource, the processing method further includes: Generate main bitstream track data corresponding to the main bitstream and knowledge image track data corresponding to the knowledge image bitstream. The main bitstream track data includes an index identifier, and the index identifier is used to indicate the knowledge image track data on which the main bitstream track data depends.

17. A processing device for track data in a multimedia resource, characterized in that, Including: A receiving unit, configured to receive a signaling file corresponding to the multimedia resource, where the signaling file contains descriptors corresponding to multiple track data of the multimedia resource. The multiple track data includes main bitstream track data corresponding to the main bitstream and knowledge image track data corresponding to the knowledge image bitstream. The dependency identifier included in the descriptor corresponding to the main bitstream track data points to the descriptor corresponding to the knowledge image track data; wherein, the descriptor corresponding to the knowledge image track data further includes a sample index identifier, and the sample index identifier is used to indicate the sample index number interval in the main bitstream track data for indexing the knowledge image track data. An analysis unit, configured to analyze the signaling file and determine the dependency relationship between the main bitstream track data and the knowledge image track data according to the dependency identifier. An obtaining unit, configured to sequentially obtain the knowledge image track data and the main bitstream track data from the data source side according to the dependency relationship. Wherein, the obtaining unit is configured to: obtain a sample index identifier from the descriptor corresponding to the knowledge image track data on which the main bitstream track data depends according to the dependency relationship, and determine the sample index number interval to be referenced by the main bitstream track data according to the sample index identifier; obtain the knowledge image track data indicated by the sample index number interval from the data source side, and obtain the main bitstream track data.

18. A processing device for track data in a multimedia resource, characterized in that, Including: A generating unit, configured to generate a signaling file corresponding to a multimedia resource, where the signaling file contains descriptors respectively corresponding to multiple track data of the multimedia resource, the multiple track data includes main bitstream track data corresponding to a main bitstream and knowledge image track data corresponding to a knowledge image bitstream, and a dependency identifier included in the descriptor corresponding to the main bitstream track data points to the descriptor corresponding to the knowledge image track data; wherein, the descriptor corresponding to the knowledge image track data further includes a sample index identifier, and the sample index identifier is used to indicate a sample index number range in the main bitstream track data for indexing the knowledge image track data. A sending unit, configured to send the signaling file to a data receiver, so that the data receiver determines a dependency relationship between the main bitstream track data and the knowledge image track data according to the dependency identifier in the signaling file, obtains a sample index identifier from the descriptor corresponding to the knowledge image track data on which the main bitstream track data depends according to the dependency relationship, determines a sample index number range to be referred to by the main bitstream track data according to the sample index identifier, obtains the knowledge image track data indicated by the sample index number range from a data source side, and obtains the main bitstream track data.

19. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method for processing track data in a multimedia resource according to any one of claims 1 to 14, or implements the method for processing track data in a multimedia resource according to any one of claims 15 to 16.

20. An electronic device, characterized in that, Comprising: One or more processors; A storage device, configured to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method for processing track data in a multimedia resource according to any one of claims 1 to 14, or implement the method for processing track data in a multimedia resource according to any one of claims 15 to 16.

21. A computer program product, characterized in that, The computer program product includes a computer program, the computer program is stored in a computer-readable storage medium, and a processor of an electronic device reads and executes the computer program from the computer-readable storage medium, so that the electronic device executes the method for processing track data in a multimedia resource according to any one of claims 1 to 14, or implements the method for processing track data in a multimedia resource according to any one of claims 15 to 16.

Images