Media data processing method and apparatus, device and storage medium

Abstract

The embodiments of the present application are applied in the technical fields such as cloud technology and the Internet of Vehicles. Disclosed are a media data processing method and apparatus, a device and a storage medium. The method comprises: receiving object indication information; on the basis of the object indication information, acquiring a media file fragment to be decoded from an encapsulated media file corresponding to N media frames; and decoding said media file fragment to obtain media data corresponding to said media file fragment. By means of the present application, the acquisition efficiency of media data can be improved, and the resource overhead of a decoding device can be reduced.

Description

Media data processing method, device, equipment and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on August 18, 2023, with application number 2023110550365 and application name “Media Data Processing Method, Device, Equipment and Storage Medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to fields such as cloud technology and Internet of Vehicles technology, and in particular to a media data processing method, device, equipment and storage medium. Background Art

[0003] With the development of digital media technology and computer technology, media data (such as video data, point cloud data, etc.) has been applied to various fields, such as mobile communications, online games, online television, etc., bringing great convenience to people's entertainment and life. Under the condition of limited bandwidth, the encoding device needs to encode and encapsulate the acquired media data to obtain an encapsulated media file about the media data, and send the encapsulated media file about the media data to the decoding end. Usually, the decoding device only needs to encapsulate part of the media data in the media file, but at present, the full amount of the encapsulated media file needs to be decoded before the media data required by the decoding device can be obtained, resulting in relatively low efficiency in obtaining media data and unnecessary resource overhead.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a media data processing method, apparatus, device, and storage medium, which can improve the efficiency of acquiring media data and reduce the resource overhead of decoding devices.

[0006] An embodiment of the present application provides a method for processing media data, including:

[0007] Receive object indication information; the object indication information is used to reflect the object attribute characteristics of the media objects included in the N media frames, as well as the distribution characteristics of the media objects in the N media frames; N is a positive integer;

[0008] Obtaining, according to the object indication information, a media file segment to be decoded from the encapsulated media file corresponding to the N media frames;

[0009] The media file segment to be decoded is decoded to obtain media data corresponding to the media file segment to be decoded.

[0010] An embodiment of the present application provides a method for processing media data, including:

[0011] Get the encapsulated media file of N media frames; N is a positive integer;

[0012] If N media frames include media objects, object indication information about the N media frames is generated; the object indication information is used to indicate object attribute characteristics corresponding to the media objects included in the N media frames, as well as distribution characteristics of the media objects in the N media frames;

[0013] The encapsulated media file and object indication information are sent to a decoding device.

[0014] An embodiment of the present application provides, on one hand, a media data processing device, including:

[0015] A receiving module, configured to receive object indication information; the object indication information is used to reflect object attribute characteristics of the media objects included in the N media frames, as well as distribution characteristics of the media objects in the N media frames; N is a positive integer;

[0016] A first acquisition module is configured to acquire, according to the object indication information, a media file segment to be decoded from the encapsulated media file corresponding to the N media frames;

[0017] The decoding module is used to decode the media file segment to be decoded to obtain media data corresponding to the media file segment to be decoded.

[0018] An embodiment of the present application provides, on one hand, a media data processing device, including:

[0019] The second acquisition module is used to acquire an encapsulated media file of N media frames, where N is a positive integer;

[0020] a generating module configured to generate object indication information regarding the N media frames if the N media frames include media objects; the object indication information is configured to indicate object attribute characteristics corresponding to the media objects included in the N media frames, as well as distribution characteristics of the media objects in the N media frames;

[0021] The sending module is used to send the encapsulated media file and object indication information to the decoding device.

[0022] On the one hand, an embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the steps of a method are implemented when the processor executes the computer program.

[0023] On one hand, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and the steps of the method are implemented when the computer program is executed by a processor.

[0024] On the one hand, an embodiment of the present application provides a computer program product, including a computer program, which implements the steps of the method when the computer program is executed by a processor.

[0025] In the present application, a decoding device can receive object indication information, which is used to reflect the object attribute characteristics of the media objects included in N media frames, as well as the distribution characteristics of the media objects in the N media frames. That is, the object indication information is used to reflect what type of media objects are included in the N media frames, which media frames include media objects, and which positions in the media frames include media objects, etc. Based on this object indication information, the decoding device can quickly understand the media frames (i.e., media data) it needs. Therefore, based on the object indication information, it can quickly obtain the media file segments to be decoded from the encapsulated media files corresponding to the N media frames. The media file segments to be decoded can refer to the media file segments corresponding to the media data required by the decoding device, which can improve the efficiency of obtaining the media file segments to be decoded. Furthermore, only the media file segments to be decoded need to be decoded to obtain the media data corresponding to the media file segments to be decoded (such as the media data required by the decoding device), without having to decode the entire media file. This can reduce the amount of decoded data, improve the efficiency of obtaining media data, and reduce the resource (such as computing resource) overhead of the decoding device. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0027] FIG1 is a diagram of a media data processing architecture provided by the present application;

[0028] FIG2 is a flow chart of a media data processing method provided by the present application;

[0029] FIG3 is a schematic diagram of a single-track package provided in an embodiment of the present application;

[0030] FIG4 is a schematic diagram of a component-based multi-track package provided by an embodiment of the present application;

[0031] FIG5 is a schematic diagram of a multi-track package based on slicing provided in an embodiment of the present application;

[0032] FIG6 is a schematic diagram of a multi-track package based on slicing provided in an embodiment of the present application;

[0033] FIG7 is a schematic diagram of a target media file provided in an embodiment of the present application;

[0034] FIG8 is a schematic diagram of a target media file provided in an embodiment of the present application;

[0035] FIG9 is a flow chart of a method for processing media data according to an embodiment of the present application;

[0036] FIG10 is a schematic structural diagram of a media data processing device provided in an embodiment of the present application;

[0037] FIG11 is a schematic structural diagram of a media data processing device provided in an embodiment of the present application;

[0038] FIG12 is a schematic structural diagram of a computer device provided in an embodiment of the present application;

[0039] FIG13 is a schematic structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0041] The present application relates to the field of cloud technology. Among them, the present application relates to cloud computing in the field of cloud technology. Cloud computing (cloud coMPuting) is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The network that provides resources is called a "cloud". The resources in the "cloud" appear to be infinitely expandable to users, and can be obtained at any time, used on demand, and expanded at any time. The present application can generate object indication information about media frames through cloud computing.

[0042] The media frames in this application may include video frames in video data, point cloud frames in point cloud data, etc. Among them, when the application scenario of this application is a video data processing scenario (that is, when the media frames in this application are video frames in video data), the embodiments of this application relate to video data processing technology, and the complete processing process for video data may specifically include: video acquisition, video encoding, video file encapsulation, video transmission, video file decapsulation, video decoding and final video presentation. When the application scenario of this application is a point cloud data processing scenario (that is, when the media frames in this application are point cloud frames in point cloud data), the embodiments of this application relate to point cloud data processing technology, and the complete processing process for point cloud data may include: point cloud data acquisition, encoding and file encapsulation of point cloud data, transmission of point cloud data, file decapsulation and decoding of point cloud data, and rendering process of point cloud data.

[0043] Among them, the media in this application can be immersive media, which refers to media content that can bring an immersive experience to consumers. Immersive media can be divided into 3DoF media, 3DoF+ media and 6DoF media according to the degree of freedom of consumers in consuming media content. Point cloud data is a typical 6DoF media. The so-called point cloud data is a set of irregularly distributed discrete data points in space that express the spatial structure and surface properties of three-dimensional objects or scenes. Each data point in the point cloud data has at least geometric position information (i.e., three-dimensional position information). Depending on the application scenario, the data point also has color attributes, material or other attribute data. Typically, each point in the point cloud has the same number of additional attributes.

[0044] Point cloud data can flexibly and conveniently express the spatial structure and surface properties of three-dimensional objects or scenes. Therefore, it is widely used in virtual reality (VR) games, computer-aided design (CAD), geographic information systems (GIS), automatic navigation systems (ANS), digital cultural heritage, free viewpoint broadcasting, three-dimensional immersive telepresence, and three-dimensional reconstruction of biological tissues and organs.

[0045] With the continuous advancement of science and technology, it is now possible to obtain large amounts of highly accurate point cloud data at a relatively low cost and in a relatively short period of time. For example, point cloud data can be obtained by capturing real-world visual scenes using acquisition equipment (a set of cameras or camera devices with multiple lenses and sensors). Three-dimensional (3D) scanning can obtain point clouds of static real-world three-dimensional objects or scenes, with millions of point clouds acquired per second. 3D photography can obtain point clouds of dynamic real-world three-dimensional objects or scenes, with tens of millions of point clouds acquired per second. Furthermore, in the medical field, point cloud data of biological tissues and organs can be obtained through magnetic resonance imaging (MRI), computed tomography (CT), and electromagnetic positioning information. For another example, point cloud data can also be directly generated by computers based on virtual three-dimensional objects and scenes. With the continuous accumulation of large-scale point cloud data, the efficient storage, transmission, publication, sharing, and standardization of point cloud data have become key to point cloud applications.

[0046] FIG1 shows a media data processing architecture diagram provided by an exemplary embodiment of the present application. As shown in FIG1 , the data processing process in the encoding device mainly includes: (1) the acquisition process of point cloud data; (2) the encoding and file encapsulation process of point cloud data. The data processing process in the decoding device mainly includes: (3) the file decapsulation and decoding process of point cloud data; (4) the rendering process of point cloud data. In addition, the transmission process of point cloud data between the encoding device and the decoding device can be based on various transmission protocols. The transmission protocols here may include but are not limited to: DASH (Dynamic Adaptive Streaming over HTTP, dynamic adaptive streaming media transmission) protocol, LS (HTTP Live Streaming, dynamic bit rate adaptive transmission) protocol, SMTP (Smart Media Transport Protocol, intelligent media transmission protocol), TCP (Transmission Control Protocol, transmission control protocol), etc.

[0047] The following is a detailed description of the media data processing process:

[0048] (1) Obtain point cloud data.

[0049] Methods for acquiring point cloud data include: acquiring it through real-world audio-visual scenes using a capture device, and generating it through a computer. In one implementation, the capture device can be a hardware component within the encoding device, such as a microphone, camera, or sensor in a terminal. In another implementation, the capture device can be a hardware device connected to the encoding device, such as a camera connected to a server, configured to provide the encoding device with a service for acquiring media content from the point cloud data. The capture device may include, but is not limited to, audio equipment, video equipment, and sensor equipment. Audio equipment may include audio sensors and microphones. Video equipment may include standard cameras, stereo cameras, and light field cameras. Sensor equipment may include laser equipment and radar equipment. Point clouds can also be acquired through methods such as 3D laser scanning and 3D photogrammetry. Computers can generate point clouds for virtual 3D objects and scenes. 3D scanning can obtain point clouds for static real-world 3D objects or scenes, generating millions of point clouds per second. 3D imaging can obtain point clouds for dynamic real-world 3D objects or scenes, generating tens of millions of point clouds per second. Furthermore, in the medical field, MRI, CT, and electromagnetic positioning information can be used to obtain point clouds of biological tissues and organs. These technologies reduce the cost and time required to acquire point cloud data, while improving data accuracy. Changes in point cloud data acquisition methods have made it possible to acquire large amounts of point cloud data. Multiple capture devices can be deployed at specific locations in real space to simultaneously capture audio and video content from different angles within that space. The captured audio and video content is synchronized in both time and space. Due to different acquisition methods, the compression encoding methods corresponding to different point cloud data may also vary.

[0050] It is understood that point cloud data includes multiple point cloud samples. A point cloud sample can also be called a point cloud frame. A point cloud sample can include at least one of geometric data and attribute data. Attribute data includes color attributes, reflectivity, and so on. Geometric data is used to reflect the position information of the point cloud sample in the captured object, color attributes are used to reflect the color information of the captured object, and reflectivity is used to reflect the reflectivity of the captured object.

[0051] (2) The process of encoding point cloud data and file packaging.

[0052] Point cloud data encoding involves encoding both geometric data and attribute data. Geometric data encoding encodes the geometric data of point cloud samples within the point cloud data, generating the geometrically encoded data for the point cloud samples. Geometric data encoding schemes can include the following: (a) Octree-based geometric encoding (Octree): An octree is a tree-like data structure that evenly partitions the point cloud bounding box (i.e., the smallest cube containing all point clouds) in 3D space, with each node having eight child nodes. By indicating the occupancy of each child node in the octree as "1" or "0," occupancy code information is obtained, which is used as the geometrically encoded data for the point cloud samples. (b) Triangle-based geometric encoding (TriSouP): The point cloud (i.e., point cloud samples) is divided into blocks of a certain size. The intersection points of the point cloud surfaces at the edges of the blocks are located and triangles are constructed. The geometrically encoded data for the point cloud samples is obtained by encoding the intersection locations. Attribute data encoding refers to encoding the color attributes, reflectivity, etc. of point cloud samples to obtain attribute encoding data of the point cloud samples.

[0053] The encapsulation process of point cloud data includes: the encoding device encapsulates the encoding data of the point cloud sample and the parameter header contained in the point cloud sample to obtain the code stream data of the point cloud data, and encapsulates the code stream data of the point cloud data into the media track. The encoding data of the point cloud sample here includes at least one of the attribute encoding data and geometric encoding data of the point cloud sample, and the parameter header contained in the point cloud sample includes at least one of the geometric parameter header, the attribute parameter header and the sequence parameter header. The geometric parameter header includes the parameters required for decoding the geometric encoding data of the point cloud sample, the attribute parameter header includes the parameters required for decoding the attribute encoding data of the point cloud sample, and the sequence parameter header includes the parameters required for decoding the point cloud samples in the sequence where the point cloud samples are located (i.e., shared parameters). Since the parameters required for decoding some point cloud samples are the same, only some point cloud samples in the point cloud data contain parameter headers.

[0054] The term "media track" refers to a collection of media data used to encapsulate the bitstream data of point cloud data. This media track can consist of multiple time-sequential media track samples, with one media track sample being used to encapsulate the bitstream data of a point cloud frame. Point cloud data bitstream data can be encapsulated in one or more media tracks. For example, the resulting encapsulated media file can contain a video media track, an audio media track, and a subtitle media track. A sample is the unit of encapsulation used in media file encapsulation. A track consists of many samples, each of which corresponds to a specific timestamp. For example, a video media track can consist of many samples, typically a video frame, while a sample in a point cloud media track can be a point cloud frame. Each sample has its own sequence number (Sample Number), such as the first sample in a track has a sequence number of 1. Each track has a sample entry, which indicates metadata related to all samples in the track. For example, the sample entry for a video track typically contains metadata related to decoder initialization.

[0055] Specifically, when the media data is static media data, the code stream data corresponding to the static media data can be encapsulated into a data item (Item). A data item is a collection of media data in the process of encapsulating a static media file, such as a static image being encapsulated as an item. The point cloud frame can be sliced ​​to obtain a point cloud slice (also known as a point cloud strip). The point cloud slice / point cloud strip represents a set of syntax elements (such as geometric slices and attribute slices) of part or all of the encoded point cloud frame data. A point cloud slice corresponds to a point within a certain spatial area in the point cloud frame. The encoding device can perform encapsulation processing according to a specific media container file format (such as ISOBMFF), and combine one or more encoded bit streams into a sequence (Fs) of initialization segments and media segments (such as media file segments) for streaming transmission or a media file (F) for file playback. At the same time, the file encapsulation also includes metadata in the file F or the media segment Fs, and uses a transmission mechanism to transmit the segment Fs to a decoding device (such as a player).

[0056] (3) The process of file decapsulation and decoding of point cloud data;

[0057] The decoding device can obtain the media file resources (e.g., media tracks, media data items) and corresponding media presentation description information of the point cloud data from the encoding device. The media file resources and media presentation description information of the point cloud data are transmitted from the encoding device to the decoding device via a transmission mechanism (e.g., DASH, SMT). The file decapsulation process of the decoding device is the inverse of the file encapsulation process of the encoding device. The decoding device decapsulates the media file resources according to the file format requirements of the point cloud media to obtain an encoded bitstream. The encoded bitrate, also referred to as the bitstream data, can be a GPCC bitstream or a VPCC bitstream. The decoding process of the decoding device is the inverse of the encoding process of the encoding device. The decoding device decodes the encoded bitstream to restore the point cloud data point cloud samples. After receiving the initialization segment and the sequence (Fs) of media segments for streaming transmission or the media file (F) for file playback, the decoding device can decapsulate the segment sequence (Fs) or the media file (F), extract the bitstream data of the point cloud data, and decode the bitstream data of the point cloud data based on the corresponding metadata to obtain the corresponding point cloud data.

[0058] (4) Rendering process of point cloud data.

[0059] The decoding device renders the point cloud data obtained by decoding the GPCC bit stream according to the metadata related to rendering and window in the media presentation description information. Once the rendering is completed, the visual scene corresponding to the point cloud data is presented.

[0060] In one embodiment, an encoding device: samples a real-world visual scene through an acquisition device to obtain point cloud data corresponding to the real-world visual scene; encodes the obtained point cloud data through a geometry-based point cloud compression (GPCC) or a traditional video-based point cloud compression (VPCC) encoding method to obtain a GPCC bitstream (including an encoded geometry bitstream and an attribute bitstream) or a VPCC bitstream; and encapsulates the GPCC bitstream or the VPCC bitstream to obtain a media file (i.e., point cloud media) corresponding to the point cloud data. Specifically, the encoding device synthesizes one or more encoded bitstreams into a media file for file playback, or a sequence of initialization segments and media segments for streaming, according to a specific media container file format; wherein the media container file format refers to the ISO base media file format specified in the International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC) 14496-12. In one embodiment, the encoding device also encapsulates the metadata into a media file or a sequence of initialization / media segments, and transmits the sequence of initialization / media segments to the decoding device via a transmission mechanism (such as a dynamic adaptive streaming media transmission interface).

[0061] On the decoding device: Receive the point cloud media file sent by the encoding device, including: a media file for file playback, or a sequence of initialization segments and media segments for streaming; decapsulate the point cloud media file to obtain an encoded GPCC bitstream or VPCC bitstream and metadata related to the point cloud media file presentation; parse the encoded GPCC bitstream (i.e., decode the encoded GPCC bitstream to obtain point cloud data); finally, based on the current user's viewing (viewing window) direction, render the decoded point cloud data and display it on the screen of a head-mounted display or any other display device carried by the decoding device. It should be noted that the current user's viewing (viewing window) direction is determined by head detection and possibly visual detection functions. In addition to using the renderer to render the point cloud data in the current user's viewing (viewing window) direction, the audio decoder can also be used to decode and optimize the audio in the current user's viewing (viewing window) direction. It will be appreciated that during media processing, point cloud data can be rendered and displayed on the screen of a head-mounted display or any other display device based on the current viewing position, viewing direction, or viewport determined by various types of sensors (e.g., head, position, or eye sensors). Point cloud data that is partially accessed and decoded based on the current viewing position or viewing direction can be used to optimize the media processing process. In the viewport-based transmission process, the current viewing position and viewing direction are also passed to the strategy module for determining the track to be received.

[0062] Further, please refer to Figure 2, which is a flowchart of a media data processing method provided by an embodiment of the present application. As shown in Figure 2, the method can be executed by an encoding device, and the computer device can refer to an encoding device, wherein the method can include but is not limited to the following steps:

[0063] S101: Obtain an encapsulated media file of N media frames.

[0064] Specifically, the computer device can obtain an encapsulated media file about N media frames, where N is a positive integer. Among them, the embodiment of the present application can be applied to point cloud data scenarios, that is, the N media frames can be point cloud frames in point cloud data, and of course can also be applied to other types of media scenarios, such as video data scenarios, that is, the N media frames can be video frames in video data. Specifically, after obtaining N media frames, the computer device can encode the N media frames to obtain code stream data (such as media bit stream) of the N media frames. Further, the computer device can encapsulate the code stream data of the N media frames to obtain an encapsulated media file about the code stream data of the N media frames.

[0065] Specifically, when encapsulating the code stream data of N media frames, the following three modes are included but not limited to: Mode 1, single-track encapsulation; Mode 2, component-based multi-track encapsulation; Mode 3: fragment-based multi-track encapsulation.

[0066] Specifically, as shown in Figure 3, Figure 3 is a schematic diagram of a single-track encapsulation provided by an embodiment of the present application. As shown in Figure 3, a computer device can encapsulate the code stream data of N media frames in a media track. The media track includes a sample entry and one or more samples. The sample entry of the media track is used to encapsulate metadata information related to all samples in the current media track. The metadata information is used to instruct the decoding device to decapsulate the samples in the current media track based on the metadata information in the sample entry. A sample in the media track is used to encapsulate the code stream data of a media frame. The sample can be used to encapsulate parameter information of the media frame. The parameter information refers to a set of parameters required when decoding the code stream data of the corresponding media frame. The sample can be used to encapsulate the geometric data of the media frame. The geometric data includes geometric information (such as position information) of the corresponding media frame. The sample can be used to encapsulate attribute data of the media frame. The attribute data includes attribute information (such as color attributes, reflectivity, etc.) of the corresponding media frame.

[0067] Specifically, as shown in Figure 4, Figure 4 is a schematic diagram of a component-based multi-track encapsulation provided by an embodiment of the present application. As shown in Figure 4, a computer device can encapsulate the code stream data of N media frames in multiple media tracks. Specifically, the geometric data obtained through the geometry component can be encapsulated in a media track, such as media track 1 (geometry component track). The media track 1 includes a sample entry and one or more samples. The sample entry of the media track 1 is used to encapsulate metadata information related to all samples in the current media track. The metadata information is used to instruct the decoding device to decapsulate the samples in the current media track based on the metadata information in the sample entry. A sample in the media track 1 is used to encapsulate parameter information (such as geometric parameter information) and geometric data of a media frame. Specifically, the computer device can encapsulate the attribute data obtained through the attribute component 1 in media track 2 (i.e., attribute component track 1). Similarly, the media track 2 includes a sample entry and one or more samples. The sample entry of the media track 2 is used to encapsulate metadata information related to all samples in the current media track. A sample in the media track 2 is used to encapsulate parameter information (such as attribute 1 parameter information) and attribute 1 data of a media frame. Specifically, the computer device can encapsulate the attribute data obtained through attribute component 2 in media track 3 (i.e., attribute component track 2). Similarly, media track 3 includes a sample entry and one or more samples. The sample entry of media track 3 is used to encapsulate metadata information related to all samples in the current media track. A sample in media track 3 is used to encapsulate parameter information (such as attribute 2 parameter information) and attribute 2 data for a media frame. At the same time, an association relationship can be established between media track 1, media track 2, and media track 3.

[0068] Specifically, as shown in Figure 5, Figure 5 is a schematic diagram of a multi-track encapsulation based on fragmentation provided by an embodiment of the present application. As shown in Figure 5, the computer device can fragment each media frame of N media frames to obtain multiple fragments, such as dividing each media frame into 3 fragments, namely fragment 1, fragment 2 and fragment 3. The computer device can encapsulate the code stream data corresponding to N media frames in multiple media tracks based on fragmentation. Specifically, the computer device can encapsulate the fragmentation information of the media frame in a fragmentation base track, which also includes a sample entry and multiple samples. Similarly, the sample entry of the fragmentation base track is used to store metadata information of the samples in the fragmentation base track, and a sample of the fragmentation base track is used to store the fragmentation information of a media frame, that is, the geometry header (such as the parameter set required for geometric data decoding) and attribute header (such as the parameter set required for attribute data decoding) of the corresponding media frame are encapsulated in each sample.

[0069] As shown in Figure 5, the computer device can encapsulate the geometric data and attribute data together, encapsulate the code stream data corresponding to slice 1 and slice 2 in each media frame in one media track, that is, encapsulate it in slice track 1, and encapsulate the code stream data corresponding to slice 3 in another media track, that is, encapsulate it in slice track 2. Slice track 1 also includes a sample entry and multiple samples. The sample entry is used to encapsulate the metadata information of all samples in slice track 1, and each sample in slice track 1 is used to encapsulate the code stream data corresponding to slice 1 and slice 2 of a media frame. Specifically, a sample can be used to store the geometric header (that is, the geometric parameters of the slice), geometric code stream, attribute segment (that is, the attribute parameters of the slice) and attribute code stream of slice 1 in a media frame, as well as the geometric header (that is, the geometric parameters of the slice), geometric data, attribute segment (that is, the attribute parameters of the slice) and attribute data of slice 2. Slice Track 2 also includes a sample entry and multiple samples. The sample entry is used to encapsulate metadata information for all samples in Slice Track 2. Each sample in Slice Track 2 is used to encapsulate the codestream data corresponding to Slice 3 of a media frame. Specifically, a sample can be used to store the geometry header (i.e., the geometric parameters of the slice), geometry codestream, attribute segment (i.e., the attribute parameters of the slice), and attribute codestream of Slice 3 in a media frame. At the same time, an association can be established between the Slice Base Track, Slice Track 1, and Slice Track 2.

[0070] Specifically, as shown in FIG6 , FIG6 is a schematic diagram of a multi-track encapsulation based on slicing provided by an embodiment of the present application. As shown in FIG6 , compared with the slicing encapsulation in FIG5 , the slicing encapsulation in FIG6 is different in that the geometric data and the attribute data are encapsulated separately. Similarly, the computer device can slice each of the N media frames to obtain multiple slices, such as dividing each media frame into three slices, namely slice 1, slice 2 and slice 3. Based on the slices, the code stream data corresponding to the N media frames are encapsulated in multiple media tracks. Specifically, the computer device can encapsulate the slicing basic information of the media frame in the slicing basic track. The slicing basic track also includes a sample entry and multiple samples. Similarly, the sample entry of the slicing basic track is used to store the metadata information of the samples in the slicing basic track. A sample of the slicing basic track is used to store the slicing information of a media frame, that is, the geometry header (such as the parameter set required for decoding the geometric data) and the attribute header (such as the parameter set required for decoding the attribute data) of the corresponding media frame are encapsulated in each sample.

[0071] As shown in Figure 6, the computer device can encapsulate the geometric data and attribute data separately, encapsulating the geometric data corresponding to Slice 1 and Slice 2 in each media frame in one media track, that is, in Slice Track 1, encapsulating the attribute data corresponding to Slice 1 and Slice 2 in each media frame in one media track, that is, in Slice Track 2, encapsulating the geometric data corresponding to Slice 3 in another media track, that is, in Slice Track 3, and encapsulating the attribute data corresponding to Slice 3 in another media track, that is, in Slice Track 4. Specifically, Slice Track 1 also includes a sample entry and multiple samples. The sample entry is used to encapsulate the metadata information of all samples in Slice Track 1, and each sample in Slice Track 1 is used to encapsulate the geometric data corresponding to Slice 1 and Slice 2 of a media frame. Specifically, a sample can be used to store the geometric slice header (that is, the geometric parameters of the slice) and geometric code stream of Slice 1 in a media frame, as well as the geometric slice header (that is, the geometric parameters of the slice) and geometric code stream of Slice 2. Similarly, slice track 2 also includes a sample entry and multiple samples. The sample entry is used to encapsulate metadata information for all samples in slice track 2. Each sample in slice track 2 is used to encapsulate attribute data corresponding to slice 1 and slice 2 of a media frame. Specifically, a sample can be used to store the attribute slice header (i.e., the attribute parameters of the slice) and attribute code stream of slice 1, as well as the attribute slice header (i.e., the attribute parameters of the slice) and attribute code stream of slice 2 in a media frame.

[0072] Similarly, as shown in Figure 6, slice track 3 also includes a sample entry and multiple samples. The sample entry is used to encapsulate the metadata information of all samples in slice track 3, and each sample in slice track 3 is used to encapsulate the geometric data corresponding to slice 3 of a media frame. Specifically, one sample can be used to store the geometric header (i.e., the geometric parameters of the slice) and the geometric code stream of slice 3 in a media frame. Similarly, slice track 4 also includes a sample entry and multiple samples. The sample entry is used to encapsulate the metadata information of all samples in slice track 4, and each sample in slice track 4 is used to encapsulate the attribute data corresponding to slice 3 of a media frame. Specifically, one sample can be used to store the attribute header (i.e., the attribute parameters of the slice) and the attribute code stream of slice 3 in a media frame. At the same time, an association relationship can be established between the slice base track, slice track 1, slice track 2, slice track 3 and slice track 4.

[0073] S102: If the N media frames include media objects, generate object indication information about the N media frames.

[0074] Specifically, if N media frames include media objects, the computer device can generate object indication information about the N media frames, and the object indication information is used to indicate the object attribute characteristics corresponding to the media objects included in the N media frames, as well as the distribution characteristics of the media objects in the N media frames. Among them, the media objects can refer to items (such as cups, chairs, etc.), virtual props, virtual characters, people, animals, etc. in the media frames. It can be understood that the media objects included in each media frame can be quickly determined through the object indication information. In this way, it is possible to quickly locate a specific media frame (such as a media frame including a specific media object), and then quickly obtain the media data corresponding to the specific media frame, which can improve the efficiency of media data acquisition. Among them, the object attribute characteristics and distribution characteristics of the media objects included in each media frame in the N media frames can be obtained by performing object recognition on the N media frames during the acquisition stage of the N media frames, or can be obtained by performing algorithmic analysis on the code stream data corresponding to the N media frames, or can be obtained by other methods. The embodiments of the present application do not limit this.

[0075] Optionally, the object attribute characteristics in the object indication information may include one or more of the number of objects, object identifiers, and object description information of the media objects included in the N media frames. The object identifier may be an OID identifier (i.e., object identifiers, which is an identifier mechanism standardized by the International Telecommunication Union (ITU) and the International Organization for Standardization (ISO / IEC) and is used to name any object, concept, or "thing" with a clear and permanent name worldwide). The specific object information of the corresponding media object can be clearly determined through the OID identifier. The object description information may refer to the object specific information of the media object described in the form of a human-readable character string, such as object structure, object color, object material, object function, etc.

[0076] Optionally, the distribution feature in the object indication information is used to indicate the target media frame with the media object in N media frames, that is, the distribution feature can be used to indicate the target media frame to which the media object included in the N media frames belongs. Alternatively, the distribution feature is used to indicate the target media frame with the media object in N media frames, and the spatial region to which the media object belongs in the target media frame, that is, the media object included in the spatial region in each media frame. Alternatively, the distribution feature is used to indicate the target media frame with the media object in N media frames, and the slice to which the media object belongs in the target media frame. Alternatively, the distribution feature is used to indicate the target media frame with the media object in N media frames, and the spatial region to which the media object belongs in the target media frame, and the slice to which the media object belongs in the target media frame. It can be understood that the distribution feature can be used to indicate one or more of the information such as the media objects included in each media frame, the media objects included in the spatial region within each media frame, and the slice to which the media objects included in each media frame are located. In this way, through the object indication information about N media frames, the media data of a specific media frame (such as a media frame including a specific media object) can be quickly located, the media data of a specific slice (such as a point cloud slice or video slice including a specific media object) can be quickly located, and the media data of a specific spatial area (such as a spatial area including a specific media object) can be quickly located, thereby improving the efficiency of obtaining media data.

[0077] Optionally, the object indication information can also be used to indicate the change characteristics of the media objects included in the N media frames, where the change characteristics include object change characteristics and distribution change characteristics. The object change characteristics are determined based on the object attribute characteristics of the media objects in the reference media frame and the object attribute characteristics of the dynamic media frame, and the distribution change characteristics are determined based on the distribution characteristics of the media objects in the reference media frame and the distribution characteristics of the dynamic media frame. The dynamic media frame belongs to the N media frames, and the dynamic media frame refers to the media frame that records dynamic pictures or motion processes, such as the media frames in film and television works; the N media frames can also include static media frames, and the static media frame refers to the media frame that records static pictures, such as game frames. The reference media frame can refer to a dynamic media frame whose encapsulation order is before the encapsulation order of the dynamic media frame and which encapsulates the complete object indication information (that is, the corresponding metadata track sample is a synchronized metadata track sample).

[0078] Optionally, when the encapsulated media file regarding N media frames includes multiple media files with associated media objects, the object indication information also includes object relationship indication information, and the object relationship indication information is used to indicate that the multiple media files with associated media objects have an associated relationship. Specifically, the encapsulated media file regarding N media frames includes a first media file and a second media file, and the first media file and the second media file can be sub-files in the encapsulated media file. The first media file and the second media file can both refer to one or more of the media tracks and media data items in the encapsulated media file. When there is an associated relationship between the media object in the media frame corresponding to the first media file and the media object in the media frame corresponding to the second media file, the object indication information can also include object relationship indication information, and the object relationship indication information is used to indicate that there is an associated relationship between the media object in the media frame corresponding to the first media file and the media object in the media frame corresponding to the second media file. Specifically, having an associated relationship can mean that the media object in the media frame corresponding to the first media file is the same as the media object in the media frame corresponding to the second media file. Alternatively, "having an association relationship" may mean that a media object in a media frame corresponding to a first media file has an association relationship with a media object in a media frame corresponding to a second media file, such as a binding relationship, associated motion, or associated display. In this way, by using the object relationship indication information, media data with an associated relationship can be quickly retrieved, improving the efficiency of obtaining specific media data.

[0079] Optionally, when the codestream data corresponding to N media frames is encapsulated in a media track, the object indication information may further include track object indication information belonging to each media track object. This track object indication information is used to indicate the object attribute characteristics of the media objects in the media frames belonging to the corresponding media track, that is, the object attribute characteristics of the media objects included in each media track. This allows for rapid location of media tracks that meet the requirements (e.g., media tracks that include specific media objects or media tracks that do not include media objects), improving the efficiency of acquiring specific media data.

[0080] Optionally, when the codestream data corresponding to N media frames is encapsulated in a media data item, the object indication information may further include data item object indication information corresponding to each media data item. This data item object indication information is used to indicate object attribute characteristics of the media objects in the media frames corresponding to the media data items, i.e., the object attribute characteristics of the media objects included in each media data item. This allows for rapid location of media data items that meet the criteria (e.g., media data items that include specific media objects or media data items that do not include media objects), improving the efficiency of acquiring specific media data.

[0081] S103: Send the encapsulated media file and object indication information to a decoding device.

[0082] Specifically, the computer device can send the encapsulated media file and object indication information to the decoding device. The decoding device can then, based on the object indication information, retrieve the media file segments to be decoded from the encapsulated media file, decode the media file segments to be decoded, and obtain the media data corresponding to the media file segments to be decoded. In this way, based on the object indication information, the media file segments to be decoded can be quickly retrieved from the encapsulated media file. Only the media file segments to be decoded need to be decoded (i.e., only part of the media file needs to be decoded, which can reduce the amount of decoded data). This allows for rapid acquisition and presentation of specific media data, thereby improving the efficiency of acquiring specific media data.

[0083] Optionally, the computer device may directly send the encapsulated media file and object indication information to the decoding device.

[0084] Optionally, the specific manner in which the computer device sends the encapsulated media file and object indication information to the decoding device may include: if the encapsulated media file includes S media file segments, extracting object indication information associated with each of the S media file segments from the object indication information; S is an integer greater than 1. Encapsulating the object indication information associated with media file segment i among the S media file segments into media file segment i to obtain a target media file segment i; S is an integer greater than 1, and i is a positive integer less than or equal to S. Sending the object indication information associated with each of the S media file segments, as well as the segment identifiers corresponding to each of the S media file segments, to the decoding device. If a request for obtaining the target media file segment i is received, sending the target media file segment i to the decoding device; the request for obtaining is generated by the decoding device based on the object indication information and segment identifiers associated with each of the S media file segments.

[0085] Specifically, when an encapsulated media file includes S media file segments, the object indication information may include object indication information corresponding to the S media file segments. The computer device may extract the object indication information associated with each of the S media file segments from the object indication information regarding N media objects. Taking media file segment i among the S media file segments as an example, where i is a positive integer less than or equal to S, the object indication information associated with media file segment i indicates the object attribute characteristics and distribution characteristics of the media objects included in the media frame corresponding to media file segment i. The computer device may encapsulate the object indication information associated with media file segment i into media file segment i to obtain target media file segment i. The computer device may transmit the object indication information associated with each of the S media file segments, as well as the segment identifiers corresponding to each of the S media file segments, to a decoding device. The decoding device may request the required media file segment from the S media file segments based on the object indication information associated with each of the S media file segments and the segment identifiers corresponding to each of the S media file segments.

[0086] Furthermore, the computer device may receive a request to obtain target media file segment i. This request may involve the decoding device determining, based on the object indication information and segment identifiers associated with each of the S media file segments, that media file segment i is the desired media file segment, and then generating a request based on the segment identifier corresponding to media file segment i. The computer device may send target media file segment i to the decoding device. This way, the object indication information is sent to the decoding device, and the decoding device requests the desired media file segment from the encoding device, eliminating the need to send the encapsulated media file to the decoding device all at once. This reduces media file transmission pressure and improves media file transmission efficiency. Furthermore, the decoding device only needs to decode the target media file segment, reducing the amount of media decoding data and improving the efficiency of obtaining specific media data.

[0087] In particular, when sending the object indication information associated with the S media file segments and the segment identifiers corresponding to the S media file segments to the decoding device, the computer device can generate a descriptive signaling file (DASH) based on the object indication information associated with the S media file segments and the segment identifiers corresponding to the S media file segments, and send the descriptive signaling file to the decoding device to achieve progressive transmission of the S media file segments (i.e., the requested media file segments are transmitted only when the decoding device requests them). The descriptive signaling file includes an object information descriptor, which is used to describe the object attribute characteristics of the media objects included in the media resources (such as media file segments) corresponding to each Adaptation Set. In DASH, Adaptation Sets can be a collection of one or more video streams, and an Adaptation Set can contain multiple Representations. The Representation can be a combination of one or more media components in DASH. For example, a video file of a certain resolution can be regarded as a Representation. For example, an Adaptation Set (i.e., an adaptation set) can be used to indicate one or more media file segments. The object information descriptor is a SupplementalProperty element, and its @schemeIdUri attribute is set to "urn:avs:pccs:2023:obif".

[0088] The elements and attributes included in the object information descriptor can be found in Table 1.

[0089] Table 1

[0090] The media resources corresponding to the metadata track should be present in the DASH signaling as a separate Adaptation Set, and the Adaptation Set should contain only one Representation (i.e., media combination). The Representation should use the @associationId element (an association element) to associate one or more Representations corresponding to the media tracks described by the metadata track, and the corresponding @associationType field should have the value 'obdi'.

[0091] Optionally, a specific method for the computer device to send the encapsulated media file and object indication information to the decoding device may include: the computer device may encapsulate the object indication information into the encapsulated media file, obtain the target media file, and send the target media file to the decoding device. In this way, the decoding device can obtain the media file segments to be decoded from the encapsulated media file in the target media file based on the object indication information in the target media file, decode the media file segments to be decoded, and obtain media data corresponding to the media file segments to be decoded. In this way, the media file segments to be decoded can be quickly obtained from the encapsulated media file based on the object indication information, and only the media file segments to be decoded need to be decoded (i.e., only a portion of the media file needs to be decoded, which can reduce the amount of decoded data). Specific media data can be quickly obtained and presented, and the efficiency of obtaining specific media data can be improved.

[0092] Optionally, the N media frames include K dynamic media frames having media objects, and the encapsulated media file includes P media tracks belonging to the K dynamic media frames, where P is a positive integer and K is a positive integer less than or equal to N. It is understood that the encapsulated media file includes P media tracks obtained by encapsulating the codestream data corresponding to the K dynamic media frames. For example, encapsulating the codestream data corresponding to the K dynamic media frames using a single-track encapsulation mode can result in one media track; encapsulating the codestream data corresponding to the K dynamic media frames using a component-based multi-track encapsulation mode can result in multiple media tracks. A specific method for a computer device to send the encapsulated media file and object indication information to a decoding device may include: obtaining object attribute characteristics and distribution characteristics of media objects included in dynamic media frames belonging to media track j from the object indication information, where j is a positive integer less than or equal to P. Encapsulating the object attribute characteristics of media objects in dynamic media frames belonging to media track j into object information data box j associated with media track j. Encapsulating the object attribute characteristics and distribution characteristics of media objects included in dynamic media frames belonging to media track j into the metadata track corresponding to media track j. The object information data boxes and metadata tracks corresponding to the P media tracks are added to the encapsulated media file to obtain a target media file, and the target media file is sent to a decoding device.

[0093] Specifically, the computer device may obtain, from the object indication information, object attribute characteristics and distribution characteristics of the media objects included in the dynamic media frames belonging to media track j, where media track j belongs to any one of the P media tracks. Furthermore, the computer device may use the object attribute characteristics of the media objects in the dynamic media frames belonging to media track j as the track object indication information corresponding to media track j, and encapsulate the track object indication information corresponding to media track j into an object information data box associated with media track j. It will be understood that each media track has an associated object information data box, which is used to encapsulate the object attribute characteristics and distribution characteristics of the media objects included in the media frames belonging to the corresponding media track.

[0094] The data box type of the object information data box may be "obif (a data box type)", and the object information data box may be included in the sample entry of the track. The number of object information data boxes may be 0 or one. The computer device may set the mandatory attribute of the object information data box to be non-mandatory. The specific content of the object information data box associated with media track j encapsulated by the computer device into the track object indication information corresponding to media track j can be seen in Table 2.

[0095] Table 2

[0096] Among them, num_objects in Table 2 indicates the number of media object entries contained in the current object information data box. When object_oid_flag takes a value of 0, it indicates that the object description information about the media object in the current object information data box is indicated in the form of a human-readable string, that is, the object description information of the media object is indicated in the form of a human-readable string. When object_oid_flag takes a value of 1, it indicates that the object description information about the media object in the current object information data box is indicated in the form of an OID, that is, the object description information of the media object is indicated in the form of an OID. object_info_id indicates the identifier (i.e., object identifier) ​​of the corresponding media object entry, object_oid indicates the object description information corresponding to the corresponding media object entry, indicated in the form of an OID. object_label indicates the object description information corresponding to the corresponding media object entry, indicated in the form of a human-readable string.

[0097] Optionally, object_label can be implemented as follows: the object_label string consists of N labels at different levels, with each level separated by spaces. For example, object_label1: "horse head" object_label2: "horse body", etc.

[0098] At the same time, the computer device may encapsulate the object attribute characteristics and distribution characteristics of the media objects included in the dynamic media frame belonging to media track j into the metadata track corresponding to media track j. It is understood that the object attribute characteristics and distribution characteristics of the media objects included in the dynamic media frame can be included in the media file as a media type in the form of a metadata track. Furthermore, the computer device may add the object information data boxes and metadata tracks corresponding to the P media tracks to the encapsulated media file to obtain a target media file, and then send the target media file to the decoding device.

[0099] Optionally, the metadata track corresponding to media track j includes metadata track samples corresponding to the dynamic media frames belonging to media track j, that is, one metadata track sample corresponds to each dynamic media frame. When the computer device encapsulates the object attribute characteristics and distribution characteristics of the media objects included in the dynamic media frames belonging to media track j into the metadata track corresponding to media track j, the object attribute characteristics and distribution characteristics of the media objects included in each dynamic media frame can be encapsulated into the corresponding metadata track sample. Of course, the change characteristics between the object attribute characteristics and distribution characteristics of the media objects included in each dynamic media frame and the object attribute characteristics and distribution characteristics of the media objects included in the reference media frame can also be encapsulated into the corresponding metadata track sample.

[0100] Optionally, the specific method in which the computer device encapsulates the object attribute characteristics and distribution characteristics of the media objects included in each dynamic media frame into the corresponding metadata track sample may include: adding the object attribute characteristics and distribution characteristics of the media objects included in the dynamic media frame a belonging to the media track j to the metadata track sample corresponding to the dynamic media frame a; a is less than or equal to the total number of dynamic media frames belonging to the media track j.

[0101] The computer device can encapsulate the object attribute characteristics and distribution characteristics of the media objects included in each dynamic media frame belonging to media track j in a metadata track sample corresponding to media track j. Specifically, taking dynamic media frame a belonging to media track j as an example, a is less than or equal to the total number of dynamic media frames belonging to media track j. The computer device can add the object attribute characteristics and distribution characteristics of the media objects included in dynamic media frame a belonging to media track j to the metadata track sample corresponding to dynamic media frame a. At this time, when the computer device can encapsulate the complete object indication information included in dynamic media frame a belonging to media track j in the metadata track sample corresponding to dynamic media frame a, the metadata track sample corresponding to dynamic media frame a can be a synchronized metadata track sample, that is, the synchronized metadata track sample is used to indicate the complete object indication information included in the corresponding dynamic media frame. In this way, the object attribute characteristics and distribution of the media objects included in the corresponding dynamic media frame can be obtained through the synchronized metadata track sample.

[0102] Specifically, the computer device encapsulates the complete object indication information included in the dynamic media frame a belonging to the media track j in the specific content of the metadata track sample corresponding to the dynamic media frame a (ie, the synchronous metadata track sample) can be seen in Table 3.

[0103] Table 3

[0104] Among them, num_object in Table 3 indicates the number of media object entries contained in the current metadata track sample (that is, the number of media object objects). When the num_object field takes a value of 0, it indicates that the current metadata track sample does not contain a media object entry. ref_object_info_id is used to indicate the identifier of the corresponding media object entry (that is, the object identifier of the media object). When the object_spatial_info_flag takes a value of 0, it indicates that the specific spatial area information corresponding to the media object is not indicated; when the object_spatial_info_flag takes a value of 1, it indicates that the specific spatial area information corresponding to the media object is indicated. When the object_slice_info_flag takes a value of 0, it indicates that the slice information corresponding to the media object (such as point cloud slice information) is not indicated; when the value is 1, it indicates that the slice information corresponding to the media object is indicated. 3DPoint is used to indicate the anchor coordinates of the corresponding spatial area, CuboidRegionStruct is used to indicate the size information of the corresponding spatial area, and SliceMapping is used to indicate the corresponding slice information.

[0105] Optionally, the metadata track corresponding to media track j includes metadata track samples corresponding to the dynamic media frames belonging to media track j. The computer device may encapsulate the object attribute characteristics and distribution characteristics of the media objects included in each dynamic media frame, and the change characteristics between the object attribute characteristics and distribution characteristics of the media objects included in the reference media frame, into the corresponding metadata track samples. The specific method may include: obtaining the object attribute characteristics and distribution characteristics of the media objects in the reference media frame corresponding to dynamic media frame a; a is less than or equal to the total number of dynamic media frames belonging to media track j. Determining the object change characteristics between the object attribute characteristics of the media objects in the reference media frame and the object attribute characteristics of dynamic media frame a. Determining the distribution change characteristics between the distribution characteristics of the media objects in the reference media frame and the distribution characteristics of dynamic media frame a, and adding the object change characteristics and distribution change characteristics to the metadata track samples corresponding to dynamic media frame a. In this way, only storing the change characteristics can avoid duplicate storage, reduce the amount of stored data, and thus reduce storage pressure.

[0106] It is understandable that the computer device can use a dynamic media frame whose encapsulation order is located before the encapsulation order of the dynamic media frame a and which encapsulates complete object indication information (i.e., the corresponding metadata track sample is a synchronized metadata track sample) as the reference media frame of the dynamic media frame a. The computer device can use the object attribute characteristics and distribution characteristics of the media objects in the reference media frame corresponding to the dynamic media frame a to determine the object change characteristics between the object attribute characteristics of the media objects in the reference media frame and the object attribute characteristics of the dynamic media frame a. For example, the object change characteristics can be used to indicate whether the media objects included in the dynamic media frame a still exist in the reference media frame, or whether the object description information has changed (such as color, structure, etc.). At the same time, the computer device can determine the distribution change characteristics between the distribution characteristics of the media objects in the reference media frame and the distribution characteristics of the dynamic media frame a. The distribution change characteristics can be used to indicate the change information of the spatial area or fragmentation in which the media objects are distributed. For example, the distribution change characteristics can be the information between the spatial area information and fragmentation information of the media object Y01 in the dynamic media frame a and the spatial area information and fragmentation information of the media object Y01 in the reference media frame. Furthermore, the computer device may add the object change feature and the distribution change feature to the metadata track sample corresponding to the dynamic media frame a.

[0107] Specifically, the computer device encapsulates the object attribute characteristics and distribution characteristics of the media objects included in each dynamic media frame, and the change characteristics between the object attribute characteristics and distribution characteristics of the media objects included in the reference media frame, into the specific content of the corresponding metadata track sample (i.e., the asynchronous metadata track sample). Please refer to Table 4.

[0108] Table 4

[0109] Among them, num_object_update in Table 4 indicates the number of entries of changes in either object contained in the current metadata track sample relative to the synchronized metadata track sample corresponding to the reference media frame. When the num_object_update field is 0, it means that the media object contained in the current metadata track sample is the same as the synchronized metadata track sample corresponding to the reference media frame. When object_canceled_flag is 1, it indicates that the corresponding media object is no longer included in the current metadata track sample; when object_canceled_flag is 0, it indicates that the corresponding media object is included in the current metadata track sample, but has been updated. When object_spatial_info_flag is 0, it indicates that the specific spatial area information corresponding to the media object is not indicated; when object_spatial_info_flag is 1, it indicates that the specific spatial area information corresponding to the media object is indicated. When object_slice_info_flag is 0, it indicates that the slice information (such as point cloud slice information) corresponding to the media object is not indicated; when it is 1, it indicates that the slice information corresponding to the media object is indicated. 3DPoint is used to indicate the anchor coordinates of the corresponding spatial region, CuboidRegionStruct is used to indicate the size information of the corresponding spatial region, and SliceMapping is used to indicate the corresponding slice information.

[0110] Optionally, the computer device adds the object information data boxes and metadata tracks corresponding to the P media tracks to the encapsulated media file to obtain the target media file. A specific method may include: adding the object information data box j to the track sample entry of the media track j. Adding the metadata tracks corresponding to the P media tracks to the encapsulated media file to obtain the target media file.

[0111] Specifically, the computer device can add each object information data box to the track sample entry of the corresponding media track. Specifically, taking object information data box j as an example, the computer device can add object information data box j to the track sample entry of media track j. In this way, through the object information data box in the track sample entry of each media track, the object attribute characteristics of the media objects included in the media frames belonging to the media track can be known. For example, through the object information data box in the track sample entry of each media track, the media track that meets the conditions can be quickly located. For example, the media track that meets the conditions can refer to a media track that includes a specific media object, or the media track that meets the conditions can refer to a media track that does not include a media object. It can be seen that the efficiency of obtaining media data can be improved. At the same time, the computer device can add the metadata tracks corresponding to the P media tracks to the encapsulated media file to obtain the target media file. Because the metadata tracks corresponding to each of the P media tracks encapsulate the object attribute characteristics and distribution characteristics of the media objects included in the media frames belonging to each media track, the metadata tracks can be used to quickly locate media frames that meet certain conditions (such as those that include specific media objects or those that do not include media objects), or to quickly locate spatial regions within media frames that meet certain conditions, or to quickly locate the corresponding fragments of media frames that meet certain conditions. This shows that using metadata tracks can improve the efficiency of media data acquisition.

[0112] Optionally, the computer device adds the object information data boxes and metadata tracks corresponding to the P media tracks to the encapsulated media file to obtain the target media file. A specific method may include: adding object information data box j to the track sample entry of the metadata track corresponding to media track j to obtain the added metadata track corresponding to media track j. Adding the added metadata tracks corresponding to the P media tracks to the encapsulated media file to obtain the target media file.

[0113] Specifically, since each media track is associated with its corresponding metadata track, the computer device can also add the object information data box corresponding to each media track to the corresponding metadata track. Specifically, taking object information data box j as an example, the computer device can add object information data box j to the track sample entry of the metadata track corresponding to media track j, thereby obtaining the added metadata track corresponding to media track j. The added metadata tracks corresponding to each of the P media tracks are then added to the encapsulated media file to obtain the target media file. In this way, the object information data box in the track sample entry of the metadata track corresponding to each media track can be used to quickly locate metadata tracks that meet the requirements (e.g., metadata tracks that include specific media objects). Furthermore, based on the object attribute characteristics and distribution characteristics of the media objects included in each dynamic media frame encapsulated in the metadata track, media frames that meet the requirements (e.g., media frames that include specific media objects or media frames that do not include media objects, etc.) can be quickly located. Alternatively, spatial regions within media frames that meet the requirements can be quickly located, or fragments corresponding to media frames that meet the requirements can be quickly located. Thus, using metadata tracks can improve the efficiency of media data acquisition.

[0114] As shown in Figure 7, a schematic diagram of a target media file provided by an embodiment of the present application is shown. As shown in Figure 7, taking point cloud frames as an example, after acquiring N point cloud frames 70a, a computer device may encode these N point cloud frames 70a to obtain point cloud stream data 70b corresponding to these N point cloud frames 70a. Furthermore, the computer device may encapsulate the point cloud stream data 70b to obtain a target media file 70c. As shown in Figure 7, the target media file includes a point cloud media track, an object information metadata track, and an associated entity group box. The object information metadata track should be linked to the point cloud media track described by the object information metadata track using a track reference of the 'cdsc' type. Specifically, the computer device may encapsulate the point cloud stream data 70b in the point cloud media track. Simultaneously, the computer device may acquire object indication information for the N point cloud frames and encapsulate the object indication information for the N point cloud frames into the object information metadata track corresponding to the point cloud media track. The object indication information for one point cloud frame is encapsulated into one sample in the object information metadata track. At the same time, the object indication information may also include object relationship indication information, which is used to indicate media files with associated relationships, and the object relationship indication information is encapsulated in an associated entity group. For example, the encapsulated media file (i.e., point cloud media track) obtained by encapsulating the point cloud code stream data 70b includes point cloud media track G03 and point cloud media track G04, and the point cloud frames corresponding to point cloud media track G03 and point cloud media track G04 respectively include the same media objects, or include associated media objects. At the same time, when there is an object information metadata track Y03 corresponding to point cloud media track G03, and an object information metadata track Y04 corresponding to point cloud media track G04, the object relationship indication information includes an associated entity group for indicating that there is an associated relationship between point cloud media track G03, point cloud media track G04, object information metadata track Y03, and object information metadata track Y04. The associated entity group includes an entity group identifier, such as Z01, the number of entities (i.e., 4), and the entity identifiers corresponding to the point cloud media track G03, the point cloud media track G04, the object information metadata track Y03, and the object information metadata track Y04, respectively (i.e., G03, G04, Y03, Y04). Furthermore, the computer device can send the target media file 70c corresponding to the N point cloud frames to the decoding device 70d.

[0115] As shown in Figure 8, a schematic diagram of a target media file provided by an embodiment of the present application is shown. As shown in Figure 8, taking video frames as an example, after obtaining N video frames 80a, a computer device may encode the N video frames 80a to obtain video stream data 80b corresponding to the N video frames 80a. Furthermore, the computer device may encapsulate the video stream data 80b to obtain a target media file 80c. As shown in Figure 8, the target media file includes a video media track, an object information metadata track, and an associated entity group box. The object information metadata track should be associated with the video media track described by the object information metadata track using a track reference of the 'cdsc' type. Specifically, the computer device may encapsulate the video stream data 80b in the video media track. Simultaneously, the computer device may obtain object indication information for the N video frames and encapsulate the object indication information for the N video frames into the object information metadata track corresponding to the video media track. The object indication information for one video frame is encapsulated into one sample in the object information metadata track. At the same time, the object indication information may also include object relationship indication information, which is used to indicate media files with associated relationships, and the object relationship indication information is encapsulated in an associated entity group. For example, the encapsulated media file (i.e., video media track) obtained by encapsulating the video code stream data 80b includes video media track G05 and video media track G06, and the video frames corresponding to video media track G05 and video media track G06 respectively include the same media objects, or include associated media objects. At the same time, when there is an object information metadata track Y05 corresponding to video media track G05, and when there is an object information metadata track Y06 corresponding to video media track G06, the object relationship indication information includes an associated entity group for indicating that there is an associated relationship between video media track G05, video media track G06, object information metadata track Y05, and object information metadata track Y06. The associated entity group includes an entity group identifier, such as Z02, the number of entities (i.e., 4), and entity identifiers corresponding to the video media track G05, the video media track G06, the object information metadata track Y05, and the object information metadata track Y06 (i.e., G05, G06, Y05, Y06). Furthermore, the computer device can send the target media file 80c corresponding to the N video frames to the decoding device 80d.

[0116] Optionally, the N media frames include Q static media frames having media objects, and the encapsulated media file includes Q media data items corresponding to the Q static media frames; Q is a positive integer less than or equal to N. The specific manner in which the computer device sends the encapsulated media file and object indication information to the decoding device may include: obtaining object attribute characteristics and distribution characteristics of the media object in the static media frame corresponding to the media data item r from the object indication information; r is a positive integer less than or equal to Q. The object attribute characteristics and distribution characteristics of the media object in the static media frame corresponding to the media data item r are encapsulated into a data item attribute box associated with the media data item r. The data item attribute boxes corresponding to the Q media data items are added to the encapsulated media file to obtain a target media file, and the target media file is sent to the decoding device.

[0117] Specifically, when a computer device encapsulates the code stream data corresponding to N media frames, it can encapsulate the code stream data corresponding to Q static media frames containing media objects in the N media frames into media data items. That is, the encapsulated media file includes Q media data items corresponding to the Q static media frames, and the object indication information includes the object attribute characteristics and distribution characteristics of the media objects included in each static media frame. Specifically, the computer device can obtain the object attribute characteristics and distribution characteristics of the media objects in the static media frame corresponding to media data item r from the object indication information, and encapsulate the object attribute characteristics and distribution characteristics of the media objects in the static media frame corresponding to media data item r into a data item attribute box associated with media data item r. In this way, through the data item attribute box associated with each media data item, it is possible to quickly locate the media data item that meets the conditions, and it is also possible to quickly locate the spatial region within the media frame that meets the conditions, or it is possible to quickly locate the fragment corresponding to the media frame that meets the conditions. It can be seen that the data item attribute box can improve the efficiency of obtaining media data.

[0118] The data box type of the data item property box may be "obip (a data box type)", the property type of the data item property box may be a descriptive data item property, the data item property box may be contained in an ItemPropertyContainerBox (a data item property container box), and the number of data item property boxes corresponding to each media data item may be 0 or 1. The computer device may set the mandatory property corresponding to the data item property box to a non-mandatory property, that is, the data item property box may exist or may not exist.

[0119] Specifically, the computer device encapsulates the object attribute characteristics and distribution characteristics of the media objects in the static media frame corresponding to the media data item r into the data item attribute box associated with the media data item r. The specific content can be seen in Table 5.

[0120] Table 5

[0121] In Table 5, num_objects indicates the number of media object entries contained in the current object information data box. When object_oid_flag is 0, it indicates that the object description information about the media object in the current object information data box is indicated in the form of a human-readable string, i.e., the object description information of the media object is indicated in the form of a human-readable string. When object_oid_flag is 1, it indicates that the object description information about the media object in the current object information data box is indicated in the form of an OID, i.e., the object description information of the media object is indicated in the form of an OID. object_info_id indicates the identifier (i.e., object identifier) ​​of the corresponding media object entry, and object_oid indicates the object description information corresponding to the corresponding media object entry, indicated in the form of an OID. object_label indicates the object description information corresponding to the corresponding media object entry, indicated in the form of a human-readable string. When object_spatial_info_flag is 0, it indicates that the specific spatial region information corresponding to the media object is not indicated; when object_spatial_info_flag is 1, it indicates that the specific spatial region information corresponding to the media object is indicated. When object_slice_info_flag is set to 0, it indicates that the slice information (such as point cloud slice information) corresponding to the media object is not indicated; when it is set to 1, it indicates the slice information corresponding to the media object. 3DPoint is used to indicate the anchor coordinates of the corresponding spatial region, CuboidRegionStruct is used to indicate the size information of the corresponding spatial region, and SliceMapping is used to indicate the corresponding slice information.

[0122] Optionally, when the encapsulated media file of N media frames includes a first media file and a second media file, and a media object in a media frame corresponding to the first media file segment has an association relationship with a media object in a media frame corresponding to the second media file segment, the object indication information includes object relationship indication information; the object relationship indication information is used to indicate that the media object in the media frame corresponding to the first media file has an association relationship with the media object in the media frame corresponding to the second media file. The object indication information includes object attribute characteristics and object distribution characteristics of the media object in the media frame corresponding to the first media file, and object attribute characteristics and object distribution characteristics of the media object in the media frame corresponding to the second media file. In this case, the computer device may send the encapsulated media file and the object indication information to the decoding device in a specific manner including: encapsulating the object relationship indication information into an association entity group box; encapsulating the object attribute characteristics and object distribution characteristics of the media object in the media frame corresponding to the first media file into the first media file; encapsulating the object attribute characteristics and object distribution characteristics of the media object in the media frame corresponding to the second media file into the second media file; determining the association entity group box, the encapsulated first media file, and the encapsulated second media file as a target media file, and sending the target media file to the decoding device.

[0123] Specifically, the object indication information also includes object relationship indication information, which is used to indicate the association relationship between the media objects included in multiple media files. For example, the media frames corresponding to the multiple media file segments in the encapsulated media file respectively include the same media object, and the object relationship indication information can be used to indicate that the multiple media files including the same media object have an association relationship. Specifically, taking the encapsulated media file including the first media file and the second media file as an example, the object indication information includes the object attribute characteristics and object distribution characteristics of the media object in the media frame corresponding to the first media file, and the object attribute characteristics and object distribution characteristics of the media object in the media frame corresponding to the second media file. When there is an association relationship between the media object in the media frame corresponding to the first media file and the media object in the media frame corresponding to the second media file, the object indication information also includes object relationship indication information, and the object relationship indication information is used to indicate that there is an association relationship between the media object in the media frame corresponding to the first media file and the media object in the media frame corresponding to the second media file. The computer device can encapsulate the object relationship indication information into an association entity group box.

[0124] Specifically, the computer device may generate an associated entity group based on the object relationship indication information indicating the first media file and the second media file having an associated relationship, such as the associated entity group including the file identifiers of the first media file and the second media file, as well as the file quantities of the first media file and the second media file. Furthermore, the computer device may add the associated entity group to an associated entity group box. The first media file and the second media file may be one or more media file segments in an encapsulated media file, or any media track or media data item in the encapsulated media file. The associated entity group may include one or more media tracks, media data items, and metadata tracks corresponding to the first media file and the second media file, respectively.

[0125] For example, a first media file includes a media track G01, and the object attribute characteristics and distribution characteristics of the media objects included in the media frames belonging to the media track G01 are encapsulated in the metadata track Y01 corresponding to the media track G01. A second media file includes a media track G02, and the object attribute characteristics and distribution characteristics of the media objects included in the media frames belonging to the media track G02 are encapsulated in the metadata track Y02 corresponding to the media track G02. The computer device can generate an associated entity group based on the media track G01, the metadata track Y01, the media track G02, and the metadata track Y02. For example, the associated entity group includes the identifiers corresponding to the media track G01, the metadata track Y01, the media track G02, and the metadata track Y02, respectively, as well as the total number of the media track G01, the metadata track Y01, the media track G02, and the metadata track Y02. Among them, the data box type of the associated entity group box can be "obje (a data box type)", the associated entity group box is included in the GroupsListBox (entity group list box), the number of the associated entity group boxes can be 0 or 1, and the computer device can set the mandatory characteristics corresponding to the associated entity group box to non-mandatory characteristics, that is, the associated entity group box may exist or may not exist.

[0126] Specifically, the computer device encapsulates the object relationship indication information into the associated entity group box. The specific content can be seen in Table 6.

[0127] Table 6

[0128] In Table 6, group_id indicates the identifier of the current associated entity group, num_entities_in_group indicates the number of entities (media tracks, metadata tracks, or media data items) in the current associated entity group, and entity_id indicates the identifier of the entity (media track, metadata track, or data item).

[0129] At the same time, the computer device may encapsulate the object attribute characteristics and object distribution characteristics of the media objects in the media frames corresponding to the first media file into the first media file, and encapsulate the object attribute characteristics and object distribution characteristics of the media objects in the media frames corresponding to the second media file into the second media file. For details, see the encapsulation content of the object indication information corresponding to the media track or media data item. Furthermore, the computer device may determine the associated entity group box, the encapsulated first media file, and the encapsulated second media file as target media files, and send the target media files to the decoding device.

[0130] In an embodiment of the present application, after the encoding device obtains the encapsulated media files corresponding to N media frames, when the N media files include media objects, it can generate object indication information about the N media frames, and the object indication information is used to reflect the object attribute characteristics of the media objects included in the N media frames, as well as the distribution characteristics of the media objects in the N media frames. That is, the object indication information is used to reflect what type of media objects are included in the N media frames, which media frames include media objects, and which positions of the media frames include media objects, etc. Based on the object indication information, the media frames (i.e., media data) required by the decoding device can be quickly understood from the encapsulated media files. The object indication information and the encapsulated media files are sent to the decoding device. The decoding device can quickly obtain the media file segments to be decoded from the encapsulated media files corresponding to the N media frames based on the object indication information. The media file segments to be decoded can refer to the media file segments corresponding to the media data required by the decoding device, which can improve the efficiency of obtaining the media file segments that need to be decoded. Furthermore, only the media file fragments to be decoded need to be decoded to obtain the media data corresponding to the media file fragments to be decoded (such as the media data required by the decoding device), without the need to decode the entire media file. This can reduce the amount of decoded data, improve the efficiency of obtaining media data, and reduce the resource overhead (such as computing resources) of the decoding device.

[0131] Further, please refer to Figure 9, which is a flowchart of a media data processing method provided by an embodiment of the present application. As shown in Figure 9, the method can be executed by a decoding device, wherein the method may include but is not limited to the following steps:

[0132] S201: Receive object indication information.

[0133] Specifically, the computer device can receive object indication information, and the object indication information is used to reflect the object attribute characteristics of the media objects included in the N media frames, as well as the distribution characteristics of the media objects in the N media frames, where N is a positive integer. Specifically, the embodiment of the present application can be applied to point cloud data scenarios, that is, the N media frames can be point cloud frames in point cloud data. Of course, it can also be applied to other types of media scenarios, such as video data scenarios, that is, the N media frames can be video frames in video data. Among them, media objects can refer to items (such as cups, chairs, etc.), virtual props, virtual characters, people, animals, etc. in the media frame. It can be understood that the media objects included in each media frame can be quickly determined through the object indication information. In this way, it is possible to quickly locate a specific media frame (such as a media frame including a specific media object), and then quickly obtain the media data corresponding to the specific media frame, which can improve the efficiency of obtaining media data. The object attribute characteristics and distribution characteristics of the media objects included in each of the N media frames may be obtained by performing object identification on the N media frames during the acquisition phase of the N media frames, or may be obtained by performing algorithmic analysis on the code stream data corresponding to the N media frames, or may be obtained by other means, and this embodiment of the present application does not impose any restrictions on this. The object indication information may be sent by the encoding device or by other devices.

[0134] Optionally, the object attribute characteristics in the object indication information may include one or more of the number of objects, object identifiers, and object description information of the media objects included in the N media frames. The object identifier may be an OID identifier (i.e., object identifiers, which is an identifier mechanism standardized by the International Telecommunication Union (ITU) and the International Organization for Standardization (ISO / IEC) and is used to name any object, concept, or "thing" with a clear and permanent name worldwide). The specific object information of the corresponding media object can be clearly determined through the OID identifier. The object description information may refer to the object specific information of the media object described in the form of a human-readable character string, such as object structure, object color, object material, object function, etc.

[0135] Optionally, the distribution feature in the object indication information is used to indicate the target media frame with the media object in N media frames, that is, the distribution feature can be used to indicate the target media frame to which the media object included in the N media frames belongs. Alternatively, the distribution feature is used to indicate the target media frame with the media object in N media frames, and the spatial region to which the media object belongs in the target media frame, that is, the media object included in the spatial region in each media frame. Alternatively, the distribution feature is used to indicate the target media frame with the media object in N media frames, and the slice to which the media object belongs in the target media frame. Alternatively, the distribution feature is used to indicate the target media frame with the media object in N media frames, and the spatial region to which the media object belongs in the target media frame, and the slice to which the media object belongs in the target media frame. It can be understood that the distribution feature can be used to indicate one or more of the information such as the media objects included in each media frame, the media objects included in the spatial region within each media frame, and the slice to which the media objects included in each media frame are located. In this way, through the object indication information about N media frames, the media data of a specific media frame (such as a media frame including a specific media object) can be quickly located, the media data of a specific slice (such as a point cloud slice or video slice including a specific media object) can be quickly located, and the media data of a specific spatial area (such as a spatial area including a specific media object) can be quickly located, thereby improving the efficiency of obtaining media data.

[0136] S202 : Acquire a media file segment to be decoded from the encapsulated media file corresponding to the N media frames according to the object indication information.

[0137] Specifically, a computer device can obtain media file segments to be decoded from an encapsulated media file corresponding to N media frames based on the object indication information. The encapsulated media file corresponding to the N media frames is obtained by encapsulating the codestream data corresponding to the N media frames, and the codestream data corresponding to the N media frames is obtained by encoding the N media frames. The encapsulated media file for the N media frames may be sent by an encoding device, and the decoding device can obtain the media file segments to be decoded from the encapsulated media file based on the object indication information. Because the object indication information indicates the object attribute characteristics of the media objects included in the N media frames and the distribution characteristics of the media objects within the N media frames, the media file segments to be decoded can be obtained from the encapsulated media file corresponding to the N media frames based on the object indication information. In this way, the required media data can be obtained by only decoding the media file segments to be decoded, eliminating the need to decode the encapsulated media file corresponding to the N media frames. This reduces decoding computational complexity and improves the efficiency of obtaining media data (specific media data).

[0138] Optionally, the media file segment to be decoded is codestream data that includes a target media object; or, the media file segment to be decoded is codestream data that does not include a target media object. The target media object is a media object in N media frames, such as a media object with a target color, a media object with a target identifier, a media object with a target function, or multiple media objects with an associated relationship. Multiple media objects with an associated relationship may refer to multiple media objects with a binding relationship, or multiple media objects that display or move together. Alternatively, the media file segment to be decoded is codestream data that does not include a media object. Thus, by obtaining the media file segment to be decoded using object indication information, only the corresponding media file to be decoded needs to be decoded to obtain specific media data (e.g., media data that includes the target object, media data that does not include the target media object, or media data that does not include a media object). This reduces the amount of decoded data and improves the efficiency of media data acquisition. At the same time, it is possible to support interaction with the target media object on the basis of presenting the media data corresponding to the file to be decoded.

[0139] Optionally, the object indication information includes object indication information associated with S media file segments, each of which belongs to an encapsulated media file. A specific method for a computer device to obtain the media file segment to be decoded from the encapsulated media file corresponding to N media frames based on the object indication information may include obtaining segment identifiers corresponding to the S media file segments. If the object indication information of a target media file segment among the S media file segments indicates that the target media file segment meets the decoding condition, a request for obtaining the target media file segment is generated based on the segment identifier of the target media file segment. The request is sent to an encoding device, which receives the target media file segment returned by the encoding device based on the request and determines the target media file segment as the media file segment to be decoded.

[0140] Specifically, the encoding device can generate a descriptive signaling file based on the object indication information corresponding to each of the S media file segments and the segment identifiers corresponding to each of the S media file segments, and send the descriptive signaling file to the decoding device. The object indication information corresponding to each of the S media file segments can be used to indicate object attribute characteristics of the media objects in the media frames respectively included in the S media file segments. After the computer device obtains the object indication information corresponding to each of the S media file segments and the segment identifiers corresponding to each of the S media file segments from the descriptive signaling file, it can determine whether the S media file segments meet decoding conditions based on the object indication information corresponding to each of the S media file segments. If the object indication information of the target media file segment among the S media file segments indicates that the target media file segment meets the decoding conditions, a request to obtain the target media file segment is generated based on the segment identifier of the target media file segment.

[0141] Furthermore, the computer device can send an acquisition request to the encoding device. After receiving the acquisition request for the target media file segment, the encoding device can return the target media file segment to the decoding device. The decoding device can receive the target media file segment returned by the encoding device based on the acquisition request and determine the target media file segment as the media file segment to be decoded. As can be seen, the encoding device only needs to send the target media file segment to the decoding device, rather than sending S media file segments to the decoding device, which can reduce data transmission pressure and improve data transmission efficiency. In addition, the decoding device only needs to decode the target media file segment to obtain the required media data, without having to obtain the encapsulated media file corresponding to N media frames. Decoding the encapsulated media file can reduce the amount of decoded data and also improve the efficiency of media data acquisition.

[0142] Optionally, the encoding device may directly send the object indication information to the decoding device. Optionally, the computer device may specifically receive the object indication information sent by the encoding device by receiving a target media file sent by the encoding device, where the target media file is obtained by the encoding device adding the object indication information to an encapsulated media file. The computer device may decapsulate the target media file to obtain the object indication information for N media frames, and also obtain the encapsulated media file for the N media frames.

[0143] Optionally, the encapsulated media file includes P media tracks belonging to K dynamic media frames among N media frames, where P is a positive integer and K is a positive integer less than or equal to N. The specific method in which the computer device obtains the media file segment to be decoded from the encapsulated media file corresponding to the N media frames according to the object indication information may include: obtaining the object attribute characteristics of the media object in the dynamic media frame belonging to the media track j from the object information data box j corresponding to the media track j; j is a positive integer less than or equal to P. According to the object attribute characteristics corresponding to the P media tracks respectively, the target media track that meets the decoding conditions is determined from the P media tracks. According to the object indication information corresponding to the dynamic media frame belonging to the target media track, the media file segment to be decoded is determined; the object indication information of the dynamic media frame belonging to the target media track is encapsulated in the metadata track corresponding to the target media track.

[0144] Specifically, the encapsulated media file includes P media tracks belonging to K dynamic media frames among N media frames. Each media track includes a track sample entry and one or more track samples, and a track sample is used to encapsulate the codestream data corresponding to a dynamic media frame. A computer device can obtain an object information data box j corresponding to media track j in the P media. Each media track can correspond to an object information data box, which encapsulates object attribute characteristics of the media objects in the media frame belonging to the corresponding media track. The object information data box j can be located in a track sample entry in media track j or in a track sample entry in a metadata track corresponding to media track j. The computer device can obtain the object attribute characteristics of the media objects in the dynamic media frame belonging to media track j from the object information data box j corresponding to media track j. Furthermore, the computer device can determine a target media track from the P media tracks that meets decoding requirements based on the object attribute characteristics corresponding to the P media tracks. In this way, based on the object attribute characteristics of the media objects of the corresponding media track encapsulated in the object information data box, the computer device can quickly locate the media track that meets the decoding requirements from among the P media tracks, thereby improving the efficiency of media data acquisition.

[0145] Furthermore, the object indication information also includes object attribute characteristics and distribution characteristics of the media objects in the media frames belonging to each media track. These object attribute characteristics and distribution characteristics of the media objects in the media frames belonging to each media track are encapsulated in the metadata track associated with the corresponding media track. Taking media track j as an example, the object attribute characteristics and distribution characteristics of the media objects in the media frames belonging to media track j are encapsulated in the metadata track associated with media track j. The metadata track includes a track sample entry and one or more track samples. A track sample in the metadata track is used to encapsulate the object attribute characteristics and distribution characteristics of the media objects in a media frame. The track samples in media track j correspond one-to-one with the track samples in the metadata track associated with media track j. The computer device can obtain object indication information corresponding to the dynamic media frames belonging to the target media track from the metadata track corresponding to the target media track. Based on the object indication information corresponding to the dynamic media frames belonging to the target media track, the computer device determines the media file segment to be decoded. The object indication information of the dynamic media frames belonging to the target media track is encapsulated in the metadata track corresponding to the target media track.

[0146] Optionally, the metadata track corresponding to the media track may include synchronous track samples and asynchronous track samples. The synchronous track samples encapsulate the object attribute characteristics and distribution characteristics of the media objects in the media frames belonging to the corresponding media track, while the asynchronous track samples encapsulate the object change characteristics and distribution change characteristics between the media frames belonging to the corresponding media track and the reference media frame. Object change characteristics refer to the change characteristics between the object attribute characteristics of the media objects in the media frames belonging to the corresponding media track and the object attribute characteristics of the media objects in the reference media frame, and object change characteristics refer to the change characteristics between the distribution characteristics of the media objects in the media frames belonging to the corresponding media track and the distribution characteristics of the media objects in the reference media frame. Taking the example of obtaining object indication information corresponding to dynamic media frame a belonging to media track j, the computer device can determine the track sample corresponding to dynamic media frame a from the track samples of the metadata track corresponding to media track j as the target track sample. When the target track sample encapsulates the object attribute characteristics and encapsulation characteristics of the media objects included in dynamic media frame a, the object indication information stored in the target track sample is directly used as the object indication information corresponding to dynamic media frame a. When the target track sample encapsulates the object change characteristics and distribution change characteristics corresponding to the media objects included in dynamic media frame a, the computer device can obtain the reference media frame corresponding to dynamic media frame a, as well as the object attribute characteristics and distribution characteristics of the media objects in the reference media frame. Furthermore, the computer device can determine the object indication information corresponding to dynamic media frame a based on the object attribute characteristics and distribution characteristics of the media objects in the reference media frame and the object change characteristics and distribution change characteristics stored in the target track sample. In this way, by storing only the change characteristics in the asynchronous track sample, duplicate storage can be avoided, the amount of stored data can be reduced, and storage pressure can be reduced.

[0147] Optionally, the computer device may determine the media file segment to be decoded based on the object indication information corresponding to the dynamic media frame in the target media track by: determining a target dynamic media frame that meets the decoding condition from the dynamic media frames in the target media track based on the object indication information corresponding to the dynamic media frame in the target media track; and determining the media file segment to be decoded based on the codestream data of the target dynamic media frame in the target media track and the object indication information of the target dynamic media frame.

[0148] Specifically, the computer device can determine the target dynamic media frame that meets the decoding conditions from the dynamic media frames belonging to the target media track based on the object indication information corresponding to the dynamic media frame belonging to the target media track. For example, a dynamic media frame including the target media object is obtained, or a dynamic media frame not including the target media object is obtained, etc. Further, the object indication information corresponding to the dynamic media frame can be an object attribute feature, and the object attribute feature can be an object identifier or object description information. According to the object attribute feature of the dynamic media frame belonging to the target media track, the target dynamic media frame can be determined from the dynamic media frames belonging to the target media track. Furthermore, the computer device can determine the media file segment to be decoded based on the code stream data about the target dynamic media frame in the target media track and the object indication information of the target dynamic media frame. It can be seen that the target media frame can be quickly located through the object attribute feature of the media object included in each media frame, which can improve the efficiency of obtaining media data.

[0149] Optionally, the computer device may determine the media file segment to be decoded based on the codestream data of the target dynamic media frame in the target media track and the object indication information of the target dynamic media frame, by: determining, based on the object indication information of the target dynamic media frame, a segment that meets the decoding condition from the target dynamic media frame as the first segment; and determining the codestream data of the first segment in the codestream data corresponding to the target dynamic media frame as the media file segment to be decoded.

[0150] Specifically, the object indication information of the target dynamic media frame may include the distribution characteristics of the media object in the target dynamic media frame, such as the spatial area to which the media object belongs in the target dynamic media frame, the slice (such as a point cloud slice or a video slice) to which the media object belongs in the target dynamic media frame, and one spatial area corresponds to one slice. The computer device can determine the slice that meets the decoding conditions from the target dynamic media frame based on the spatial area to which the media object belongs in the target dynamic media frame and the slice to which the media object belongs in the target dynamic media frame. As the first slice, the code stream data corresponding to the target dynamic media frame includes the code stream data of the first slice. The computer device can determine the code stream data about the first slice in the code stream data corresponding to the target dynamic media frame as the media file segment to be decoded. It can be seen that the code stream data corresponding to the slice that meets the decoding conditions can be quickly located through the distribution characteristics of the media objects in the media frame. Only the code stream data corresponding to the slice needs to be decoded, which can reduce the amount of decoded data and improve the efficiency of obtaining media data.

[0151] Optionally, the encapsulated media file includes Q media data items corresponding to Q static media frames among the N media frames, and the object indication information includes object indication information corresponding to each of the Q media data items, where Q is a positive integer less than or equal to N. A specific method in which the computer device obtains the to-be-decoded media file segment from the encapsulated media file corresponding to the N media frames based on the object indication information may include: determining, based on the object indication information corresponding to each of the Q media data items, a target media data item that meets a decoding condition from the Q media data items; and determining the to-be-decoded media file segment based on the target media data item and the object indication information corresponding to the target media data item.

[0152] Specifically, the media data items are obtained by encapsulating the code stream data corresponding to the static media frames. The object indication information corresponding to each media data item may include the object attribute characteristics and distribution of the media objects included in the static media frames associated with the corresponding media data item. Based on the object indication information corresponding to each of the Q media data items, the computer device can determine the target media data item that meets the decoding criteria from the Q media data items. Specifically, the object indication information may include the object attribute characteristics of the media objects, which may include the number of objects, object identifiers, and object description information. Based on the object attribute characteristics corresponding to each of the Q media data items, the computer device can determine the target media data item that meets the decoding criteria from the Q media data items. This may include a target media data item that includes the target media object, or a target media data item that does not include the target media object. Furthermore, the computer device can determine the media file segment to be decoded based on the target media data item and the object indication information corresponding to the target media data item. Thus, the object indication information corresponding to the media data item can quickly locate the target media data item, improving the efficiency of media data acquisition.

[0153] Optionally, the computer device may determine the media file segment to be decoded based on the target media data item and the object indication information corresponding to the target media data item in a specific manner including: determining, based on the object indication information corresponding to the target media data item, a point cloud segment that meets the decoding condition from a static media frame corresponding to the target media data item as a second point cloud segment; determining code stream data corresponding to the second point cloud segment from the target media data item, and determining the code stream data corresponding to the second point cloud segment as the media file segment to be decoded.

[0154] Specifically, the object indication information corresponding to the target media data item may include the distribution characteristics of the media objects included in the static media frame corresponding to the target media data item in the corresponding static media frame, such as the spatial area to which the media object belongs in the corresponding static media frame, the slice (such as a point cloud slice or a video slice) to which the media object belongs in the corresponding static media frame, and one spatial area corresponds to one slice. The computer device can determine the slice that meets the decoding conditions from the static media frame corresponding to the target media data item based on the spatial area to which the media object belongs in the corresponding static media frame and the slice to which the media object belongs in the corresponding static media frame. As the second slice, the target media data item includes the code stream data of the second slice. The computer device can determine the code stream data about the second slice in the code stream data corresponding to the target media data item as the media file segment to be decoded. It can be seen that the code stream data corresponding to the slice that meets the decoding conditions can be quickly located through the distribution characteristics of the media objects in the media frame. Only the code stream data corresponding to the slice needs to be decoded, which can reduce the amount of decoded data and improve the efficiency of obtaining media data.

[0155] Optionally, the encapsulated media file includes a first media file and a second media file, and the object indication information includes object indication information of the first media file, object indication information of the second media file, and object relationship indication information. The object relationship indication information is used to indicate that a media object in a media frame corresponding to the first media file is associated with a media object in a media frame corresponding to the second media file. The object indication information of the first media file is encapsulated in the first media file, and the object indication information of the second media file is encapsulated in the second media file. A specific method for a computer device to obtain a media file segment to be decoded from the encapsulated media files corresponding to N media frames based on the object indication information may include: upon determining that a decoding condition is met in the first media file, obtaining a second media file associated with the first media file based on the object relationship indication information. Determining the media file segment to be decoded based on the first media file and the second media file.

[0156] Specifically, the object indication information also includes object relationship indication information, and the object relationship indication information is used to indicate the association relationship between the media objects included in the multiple media files. For example, the media frames corresponding to the multiple media file segments in the encapsulated media file respectively include the same media object, and the object relationship indication information can be used to indicate that the multiple media files including the same media object have an association relationship. Specifically, taking the encapsulated media file including the first media file and the second media file as an example, when there is an association relationship between the media object in the media frame corresponding to the first media file and the media object in the media frame corresponding to the second media file, the object relationship indication information is used to indicate that there is an association relationship between the media object in the media frame corresponding to the first media file and the media object in the media frame corresponding to the second media file.

[0157] Specifically, upon determining that a first media file meets a decoding condition, the computer device retrieves a second media file associated with the first media file based on the object relationship indication information. Based on the object indication information corresponding to the first media file and the object indication information corresponding to the second media file, the computer device can determine a media file segment to be decoded from the first media file and the second media file, such as a media file segment in the first media file and the second media file that includes a target media object, or a media file segment in the first media file and the second media file that includes associated media objects (e.g., multiple media objects in a binding relationship). Thus, the object relationship indication information can be used to quickly retrieve different media files that have an associated relationship, thereby enabling joint presentation or associated presentation of the different media files that have an associated relationship.

[0158] S203: Decode the media file segment to be decoded to obtain media data corresponding to the media file segment to be decoded.

[0159] Specifically, the computer only needs to decode the media file segment to be decoded to obtain the media data corresponding to the media file segment to be decoded, and then present the media data corresponding to the media file segment to be decoded. The media data corresponding to the media file segment to be decoded may refer to media frame data including the target media object. On the basis of presenting the media data of the target media object, interaction with the target media object may also be supported. The media data corresponding to the media file segment to be decoded may refer to fragmented data including the target media object. Only the fragmented data needs to be decoded and presented, which can reduce the resource (such as computing resource) overhead of the decoding device. The media data corresponding to the media file segment to be decoded may refer to multiple media files (such as media tracks or media data items) with an associated relationship, which can realize the joint presentation or associated presentation of different media files. It can be seen that through the object indication information, the required media data can be quickly obtained from the encapsulated media file, which can improve the efficiency of obtaining media data.

[0160] In an embodiment of the present application, a decoding device can receive object indication information sent by an encoding device. The object indication information is used to reflect the object attribute characteristics of the media objects included in N media frames, as well as the distribution characteristics of the media objects in the N media frames. That is, the object indication information is used to reflect what type of media objects are included in the N media frames, which media frames include media objects, and which positions in the media frames include media objects. Based on the object indication information, the decoding device can quickly understand the media frames (i.e., media data) it needs. Therefore, based on the object indication information, it can quickly obtain the media file segments to be decoded from the encapsulated media files corresponding to the N media frames. The media file segments to be decoded can refer to the media file segments corresponding to the media data required by the decoding device, which can improve the efficiency of obtaining the media file segments to be decoded. Furthermore, only the media file segments to be decoded need to be decoded to obtain the media data corresponding to the media file segments to be decoded (such as the media data required by the decoding device), without having to decode the entire media file. This can reduce the amount of decoded data, improve the efficiency of obtaining media data, and reduce the resource (such as computing resource) overhead of the decoding device.

[0161] Please refer to Figure 10, which is a schematic diagram of the structure of a media data processing device provided by an embodiment of the present application. As shown in Figure 10, the media data processing device may include: a receiving module 11, a first obtaining module 12 and a decoding module 13.

[0162] The receiving module 11 is configured to receive object indication information; the object indication information is used to reflect object attribute characteristics of the media objects included in the N media frames, as well as distribution characteristics of the media objects in the N media frames; N is a positive integer;

[0163] A first acquisition module 12 is configured to acquire a media file segment to be decoded from the encapsulated media file corresponding to the N media frames according to the object indication information;

[0164] The decoding module 13 is configured to decode the media file segments to be decoded to obtain media data corresponding to the media file segments to be decoded.

[0165] The object attribute feature includes one or more of the number of media objects included in the N media frames, object identifiers, and object description information.

[0166] The distribution feature is used to indicate a target media frame having a media object among the N media frames; or

[0167] The distribution feature is used to indicate a target media frame having a media object among the N media frames, and a spatial region to which the media object belongs in the target media frame; or,

[0168] The distribution feature is used to indicate the target media frame having the media object among the N media frames, and the fragment to which the media object belongs in the target media frame; or,

[0169] The distribution feature is used to indicate the target media frame having the media object among the N media frames, as well as the spatial region to which the media object belongs in the target media frame, and the slice to which the media object belongs in the target media frame.

[0170] The media file segment to be decoded is code stream data including a target media object; or the media file segment to be decoded is code stream data not including a target media object; and the target media object belongs to a media object in N media frames.

[0171] The object indication information includes object indication information associated with each of the S media file segments; the S media file segments belong to an encapsulated media file;

[0172] The first acquisition module 12 includes:

[0173] The first obtaining unit 1201 is configured to obtain segment identifiers corresponding to the S media file segments.

[0174] A first generating unit 1202 is configured to generate an acquisition request for the target media file segment according to the segment identifier of the target media file segment if the object indication information of the target media file segment among the S media file segments indicates that the target media file segment meets the decoding condition;

[0175] The first receiving unit 1203 is configured to send an acquisition request to the encoding device and receive a target media file segment returned by the encoding device based on the acquisition request;

[0176] The first determining unit 1204 is configured to determine the target media file segment as the media file segment to be decoded.

[0177] The receiving module 11 includes:

[0178] The second receiving unit 1101 is configured to receive a target media file sent by an encoding device;

[0179] The decapsulation unit 1102 is configured to decapsulate the target media file to obtain object indication information about N media frames.

[0180] The encapsulated media file includes P media tracks to which K dynamic media frames in N media frames belong; P is a positive integer, and K is a positive integer less than or equal to N;

[0181] The first acquisition module 12 includes:

[0182] The second obtaining unit 1205 is configured to obtain object attribute characteristics of the media object in the dynamic media frame belonging to the media track j from the object information data box j corresponding to the media track j; j is a positive integer less than or equal to P;

[0183] A second determining unit 1206 is configured to determine a target media track that meets a decoding condition from the P media tracks based on object attribute features corresponding to the P media tracks;

[0184] The third determining unit 1207 is configured to determine the media file segment to be decoded based on the object indication information corresponding to the dynamic media frame in the target media track; the object indication information of the dynamic media frame in the target media track is encapsulated in the metadata track corresponding to the target media track.

[0185] The third determining unit 1207 is specifically configured to:

[0186] Determining, according to the object indication information corresponding to the dynamic media frames belonging to the target media track, a target dynamic media frame that meets the decoding condition from the dynamic media frames belonging to the target media track;

[0187] The media file segment to be decoded is determined according to the code stream data about the target dynamic media frame in the target media track and the object indication information of the target dynamic media frame.

[0188] The method of determining the media file segment to be decoded according to the code stream data of the target dynamic media frame in the target media track and the object indication information of the target dynamic media frame includes:

[0189] Determining, according to the object indication information of the target dynamic media frame, a fragment satisfying a decoding condition from the target dynamic media frame as a first fragment;

[0190] The code stream data about the first fragment in the code stream data corresponding to the target dynamic media frame is determined as the media file segment to be decoded.

[0191] The encapsulated media file includes Q media data items corresponding to Q static media frames in the N media frames, and the object indication information includes object indication information corresponding to each of the Q media data items; Q is a positive integer less than or equal to N;

[0192] The first acquisition module 12 includes:

[0193] A fourth determining unit 1208 is configured to determine a target media data item that meets a decoding condition from the Q media data items according to the object indication information corresponding to each of the Q media data items;

[0194] The fifth determining unit 1209 is configured to determine a media file segment to be decoded according to the target media data item and the object indication information corresponding to the target media data item.

[0195] The fifth determining unit 1209 is specifically configured to:

[0196] Determining, according to the object indication information corresponding to the target media data item, a point cloud slice that meets the decoding condition from the static media frame corresponding to the target media data item as the second point cloud slice;

[0197] Determine the code stream data corresponding to the second point cloud piece from the target media data item, and determine the code stream data corresponding to the second point cloud piece as the media file segment to be decoded.

[0198] The encapsulated media file for the N media frames includes a first media file and a second media file, and the object indication information includes object indication information of the first media file, object indication information of the second media file, and object relationship indication information; the object relationship indication information is used to indicate that a media object in a media frame corresponding to the first media file is associated with a media object in a media frame corresponding to the second media file, the object indication information of the first media file is encapsulated in the first media file, and the object indication information of the second media file is encapsulated in the second media file;

[0199] The first acquisition module 12 includes:

[0200] The third acquiring unit 1210 is configured to acquire, when it is determined that the first media file meets the decoding condition, a second media file associated with the first media file according to the object relationship indication information;

[0201] The sixth determining unit 1211 is configured to determine a media file segment to be decoded according to the first media file and the second media file.

[0202] In an embodiment of the present application, after the encoding device obtains the encapsulated media files corresponding to N media frames, when the N media files include media objects, it can generate object indication information about the N media frames, and the object indication information is used to reflect the object attribute characteristics of the media objects included in the N media frames, as well as the distribution characteristics of the media objects in the N media frames. That is, the object indication information is used to reflect what type of media objects are included in the N media frames, which media frames include media objects, and which positions of the media frames include media objects, etc. Based on the object indication information, the media frames (i.e., media data) required by the decoding device can be quickly understood from the encapsulated media files. The object indication information and the encapsulated media files are sent to the decoding device. The decoding device can quickly obtain the media file segments to be decoded from the encapsulated media files corresponding to the N media frames based on the object indication information. The media file segments to be decoded can refer to the media file segments corresponding to the media data required by the decoding device, which can improve the efficiency of obtaining the media file segments that need to be decoded. Furthermore, only the media file fragments to be decoded need to be decoded to obtain the media data corresponding to the media file fragments to be decoded (such as the media data required by the decoding device), without the need to decode the entire media file. This can reduce the amount of decoded data, improve the efficiency of obtaining media data, and reduce the resource overhead (such as computing resources) of the decoding device.

[0203] Please refer to Figure 11, which is a schematic diagram of the structure of a media data processing device provided in an embodiment of the present application. As shown in Figure 11, the media data processing device may include: a second acquisition module 21, a generation module 22, and a sending module 23.

[0204] The second acquisition module 21 is configured to acquire an encapsulated media file of N media frames, where N is a positive integer;

[0205] A generating module 22 is configured to generate object indication information regarding the N media frames if the N media frames include media objects; the object indication information is configured to indicate object attribute characteristics corresponding to the media objects included in the N media frames, as well as distribution characteristics of the media objects in the N media frames;

[0206] The sending module 23 is configured to send the encapsulated media file and object indication information to a decoding device.

[0207] The sending module 23 includes:

[0208] The extraction unit 2301 is configured to extract object indication information respectively associated with the S media file segments from the object indication information if the encapsulated media file includes S media file segments; S is an integer greater than 1;

[0209] The first encapsulation unit 2302 is configured to encapsulate object indication information associated with a media file segment i among the S media file segments into the media file segment i to obtain a target media file segment i, where S is an integer greater than 1 and i is a positive integer less than or equal to S.

[0210] The first sending unit 2303 is configured to send object indication information respectively associated with the S media file segments and segment identifiers respectively corresponding to the S media file segments to a decoding device;

[0211] The second sending unit 2304 is configured to send the target media file segment i to the decoding device if an acquisition request for the target media file segment i is received; the acquisition request is generated by the decoding device based on the object indication information and segment identifiers respectively associated with the S media file segments.

[0212] The sending module 23 includes:

[0213] The second encapsulation unit 2305 is configured to encapsulate the object indication information into an encapsulated media file to obtain a target media file;

[0214] The third sending unit 2306 is configured to send the target media file to a decoding device.

[0215] The N media frames include K dynamic media frames having media objects, and the encapsulated media file includes P media tracks to which the K dynamic media frames belong; P is a positive integer, and K is a positive integer less than or equal to N;

[0216] The sending module 23 includes:

[0217] The fourth obtaining unit 2307 is configured to obtain object attribute characteristics and distribution characteristics of the media objects included in the dynamic media frame belonging to the media track j from the object indication information; j is a positive integer less than or equal to P;

[0218] The third encapsulation unit 2308 is configured to encapsulate object attribute characteristics of the media objects in the dynamic media frame belonging to the media track j into the object information data box j associated with the media track j;

[0219] The fourth encapsulation unit 2309 is configured to encapsulate the object attribute characteristics and distribution characteristics of the media objects included in the dynamic media frame belonging to the media track j into the metadata track corresponding to the media track j;

[0220] The first adding unit 2310 is configured to add the object information data boxes and metadata tracks corresponding to the P media tracks respectively to the encapsulated media file to obtain a target media file, and send the target media file to a decoding device.

[0221] The metadata track corresponding to the media track j includes metadata track samples corresponding to the dynamic media frames belonging to the media track j;

[0222] The fourth encapsulation unit 2309 is specifically used for:

[0223] The object attribute characteristics and distribution characteristics of the media objects included in the dynamic media frame a belonging to the media track j are added to the metadata track sample corresponding to the dynamic media frame a; a is a positive integer less than or equal to the total number of dynamic media frames belonging to the media track j.

[0224] The metadata track corresponding to the media track j includes metadata track samples corresponding to the dynamic media frames belonging to the media track j;

[0225] The fourth encapsulation unit 2309 is specifically used for:

[0226] Obtain object attribute characteristics and distribution characteristics of media objects in a reference media frame corresponding to a dynamic media frame belonging to media track j; a is a positive integer less than or equal to the total number of dynamic media frames belonging to media track j;

[0227] determining an object change characteristic between an object attribute characteristic of a media object in a reference media frame and an object attribute characteristic of a dynamic media frame a;

[0228] determining a distribution change characteristic between a distribution characteristic of media objects in a reference media frame and a distribution characteristic of a dynamic media frame a;

[0229] The object change feature and the distribution change feature are added to the metadata track sample corresponding to the dynamic media frame a.

[0230] The first adding unit 2310 is specifically configured to:

[0231] Add object information data box j to the track sample entry of media track j;

[0232] The metadata tracks corresponding to the P media tracks are added to the encapsulated media file to obtain the target media file.

[0233] The first adding unit 2310 is specifically configured to:

[0234] Add object information data box j to the track sample entry of the metadata track corresponding to media track j to obtain the added metadata track corresponding to media track j;

[0235] The metadata tracks corresponding to the P media tracks are added to the encapsulated media file to obtain the target media file.

[0236] The N media frames include Q static media frames having media objects, and the encapsulated media file includes Q media data items corresponding to the Q static media frames; Q is a positive integer less than or equal to N;

[0237] The sending module 23 includes:

[0238] The fifth obtaining unit 2311 is configured to obtain object attribute characteristics and distribution characteristics of the media object in the static media frame corresponding to the media data item r from the object indication information; r is a positive integer less than or equal to Q;

[0239] The fifth encapsulation unit 2312 is configured to encapsulate the object attribute characteristics and distribution characteristics of the media objects in the static media frame corresponding to the media data item r into a data item attribute box associated with the media data item r;

[0240] The second adding unit 2313 is configured to add the data item attribute boxes corresponding to the Q media data items to the encapsulated media file to obtain a target media file, and send the target media file to the decoding device.

[0241] The encapsulated media file of the N media frames includes a first media file and a second media file;

[0242] The object indication information includes object relationship indication information; the object relationship indication information is used to indicate that a media object in a media frame corresponding to the first media file has an association relationship with a media object in a media frame corresponding to the second media file;

[0243] The sending module 23 includes:

[0244] The sixth encapsulation unit 2314 is configured to encapsulate the object relationship indication information into an associated entity group box;

[0245] The seventh encapsulation unit 2315 is configured to encapsulate the object attribute characteristics and object distribution characteristics of the media objects in the media frame corresponding to the first media file into the first media file;

[0246] an eighth encapsulation unit 2316, configured to encapsulate object attribute characteristics and object distribution characteristics of media objects in the media frame corresponding to the second media file into the second media file;

[0247] The seventh determining unit 2317 is configured to determine the associated entity group box, the encapsulated first media file, and the encapsulated second media file as a target media file, and send the target media file to the decoding device.

[0248] In an embodiment of the present application, a decoding device can receive object indication information sent by an encoding device. The object indication information is used to reflect the object attribute characteristics of the media objects included in N media frames, as well as the distribution characteristics of the media objects in the N media frames. That is, the object indication information is used to reflect what type of media objects are included in the N media frames, which media frames include media objects, and which positions in the media frames include media objects. Based on the object indication information, the decoding device can quickly understand the media frames (i.e., media data) it needs. Therefore, based on the object indication information, it can quickly obtain the media file segments to be decoded from the encapsulated media files corresponding to the N media frames. The media file segments to be decoded can refer to the media file segments corresponding to the media data required by the decoding device, which can improve the efficiency of obtaining the media file segments to be decoded. Furthermore, only the media file segments to be decoded need to be decoded to obtain the media data corresponding to the media file segments to be decoded (such as the media data required by the decoding device), without having to decode the entire media file. This can reduce the amount of decoded data, improve the efficiency of obtaining media data, and reduce the resource (such as computing resource) overhead of the decoding device.

[0249] Please refer to Figure 12, which is a structural diagram of a computer device provided in an embodiment of the present application. As shown in Figure 12, the computer device 1000 may include: a processor 1001, a network interface 1004 and a memory 1005. In addition, the above-mentioned computer device 1000 may also include: a user interface 1003, and at least one communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. Among them, the user interface 1003 may include a display screen (Display), a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. Optionally, the network interface 1004 may include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk storage. Optionally, the memory 1005 may also be at least one storage device located away from the aforementioned processor 1001. As shown in Figure 12, the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module and a device control application.

[0250] In the computer device 1000 shown in FIG12 , the network interface 1004 can provide network communication functions; the user interface 1003 is mainly used to provide an interface for user input; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

[0251] Get the encapsulated media file of N media frames; N is a positive integer;

[0252] If N media frames include media objects, object indication information about the N media frames is generated; the object indication information is used to indicate object attribute characteristics corresponding to the media objects included in the N media frames, as well as distribution characteristics of the media objects in the N media frames;

[0253] The encapsulated media file and object indication information are sent to a decoding device.

[0254] It should be understood that the computer device 1000 described in the embodiments of the present application can execute the media data processing method described in the embodiment corresponding to FIG. 2 above, and can also execute the media data processing apparatus described in the embodiment corresponding to FIG. 10 above, and thus will not be repeated here. In addition, the description of the beneficial effects of employing the same method will not be repeated here.

[0255] Please refer to Figure 13, which is a structural diagram of a computer device provided in an embodiment of the present application. As shown in Figure 13, the computer device 2000 may include: a processor 2001, a network interface 2004 and a memory 2005. In addition, the above-mentioned computer device 2000 may also include: a user interface 2003, and at least one communication bus 2002. Among them, the communication bus 2002 is used to realize the connection and communication between these components. Among them, the user interface 2003 may include a display screen (Display), a keyboard (Keyboard), and the optional user interface 2003 may also include a standard wired interface and a wireless interface. Optionally, the network interface 2004 may include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 2005 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk storage. Optionally, the memory 2005 may also be at least one storage device located away from the aforementioned processor 2001. As shown in Figure 13, the memory 2005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module and a device control application.

[0256] In the computer device 2000 shown in FIG13 , the network interface 2004 can provide network communication functions; the user interface 2003 is mainly used to provide an interface for user input; and the processor 2001 can be used to call the device control application stored in the memory 2005 to achieve:

[0257] Receive object indication information; the object indication information is used to reflect the object attribute characteristics of the media objects included in the N media frames, as well as the distribution characteristics of the media objects in the N media frames; N is a positive integer;

[0258] Obtaining, according to the object indication information, a media file segment to be decoded from the encapsulated media file corresponding to the N media frames;

[0259] The media file segment to be decoded is decoded to obtain media data corresponding to the media file segment to be decoded.

[0260] It should be understood that the computer device 2000 described in the embodiments of the present application can execute the media data processing method described in the embodiment corresponding to FIG. 9 above, and can also execute the media data processing apparatus described in the embodiment corresponding to FIG. 11 above, and thus, these descriptions are omitted here. Furthermore, the descriptions of the beneficial effects of employing the same method are also omitted here.

[0261] In addition, it should be pointed out here that: the embodiment of the present application also provides a computer-readable storage medium, and the above-mentioned computer-readable storage medium stores a computer program executed by the media data processing device mentioned above, and the above-mentioned computer program includes program instructions. When the above-mentioned processor executes the above-mentioned program instructions, it can execute the description of the above-mentioned media data processing method in the corresponding embodiment above, so it will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0262] As an example, the above program instructions may be deployed on a computer device for execution, or deployed on at least two computer devices at one location for execution, or executed on at least two computer devices distributed at at least two locations and interconnected via a communication network. The at least two computer devices distributed at at least two locations and interconnected via a communication network may constitute a blockchain network.

[0263] The computer-readable storage medium can be the media data processing device provided in any of the aforementioned embodiments or the central storage unit of the computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SMART Media card, SMC), a secure digital (SD) card, a flash memory card, etc. equipped with the computer device. Furthermore, the computer-readable storage medium can also include both the central storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium can also be used to temporarily store data that has been output or is about to be output.

[0264] The terms "first," "second," and the like in the description, claims, and drawings of the embodiments of the present application are used to distinguish between contents in different media, rather than to describe a specific order. In addition, the terms "including" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules that are not listed, or may optionally include other steps and units inherent to these processes, methods, apparatuses, products, or devices.

[0265] It is understandable that if the above embodiments of the present application need to use user information, etc., it is necessary to obtain user permission or consent and comply with relevant laws and regulations in the relevant regions.

[0266] The present application also provides a computer program product including a computer program / instruction. When executed by a processor, the computer program / instruction implements the description of the above-described media data processing method in the corresponding embodiment above. Therefore, it will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the embodiments of the computer program product involved in this application, please refer to the description of the method embodiment of this application.

[0267] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0268] The methods and related devices provided in the embodiments of the present application are described with reference to the method flow charts and / or structural diagrams provided in the embodiments of the present application. Specifically, each process and / or box in the method flow charts and / or structural diagrams, as well as the combination of the processes and / or boxes in the flow charts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable network connection device to produce a machine, so that the instructions executed by the processor of the computer or other programmable network connection device generate a device for implementing the function specified in one or more processes in the flow chart and / or one or more boxes in the structural diagram. These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable network connection device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a product including an instruction device, which implements the function specified in one or more processes in the flow chart and / or one or more boxes in the structural diagram. These computer program instructions can also be loaded onto a computer or other programmable network-connected device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the structural diagram.

[0269] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A media data processing method, characterized in that: include: receiving object indication information; The object indication information is used to reflect the object attribute characteristics of the media objects included in the N media frames, and the distribution characteristics of the media objects in the N media frames; N is a positive integer; According to the object indication information, obtaining a media file segment to be decoded from the encapsulated media files corresponding to the N media frames; The media file segment to be decoded is decoded to obtain media data corresponding to the media file segment to be decoded.

2. The method according to claim 1, characterized in that The object attribute feature includes one or more of the object quantity, object identification and object description information of the media objects included in the N media frames.

3. The method according to claim 1 or 2, characterized in that: The distribution feature is used to indicate a target media frame having the media object among the N media frames; or, The distribution feature is used to indicate a target media frame having the media object among the N media frames, and a spatial region to which the media object belongs in the target media frame; or, The distribution feature is used to indicate the target media frame having the media object among the N media frames, and the fragment to which the media object belongs in the target media frame; or The distribution feature is used to indicate the target media frame having the media object among the N media frames, the spatial region to which the media object belongs in the target media frame, and the slice to which the media object belongs in the target media frame.

4. The method according to any one of claims 1 to 3, characterized in that: The media file segment to be decoded is code stream data including a target media object; or, the media file segment to be decoded is code stream data not including the target media object; wherein the target media object belongs to a media object in the N media frames.

5. The method according to any one of claims 1 to 4, characterized in that: The object indication information includes object indication information respectively associated with S media file segments; the S media file segments belong to the encapsulated media file; S is an integer greater than 1; The acquiring, according to the object indication information, a media file segment to be decoded from the encapsulated media files corresponding to the N media frames comprises: Obtaining segment identifiers corresponding to the S media file segments respectively; If the object indication information of the target media file segment among the S media file segments reflects that the target media file segment meets the decoding condition, generating an acquisition request for the target media file segment according to the segment identifier of the target media file segment; Sending the acquisition request to an encoding device, and receiving the target media file segment returned by the encoding device based on the acquisition request; The target media file segment is determined as the media file segment to be decoded.

6. The method according to any one of claims 1 to 5, characterized in that: The encapsulated media file includes P media tracks to which K dynamic media frames among the N media frames belong; P is a positive integer, and K is a positive integer less than or equal to N; The acquiring, according to the object indication information, a media file segment to be decoded from the encapsulated media files corresponding to the N media frames comprises: Obtaining object attribute characteristics of a media object in a dynamic media frame belonging to the media track j from the object information data box j corresponding to the media track j; j is a positive integer less than or equal to P; According to the object attribute characteristics corresponding to the P media tracks, the target media tracks satisfying the decoding condition are determined from the P media tracks. body orbit; The media file segment to be decoded is determined according to the object indication information corresponding to the dynamic media frame in the target media track; the object indication information of the dynamic media frame in the target media track is encapsulated in the metadata track corresponding to the target media track.

7. The method according to any one of claims 1 to 6, characterized in that: The step of determining the media file segment to be decoded according to the object indication information corresponding to the dynamic media frame in the target media track includes: Determining, according to the object indication information corresponding to the dynamic media frames belonging to the target media track, a target dynamic media frame that meets the decoding condition from the dynamic media frames belonging to the target media track; The media file segment to be decoded is determined according to the code stream data about the target dynamic media frame in the target media track and the object indication information of the target dynamic media frame.

8. The method according to any one of claims 1 to 7, characterized in that: The determining the media file segment to be decoded according to the code stream data about the target dynamic media frame in the target media track and the object indication information of the target dynamic media frame includes: Determining, according to the object indication information of the target dynamic media frame, a slice satisfying a decoding condition from the target dynamic media frame as a first slice; The code stream data about the first fragment in the code stream data corresponding to the target dynamic media frame is determined as a media file segment to be decoded.

9. The method according to any one of claims 1 to 8, characterized in that: The encapsulated media file includes Q media data items corresponding to Q static media frames in the N media frames, and the object indication information includes object indication information corresponding to the Q media data items respectively; Q is a positive integer less than or equal to N; The acquiring, according to the object indication information, a media file segment to be decoded from the encapsulated media files corresponding to the N media frames comprises: Determining, from the Q media data items, a target media data item that meets a decoding condition according to the object indication information respectively corresponding to the Q media data items; A media file segment to be decoded is determined according to the target media data item and the object indication information corresponding to the target media data item.

10. The method according to any one of claims 1 to 9, characterized in that: The determining, according to the target media data item and the object indication information corresponding to the target media data item, a media file segment to be decoded includes: According to the object indication information corresponding to the target media data item, determining a point cloud piece satisfying a decoding condition from the static media frame corresponding to the target media data item as a second point cloud piece; Determine the code stream data corresponding to the second point cloud piece from the target media data item, and determine the code stream data corresponding to the second point cloud piece as a media file segment to be decoded.

11. The method according to any one of claims 1 to 9, characterized in that: The encapsulated media file includes a first media file and a second media file, and the object indication information includes object indication information of the first media file, object indication information of the second media file, and object relationship indication information; the object relationship indication information is used to indicate that a media object in a media frame corresponding to the first media file has an association relationship with a media object in a media frame corresponding to the second media file, the object indication information of the first media file is encapsulated in the first media file, and the object indication information of the second media file is encapsulated in the second media file; The acquiring, according to the object indication information, a media file segment to be decoded from the encapsulated media files corresponding to the N media frames comprises: When it is determined that the first media file meets the decoding condition, acquiring a second media file associated with the first media file according to the object relationship indication information; A media file segment to be decoded is determined according to the first media file and the second media file.

12. A media data processing method, characterized in that: include: Obtaining a packaged media file of N media frames; N is a positive integer; If the N media frames include media objects, generating object indication information about the N media frames; The object indication information is used to indicate object attribute characteristics corresponding to the media objects included in the N media frames, and distribution characteristics of the media objects in the N media frames; The encapsulated media file and the object indication information are sent to a decoding device.

13. The method according to claim 12, characterized in that The sending the encapsulated media file and the object indication information to a decoding device includes: If the encapsulated media file includes S media file segments, extracting object indication information respectively associated with the S media file segments from the object indication information; S is an integer greater than 1; Encapsulating object indication information associated with a media file segment i among the S media file segments into the media file segment i to obtain a target media file segment i; S is an integer greater than 1, and i is a positive integer less than or equal to S; Sending object indication information respectively associated with the S media file segments and segment identifiers respectively corresponding to the S media file segments to a decoding device; If an acquisition request for the target media file segment i is received, the target media file segment i is sent to the decoding device; the acquisition request is generated by the decoding device based on the object indication information respectively associated with the S media file segments and the segment identifier.

14. The method according to claim 12 or 13, characterized in that The N media frames include K dynamic media frames having media objects, and the encapsulated media file includes P media tracks to which the K dynamic media frames belong; P is a positive integer, K is a positive integer less than or equal to N; The sending the encapsulated media file and the object indication information to a decoding device includes: Obtaining object attribute characteristics and distribution characteristics of media objects included in the dynamic media frame belonging to media track j from the object indication information; j is a positive integer less than or equal to P; Encapsulate the object attribute characteristics of the media objects in the dynamic media frame belonging to the media track j into the object information data box j associated with the media track j; Encapsulating object attribute characteristics and distribution characteristics of media objects included in the dynamic media frame belonging to the media track j into the metadata track corresponding to the media track j; The object information data boxes and metadata tracks respectively corresponding to the P media tracks are added to the encapsulated media file to obtain a target media file, and the target media file is sent to a decoding device.

15. The method according to any one of claims 12 to 14, characterized in that: The metadata track corresponding to the media track j includes metadata track samples corresponding to the dynamic media frames belonging to the media track j; The step of encapsulating the object attribute characteristics and distribution characteristics of the media objects included in the dynamic media frame belonging to the media track j into the metadata track corresponding to the media track j includes: The object attribute characteristics and distribution characteristics of the media objects included in the dynamic media frame a belonging to the media track j are added to the dynamic The metadata track sample corresponding to the media frame a; a is a positive integer less than or equal to the total number of dynamic media frames belonging to the media track j.

16. The method according to any one of claims 12 to 15, characterized in that: The metadata track corresponding to the media track j includes metadata track samples corresponding to the dynamic media frames belonging to the media track j; The step of encapsulating the object attribute characteristics and distribution characteristics of the media objects included in the dynamic media frame belonging to the media track j into the metadata track corresponding to the media track j includes: Obtaining object attribute characteristics and distribution characteristics of media objects in a reference media frame corresponding to a dynamic media frame a belonging to the media track j; a is a positive integer less than or equal to the total number of dynamic media frames belonging to the media track j; Determine an object change feature between an object attribute feature of a media object in the reference media frame and an object attribute feature of the dynamic media frame a; Determining distribution change characteristics between distribution characteristics of media objects in the reference media frame and distribution characteristics of the dynamic media frame a; The object change feature and the distribution change feature are added to the metadata track sample corresponding to the dynamic media frame a.

17. The method according to any one of claims 12 to 16, characterized in that: The step of adding the object information data boxes and metadata tracks respectively corresponding to the P media tracks to the encapsulated media file to obtain the target media file includes: Add the object information data box j to the track sample entry of the media track j; The metadata tracks corresponding to the P media tracks are respectively added to the encapsulated media file to obtain a target media file.

18. The method according to any one of claims 1 to 17, characterized in that: The step of adding the object information data boxes and metadata tracks respectively corresponding to the P media tracks to the encapsulated media file to obtain the target media file includes: Add the object information data box j to the track sample entry of the metadata track corresponding to the media track j to obtain the added metadata track corresponding to the media track j; The metadata tracks corresponding to the P media tracks are added to the encapsulated media file to obtain a target media file.

19. The method according to any one of claims 12 to 18, characterized in that: The N media frames include Q static media frames having media objects, and the encapsulated media file includes Q media data items corresponding to the Q static media frames; Q is a positive integer less than or equal to N; The sending the encapsulated media file and the object indication information to a decoding device includes: Obtaining object attribute characteristics and distribution characteristics of a media object in a static media frame corresponding to a media data item r from the object indication information; r is a positive integer less than or equal to Q; Encapsulating object attribute characteristics and distribution characteristics of media objects in the static media frame corresponding to the media data item r into a data item attribute box associated with the media data item r; The data item attribute boxes respectively corresponding to the Q media data items are added to the encapsulated media file to obtain a target media file, and the target media file is sent to a decoding device.

20. The method according to any one of claims 12 to 19, characterized in that: The packaged media file includes a first media file and a second media file; The object indication information includes object relationship indication information; the object relationship indication information is used to indicate that a media object in a media frame corresponding to the first media file has an association relationship with a media object in a media frame corresponding to the second media file; The sending the encapsulated media file and the object indication information to a decoding device includes: Encapsulating the object relationship indication information into an associated entity group box; encapsulating object attribute characteristics and object distribution characteristics of media objects in the media frame corresponding to the first media file into the first media file; encapsulating object attribute characteristics and object distribution characteristics of media objects in the media frame corresponding to the second media file into the second media file; The associated entity group box, the encapsulated first media file, and the encapsulated second media file are determined as target media files, and the target media files are sent to a decoding device.

Images