A data processing method and related equipment for point cloud media

Abstract

An embodiment of the present application provides a method and related equipment for processing point cloud media data, wherein the method includes: obtaining a media file of point cloud media, wherein the media file includes a point cloud code stream of the point cloud media, wherein the point cloud code stream includes an attribute parameter set, wherein the attribute parameter set includes parameter information required for decoding attribute data in the point cloud code stream; wherein the media file also includes attribute parameter set indication information, wherein the attribute parameter set indication information is used to indicate the encapsulation location of the attribute parameter set; and, according to the attribute parameter set indication information, decoding the point cloud code stream to present the point cloud media. The embodiment of the present application can indicate the encapsulation location of the attribute parameter set, thereby reducing unnecessary codec dependencies, enabling the transmission, decoding, and presentation of point cloud media, and optimizing the utilization of network bandwidth and computing resources on the decoding side.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and specifically to a point cloud media data processing method, a point cloud media data processing device, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the continuous development of point cloud technology, compression encoding of point cloud media has become a significant research issue. Current compression encoding techniques for point cloud media allow a single attribute parameter set to contain the parameter information required for decoding multiple sets of attribute data within the definition of attribute parameters in a point cloud codestream. However, current compression encoding techniques for point cloud media still present some challenges. For example, supporting partial transmission or partial decoding can lead to unnecessary encoding and decoding dependencies and waste of decoding resources. Therefore, improving attribute encoding techniques has become a hot topic in the field of point cloud coding technology. Summary of the Invention

[0003] The embodiments of the present application provide a data processing method and related equipment for point cloud media, which can indicate the encapsulation position of an attribute parameter set, obtain the attribute parameter set according to the indicated encapsulation position, reduce unnecessary encoding and decoding dependencies, realize the transmission, decoding and presentation of point cloud media, and optimize the utilization of network bandwidth and computing resources on the decoding side.

[0004] In one aspect, an embodiment of the present application provides a method for processing point cloud media data, the method comprising:

[0005] Obtaining a media file of the point cloud media, the media file including a point cloud code stream of the point cloud media, the point cloud code stream including an attribute parameter set, the attribute parameter set including parameter information required for decoding attribute data in the point cloud code stream; the media file also includes attribute parameter set indication information, the attribute parameter set indication information being used to indicate a packaging location of the attribute parameter set;

[0006] According to the attribute parameter set indication information, the point cloud code stream is decoded to present the point cloud media.

[0007] In one aspect, an embodiment of the present application provides a method for processing point cloud media data, the method comprising:

[0008] Get point cloud media;

[0009] Encoding the point cloud media to obtain a point cloud code stream; the point cloud code stream includes an attribute parameter set, and the attribute parameter set includes parameter information required for decoding the attribute data in the point cloud code stream;

[0010] The point cloud code stream is encapsulated to obtain a media file of the point cloud media; the media file also includes attribute parameter set indication information, and the attribute parameter set indication information is used to indicate the encapsulation position of the attribute parameter set.

[0011] In one aspect, an embodiment of the present application provides a point cloud media data processing device, the device comprising:

[0012] an acquisition unit, configured to acquire a media file of the point cloud media, the media file including a point cloud code stream of the point cloud media, the point cloud code stream including an attribute parameter set, the attribute parameter set including parameter information required for decoding attribute data in the point cloud code stream; the media file also including attribute parameter set indication information, the attribute parameter set indication information being used to indicate a packaging location of the attribute parameter set;

[0013] The processing unit is used to decode the point cloud code stream according to the attribute parameter set indication information to present the point cloud media.

[0014] In one embodiment, the media file includes a track, each track includes one or more samples, each sample corresponds to a point cloud frame in the point cloud media; each sample is divided into one or more subsamples, each subsample includes a type of point cloud data;

[0015] The media file also includes a subsample information data box, and the attribute parameter set is encapsulated in the subsample information data box.

[0016] In one embodiment, the attribute parameter set indication information is represented as a subsample information data box, and the attribute parameter set indication information is further used to indicate the relationship between the attribute parameter set and the attribute data in the point cloud code stream;

[0017] The subsample data box includes at least one of the following fields: a payload type field, a reference attribute information flag field, a reference attribute type flag field, a reference attribute identification field, and a reference attribute type field;

[0018] The payload type field is used to indicate the type of point cloud data contained in the subsample. If the payload type field is the first value, it indicates that the type of point cloud data contained in the subsample is an attribute parameter set. If the payload type field is the second value, it indicates that the type of point cloud data contained in the subsample is a geometry parameter set, which contains parameter information required for decoding geometry data.

[0019] The reference attribute information flag field is used to indicate the scope of validity of the current attribute parameter set for attribute data. If the reference attribute information flag field is a first preset value, it indicates that the current attribute parameter set is valid only for a specific group or a specific type of attribute data in the current sample. If the reference attribute information flag field is a second preset value, it indicates that the current attribute parameter set is valid for all attribute data in the current sample.

[0020] The reference attribute type flag field is used to indicate the type of attribute data to which the current attribute parameter set is effective. If the reference attribute type flag field is a first value, it indicates that the current attribute parameter set is only effective for a specific type of attribute data in the current sample. If the reference attribute type flag field is a second value, it indicates that the current attribute parameter set is only effective for a specific set of attribute data in the current sample.

[0021] The reference attribute identification field is used to indicate the identifier of the attribute data corresponding to the current attribute parameter set; the reference attribute type field is used to indicate the type of the attribute data corresponding to the current attribute parameter set;

[0022] The current attribute parameter set refers to the attribute parameter set being used, and the current sample refers to the sample being decoded.

[0023] In one embodiment, the point cloud code stream is encapsulated in a media file in a multi-track manner; the media file includes a parameter component track, and the attribute parameter set is independently encapsulated in the parameter component track;

[0024] The parameter component track contains one or more samples, each sample corresponds to a point cloud frame of the point cloud media, and each sample contains the parameter information required to decode the attribute data of the corresponding point cloud frame.

[0025] In one embodiment, the media file includes one or more parameter component tracks; when the media file includes only one parameter component track, the attribute parameter set in the point cloud code stream is encapsulated into the parameter component track; when the media file includes multiple parameter component tracks, the attribute parameter set in the point cloud code stream is encapsulated into a separate parameter component track or the attribute parameter set in the point cloud code stream is encapsulated into multiple parameter component tracks based on type.

[0026] In one embodiment, the parameter component track includes a track data box;

[0027] The track data box includes a track reference type data box, which is used to indicate that the parameter component track is associated with one or more attribute component tracks corresponding to the attribute parameter set in the parameter component track.

[0028] In one embodiment, the media file complies with a first encoding standard; the parameter component track includes a component information data box, which is set in a header sample entry of the parameter component track; the component information data box includes at least one of the following fields: a component type field, a header type field, a number of attribute headers field, a reference attribute type flag field, a reference attribute identifier field, and a reference attribute type field;

[0029] The component type field is used to indicate the component type in the parameter component track; when the component type field is set to the target preset value, it indicates that the component type in the parameter component track is a parameter set;

[0030] The header type field is used to indicate the parameter set type contained in the parameter component track; if the header type field is the first setting value, it indicates that the parameter set type contained in the parameter component track is an attribute parameter set; if the header type field is the second setting value, it indicates that the parameter set type contained in the parameter component track is a geometry parameter set; if the header type field is the third setting value, it indicates that the parameter set type contained in the parameter component track is a geometry parameter set and an attribute parameter set;

[0031] The attribute header quantity field is used to indicate the number of attribute parameter sets contained in the sample in the parameter component track; wherein the order of the attribute parameter sets contained in the sample is consistent with the order of the attribute parameter set identifiers or attribute parameter set types indicated in the component information data box;

[0032] The reference attribute type field is used to indicate the type of attribute data to which the current attribute parameter set is effective. If the reference attribute type flag field is the first preset value, it indicates that the current attribute parameter set is only effective for a specific type of attribute data in the current sample. If the reference attribute type flag field is the second preset value, it indicates that the current attribute parameter set is effective for a specific set of attribute data in the current sample.

[0033] The reference attribute identifier field is used to indicate the identifier of the attribute data corresponding to the current attribute parameter set;

[0034] The reference attribute type field is used to indicate the type of attribute data corresponding to the current attribute parameter set;

[0035] The current attribute parameter set refers to the attribute parameter set being used, and the current sample refers to the sample being decoded.

[0036] In one embodiment, the media file complies with the second encoding standard; the parameter component track includes a component information data box and a configuration data box; the parameter set type corresponding to the setting unit included in the configuration data box is the same as the parameter set type indicated by the component information data box;

[0037] The component information data box includes at least one of the following fields: a component type field, a parameter setting type quantity field, and a parameter setting type field;

[0038] The component type field is used to indicate the type of the component in the parameter component track; if the component type field is a target setting value, it indicates that the type of the component in the parameter component track is a parameter set;

[0039] The parameter set type quantity field is used to indicate the number of parameter sets contained in the parameter component track;

[0040] The parameter setting type field is used to indicate the type of parameter set contained in the parameter component track; if the parameter setting type field is the first preset value, it means that the type of parameter set contained in the parameter component track is an attribute parameter set; if the parameter setting type field is the second preset value, it means that the type of parameter set contained in the parameter component track is a geometric parameter set; if the parameter setting type field is the third preset value, it means that the type of parameter set contained in the parameter component track is a sequence parameter set.

[0041] In one embodiment, the media file includes a geometry component track, and the attribute parameter set is encapsulated in the geometry component track.

[0042] In one embodiment, the geometry component track includes a component information data box, and the attribute parameter set indication information is represented as the component information data box;

[0043] The component information data box includes a component type field, which is used to indicate the type of component in the geometry component track; if the component type field is the first set value, it indicates that the type of component in the geometry component track is geometry data and parameter set, and the parameter set includes an attribute parameter set and a geometry parameter set.

[0044] In one embodiment, the media file includes one or more attribute component tracks, and the attribute parameter set is encapsulated in each attribute component track;

[0045] The attribute parameter set encapsulated in each attribute component track is obtained by copying the same attribute parameter set.

[0046] In one embodiment, the media file includes one or more attribute component tracks, the attribute parameter set includes an index, and the index of the attribute parameter set is encapsulated in each attribute component track;

[0047] The attribute parameter set or the data unit formed by reorganizing the attribute parameter set can be found through the index encapsulated in any attribute component track.

[0048] In one embodiment, the point cloud media is transmitted in a streaming manner; when the acquisition unit acquires the media file of the point cloud media, it can be specifically configured to:

[0049] Obtaining transmission signaling of the point cloud media, which includes stream indication information. The stream indication information is used to indicate the data stream corresponding to the parameter component track;

[0050] Get the media file of the point cloud media according to the transmission signaling.

[0051] In one embodiment, the transmission signaling is dynamic adaptive stream signaling, and the stream indication information refers to a component descriptor, which is set in the data stream corresponding to the parameter component track; the component descriptor includes a component type element and a parameter type element, the component type element is used to indicate the type of the component in the parameter component track; the parameter type element is used to indicate the type of the parameter set included in the parameter component track;

[0052] If the component type element is a target character, the type of the component in the parameter component track is a parameter collection component.

[0053] In one embodiment, when the processing unit decodes the point cloud stream according to the attribute parameter set indication information to present the point cloud media, it can be specifically used to:

[0054] Obtaining the attribute parameter set according to the encapsulation position indicated by the attribute parameter set indication information;

[0055] The attribute data in the point cloud code stream is decoded using the parameter information in the attribute parameter set to present the point cloud media.

[0056] In one aspect, an embodiment of the present application provides a point cloud media data processing device, the device comprising:

[0057] An acquisition unit, used for acquiring point cloud media;

[0058] A processing unit is used to encode the point cloud media to obtain a point cloud code stream; the point cloud code stream includes an attribute parameter set, and the attribute parameter set includes parameter information required for decoding attribute data in the point cloud code stream;

[0059] The processing unit is further used to encapsulate the point cloud code stream to obtain a media file of the point cloud media; the media file also includes attribute parameter set indication information, and the attribute parameter set indication information is used to indicate the encapsulation position of the attribute parameter set.

[0060] In one embodiment, the media file includes a track, the point cloud code stream is encapsulated in the track, the track includes one or more samples, and one sample corresponds to one point cloud frame in the point cloud media; one sample is divided into one or more sub-samples, and each sub-sample includes one type of point cloud data;

[0061] The media file also includes a subsample information data box, and the attribute parameter set is encapsulated in the subsample information data box.

[0062] In one embodiment, the point cloud stream is encapsulated into a media file in a multi-track manner; the media file includes a parameter component track, and the attribute parameter set is independently encapsulated in the parameter component track;

[0063] The parameter component track contains one or more samples, each sample corresponds to a point cloud frame of the point cloud media, and each sample contains the parameter information required to decode the attribute data of the corresponding point cloud frame.

[0064] In one embodiment, the media file includes a geometry component track, and the attribute parameter set is encapsulated in the geometry component track.

[0065] In one embodiment, the media file includes one or more attribute component tracks, and the attribute parameter set is encapsulated in each attribute component track;

[0066] The attribute parameter set encapsulated in each attribute component track is obtained by copying the same attribute parameter set.

[0067] In one embodiment, the media file includes one or more attribute component tracks, the attribute parameter set includes an index, and the index of the attribute parameter set is encapsulated in each attribute component track;

[0068] The attribute parameter set or the data unit formed by reorganizing the attribute parameter set can be found through the index encapsulated in any attribute component track.

[0069] In one embodiment, the processing unit is further configured to:

[0070] The media file of the point cloud media is sent through transmission signaling, which includes stream indication information. The stream indication information is used to indicate the data stream corresponding to the parameter component track.

[0071] In one aspect, an embodiment of the present application provides a computer device, comprising:

[0072] a processor suitable for executing a computer program;

[0073] Computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the data processing method of the point cloud media as described above is implemented.

[0074] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program is loaded by a processor and executes the data processing method of the point cloud media as described above.

[0075] In one aspect, embodiments of the present application provide a computer program product, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the aforementioned point cloud media data processing method.

[0076] In an embodiment of the present application, a media file of point cloud media is obtained, and the media file includes a point cloud code stream and attribute parameter set indication information of the point cloud media. The point cloud code stream includes an attribute parameter set, and the attribute parameter set includes parameter information required for decoding attribute data in the point cloud code stream; the attribute parameter set indication information is used to indicate the encapsulation position of the attribute parameter set; according to the attribute parameter set indication information, the encapsulation position of the attribute parameter set can be directly known, and the point cloud code stream can be decoded based on the attribute parameter set to present the point cloud media, which guides the transmission, decoding and presentation of the point cloud media, can reduce unnecessary encoding and decoding dependencies, and save transmission resources and computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0078] Figure 1a is a schematic diagram of a 6DoF provided by an exemplary embodiment of the present application;

[0079] Figure 1b is a schematic diagram of a 3DoF provided by an exemplary embodiment of the present application;

[0080] Figure 1c is a schematic diagram of a 3DoF+ provided by an exemplary embodiment of the present application;

[0081] Figure 2a This is an architectural diagram of a point cloud media data processing system provided by an exemplary embodiment of the present application;

[0082] Figure 2b This is a flow chart of data processing of a point cloud media provided by an exemplary embodiment of the present application;

[0083] Figure 3 This is a flowchart of a method for processing point cloud media data provided by an exemplary embodiment of the present application;

[0084] Figure 4a is a schematic diagram of dividing a parameter set and attribute data into data units provided by an exemplary embodiment of the present application;

[0085] Figure 4b is a schematic diagram of dividing a parameter set and attribute data into data units provided by another exemplary embodiment of the present application;

[0086] Figure 5 is a flowchart of a method for processing point cloud media data provided by another exemplary embodiment of the present application;

[0087] Figure 6 This is a schematic structural diagram of a point cloud media data processing device provided by an exemplary embodiment of the present application;

[0088] Figure 7 is a structural diagram of a point cloud media data processing device provided by another exemplary embodiment of the present application;

[0089] Figure 8 It is a structural diagram of a computer device provided by another exemplary embodiment of the present application. DETAILED DESCRIPTION

[0090] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0091] In this application, the terms "first", "second", etc. are used to distinguish identical or similar items with substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there any limitation on the quantity and execution order.

[0092] In this application, the term "at least one" means one or more, and "multiple" means two or more; similarly, the term "at least one group" means one or more groups, and "multiple groups" means two or more groups. For example, a point in a point cloud has multiple sets of attribute data, which means that the point has two or more sets of attribute data.

[0093] The following is an introduction to other technical terms involved in this application:

[0094] 1. Immersive Media

[0095] Immersive media refers to media files that can provide immersive media content, allowing viewers immersed in the media content to obtain visual, auditory and other sensory experiences in the real world. Immersive media can be divided into 6DoF (Degree of Freedom) immersive media, 3DoF immersive media, and 3DoF+ immersive media according to the degree of freedom of viewers when consuming media content. Figure 1a As shown, 6DoF means that viewers of immersive media can freely translate along the X, Y, and Z axes. For example, viewers can move freely within 360-degree VR content. Similar to 6DoF are 3DoF and 3DoF+ production technologies. Figure 1b A schematic diagram of a 3DoF provided in an embodiment of the present application; Figure 1b As shown, 3DoF means that the viewer of immersive media is fixed at the center point of a three-dimensional space, and the viewer's head rotates along the X-axis, Y-axis, and Z-axis to view the images provided by the media content. Figure 1c A schematic diagram of 3DoF+ provided in an embodiment of the present application is shown as follows: Figure 1c As shown, 3DoF+ means that when the virtual scene provided by the immersive media has certain depth information, the head of the immersive media viewer can move within a limited space based on 3DoF to view the images provided by the media content.

[0096] 2. Point Cloud

[0097] A point cloud is a set of randomly distributed discrete points in space that represent the spatial structure and surface properties of a three-dimensional object or scene. Each point in a point cloud includes at least geometric data, which represents the point's three-dimensional position. Depending on the application scenario, a point in a point cloud may also include one or more sets of attribute data. Each set of attribute data reflects a specific attribute of the point, such as color, material, or other information. Typically, each point in a point cloud has the same number of sets of attribute data.

[0098] Point clouds can flexibly and conveniently express the spatial structure and surface properties of three-dimensional objects or scenes, and are therefore widely used in scenarios such as virtual reality (VR) games, computer-aided design (CAD), geographic information systems (GIS), autonomous navigation systems (ANS), digital cultural heritage, free viewpoint broadcasting, three-dimensional immersive telepresence, and three-dimensional reconstruction of biological tissues and organs.

[0099] Point clouds are primarily acquired through computer generation, three-dimensional (3D) laser scanning, and 3D photogrammetry. Specifically, point clouds can be obtained by capturing real-world visual scenes using a collection device (a set of cameras or a camera device with multiple lenses and sensors). 3D laser scanning can generate point clouds of static real-world 3D objects or scenes, with millions of point clouds acquired per second. 3D photography can generate point clouds of dynamic real-world 3D objects or scenes, with tens of millions of point clouds acquired per second. Furthermore, in the medical field, point clouds of biological tissues and organs can be obtained through magnetic resonance imaging (MRI), computed tomography (CT), and electromagnetic positioning information. Furthermore, point clouds can also be generated directly by computers based on virtual 3D objects and scenes. For example, computers can generate point clouds of virtual 3D objects and scenes. With the continuous accumulation of large-scale point cloud data, efficient storage, transmission, publication, sharing, and standardization of point cloud data have become key to point cloud applications.

[0100] 3. Point Cloud Media

[0101] Point cloud media is a typical 6DoF immersive media. Point cloud media includes a frame sequence consisting of one or more point cloud frames, and each point cloud frame consists of geometric data and attribute data of one or more points in the point cloud. Among them, geometric data can also be called three-dimensional position information. The geometric data of a point in the point cloud refers to the spatial coordinates (x, y, z) of the point, and can include the coordinate values ​​of the point in each coordinate axis direction of the three-dimensional coordinate system, for example, the coordinate value x in the X-axis direction, the coordinate value y in the Y-axis direction, and the coordinate value z in the Z-axis direction. A point in the point cloud can include one or more sets of attribute data, each set of attribute data is used to reflect an attribute of the point. For example, a point in the point cloud has a set of color attribute data, and the color attribute data is used to reflect the color attribute of the point (such as red, yellow, etc.); for another example, a point in the point cloud has a set of reflectivity attribute data, and the reflectivity attribute data is used to reflect the laser reflection intensity attribute of the point. When a point in a point cloud has multiple sets of attribute data, the types of the multiple sets of attribute data can be the same or different. For example, a point in a point cloud can have one set of color attribute data and one set of reflectivity attribute data; for another example, a point in a point cloud can have two sets of color attribute data, which are used to reflect the color attributes of the point at different times.

[0102] In particular, each point cloud frame also includes a geometric parameter set corresponding to the geometric data or an attribute parameter set corresponding to the attribute data. The geometric parameter set and the attribute parameter set can constitute a parameter set. The geometric parameter set can include parameter information required to encode or decode the geometric data in each point cloud frame; the attribute parameter set corresponding to the attribute data can include parameter information required to encode or decode the attribute data in each point cloud frame. In other words, the point cloud data contained in a point cloud frame in the point cloud media can include geometric data, a geometric parameter set, one or more sets of attribute data, and one or more attribute parameter sets.

[0103] Among them, in AVS (Audio Video Coding Standard, China's national audio and video coding standard) point cloud coding technology, a point cloud frame usually only contains a point cloud geometry header (i.e., Geometry Header) and a point cloud attribute header (i.e., Sequence Header). The point cloud geometry header includes a parameter set required for encoding or decoding the geometric data of a single-frame point cloud frame, and the point cloud attribute header includes a parameter set required for encoding or decoding the attribute data in a single-frame point cloud frame. That is, the point cloud attribute header can contain all parameter information corresponding to one or more groups of attribute data in the point cloud frame; while in MPEG (Moving Picture Expert Group, international audio and video coding standard) point cloud coding technology, a geometry parameter set (GPS, Geometry Parameter Set) and one or more attribute parameter sets (i.e., APS, Attribute Parameter Set) can be defined in the point cloud code stream corresponding to the point cloud media. An APS can correspond to different types of attribute data in the point cloud frame. In the embodiment of the present application, no distinction is made between the attribute header and the APS, and both are described as attribute parameter sets.

[0104] 4. Track

[0105] A track refers to a collection of media data during the point cloud media encapsulation process. A track consists of multiple time-sequential samples, and one sample corresponds to one point cloud frame of the point cloud media. Point cloud media can be encapsulated in either a single-track or multi-track manner. The so-called single-track method means that all point cloud data of the point cloud media is encapsulated into the same track. In this case, the media file of the point cloud media contains only one track (i.e., a single track obtained by single-track encapsulation). In the single track obtained by the single-track method, a sample refers to a point cloud frame in the point cloud media, and a sample contains all the data of the corresponding point cloud frame (including geometric data and attribute data).

[0106] The so-called multi-track method refers to encapsulating the point cloud data of point cloud media into multiple different tracks. In this case, the media file of the point cloud media can contain multiple tracks. Furthermore, the multi-track method can include a type-based multi-track method. The type-based multi-track method is to encapsulate one type of data into a component track. For example, if the point cloud media contains geometric data, a set of color attribute data, and a set of reflectance attribute data, then the geometric data can be encapsulated into the geometry component track, the color attribute data can be encapsulated into the color attribute component track, and the reflectance attribute data can be encapsulated into the reflectance attribute component track. In any component track obtained by the type-based multi-track method, a sample only contains partial data of a point cloud frame of the point cloud media. For example, a sample in the geometry component track contains the geometric data of a point cloud frame in the point cloud media, and a sample in the color attribute component track contains a set of color attribute data for a point cloud frame in the point cloud media. In particular, metadata information can also be included as a media type in the media file of the point cloud media using a metadata track. Among them, each component track (such as the geometry component track and the attribute component track) can include one or more samples, each sample can correspond to a point cloud frame in the point cloud media, and the samples corresponding to the same point cloud frame should have the same presentation time.

[0107] In addition, when the point cloud media contains a parameter set, the single-track method obtains a single track in which one sample contains all the data of the corresponding point cloud frame, that is, one sample includes the geometric data, attribute data and parameter set of a point cloud frame. The parameter set includes a geometric parameter set and an attribute parameter set. The geometric parameter set includes the parameter information required for decoding the geometric data of the point cloud frame; the attribute parameter set includes the parameter information required for decoding the attribute data of the point cloud frame. The type-based multi-track method encapsulates one type of data into one track, and the parameter set may be encapsulated in multiple locations, including but not limited to: ① The parameter set can be encapsulated into the geometric component track corresponding to the geometric data; ② The parameter set can be encapsulated into the attribute component track corresponding to the attribute data; ③ The parameter information can be encapsulated into the parameter component track as a type of point cloud data, and one sample in the parameter component track includes the parameter information of a point cloud frame in the point cloud media.

[0108] It should be understood that after being encapsulated, point cloud media exists in the form of components in the track. For example, the attribute data in the point cloud media exists in the form of attribute components in the track after being encapsulated, the geometric data in the point cloud media exists in the form of geometric components in the track after being encapsulated, and the parameter set exists in the form of parameter components in the track after being encapsulated.

[0109] 5. Sample and Subsample

[0110] A sample is a unit of encapsulation in a media file. A track is composed of many samples. For example, a video track can be composed of many samples, and a sample is usually a video frame. In the embodiment of the present application, a media file of point cloud media contains one or more tracks, and a sample in a track corresponds to a point cloud frame.

[0111] A sample can be divided into one or more subsamples, each of which can contain a type of point cloud data in a point cloud codestream. The point cloud data in a point cloud codestream can contain the following types: parameter sets (including geometric parameter sets and attribute parameter sets), geometric data, and attribute data. As can be seen, a subsample can contain a parameter set in a point cloud codestream, or a subsample can contain geometric data in a point cloud codestream, or a subsample can contain attribute data in a point cloud codestream. In one implementation, a point cloud frame can be divided into one or more point cloud slices (or point cloud strips). A point cloud slice represents a set of grammatical elements (e.g., geometric point cloud slices, attribute point cloud slices) of data after a point cloud frame is partially or completely encoded. In this case, the attribute data and / or geometric data contained in the subsample belong to the one or more point cloud slices. The types of subsamples corresponding to point cloud slices can include at least two types. One type is a subsample based on the data type carried by the point cloud slice. Under this type, a subsample only contains one type of data carried by the point cloud slice and related information, such as a subsample only contains geometric data types and geometric data related information. The other is a subsample based on a point cloud slice. In this type, a subsample can contain all the information of a point cloud slice, that is, the geometry slice header and geometry data, the attribute slice header and attribute data.

[0112] 6. Sample Entry

[0113] The sample entry is used to indicate metadata information related to all samples in the track. For example, the sample entry of a video track usually contains metadata information related to decoder initialization.

[0114] 7. Point cloud spatial block (Tile):

[0115] Point cloud spatial blocks are also called hexahedral spatial block areas within the boundary space of the point cloud frame. A point cloud spatial block consists of one or more point cloud slices, and there is no encoding and decoding dependency between point cloud spatial blocks.

[0116] 8. ISOBMFF (ISO Based Media File Format)

[0117] ISOBMFF is a packaging standard for media files, and a typical ISOBMFF file is an MP4 file.

[0118] 9. DASH (Dynamic Adaptive Streaming over HTTP) DASH is an adaptive bitrate technology that enables high-quality streaming media to be delivered over the Internet through traditional HTTP network servers.

[0119] 10. MPD (Media Presentation Description, media presentation description signaling in DASH):

[0120] MPD is used to describe the media segment information in a media file.

[0121] 11. Representation

[0122] Representation refers to the combination of one or more media components in DASH. For example, a video file of a certain resolution can be regarded as a Representation. In this application, a video file of a certain time domain level can be regarded as a Representation.

[0123] 12. Point Cloud Compression (PCC)

[0124] Point cloud coding refers to the process of encoding the geometric data and attribute data of each point in the point cloud according to a parameter set to obtain a point cloud code stream. Point cloud coding can include two main processes: geometric data encoding and attribute data encoding. The parameter set can include a geometric parameter set and an attribute parameter set. Therefore, in the encoding process, point cloud coding technology can be used to encode the geometric data of each point in the point cloud media according to the geometric parameter set in the parameter set to obtain a geometric code stream; and G-PCC can be used to encode the attribute data of each point in the point cloud media according to the attribute parameters in the parameter set to obtain an attribute code stream; the geometric code stream and the attribute code stream together constitute the point cloud code stream of the point cloud media. At present, the mainstream point cloud coding technology can be divided into point cloud coding based on geometry structure and point cloud coding based on projection according to different types of point clouds. Here, G-PCC (Geometry-based Point Cloud Compression) in MPEG and the point cloud coding standard AVS-PCC in AVS are used.

[0125] 15. Point Cloud Decoding

[0126] Point cloud decoding refers to the process of decoding the point cloud code stream obtained by point cloud encoding to reconstruct the point cloud; in detail, it refers to the process of reconstructing the geometric information and attribute data of each point in the point cloud based on the geometric code stream and attribute code stream in the point cloud code stream. After obtaining the point cloud code stream on the decoding side, for the geometric code stream, first obtain the geometric parameter set required for decoding the geometric data, and then decode the geometric code stream according to the geometric parameter set to obtain the geometric data of each point in the point cloud. For the attribute code stream, first obtain the attribute parameter set required for decoding the attribute data, and then decode the attribute code stream according to the attribute parameter set to obtain the attribute data of each point in the point cloud; then, the reconstructed attribute data of each point in the point cloud are matched one by one with the reconstructed geometric data in sequence to reconstruct the point cloud.

[0127] Based on the above description, an embodiment of the present application provides a data processing solution based on point cloud media. The general principle of this point cloud media data processing solution is as follows: on the encoding side, attribute parameter set indication information can be added to the media file of the point cloud media, and the attribute parameter set indication information is used to indicate the encapsulation location of the attribute parameter set. On the decoding side, the encapsulation location of the attribute parameter set required for decoding the attribute data in the point cloud code stream can be determined based on the attribute parameter set indication information. The attribute parameter set can then be obtained from the encapsulation location, and the attribute data in the point cloud code stream can be decoded based on the attribute parameter set, thereby presenting the point cloud media.

[0128] In one embodiment, when a point cloud code stream is encapsulated into a single track, the track may include a sample, the sample includes a sub-sample, and an attribute parameter set and the relationship between the attribute parameter set and the attribute data in the point cloud media may be identified in the sub-sample; in this implementation, the attribute parameter set indication information in the media file is used to indicate that the encapsulation position of the attribute parameter set is a sub-sample, and the attribute parameter set can also be used to indicate the relationship between the attribute parameter set and the attribute data in the point cloud code stream.

[0129] In another embodiment, when the point cloud code stream is encapsulated into multiple tracks, the attribute parameter set indication information used to indicate the encapsulation position of the attribute parameter set may include any one or more of the following: (1) the attribute parameter set is encapsulated as a separate track to obtain a parameter component track, and the attribute parameter set indication information is used to indicate that the encapsulation position of the attribute parameter set is the parameter component track. (2) Since the attribute component track must depend on the geometry component track, even if a certain attribute parameter set is referenced by different attribute components, when the attribute parameter set is located in the geometry component track, no additional decoding dependency will be introduced. Under this premise, the attribute parameter set can be directly encapsulated into the geometry component track. In this case, the attribute parameter set indication information is used to indicate that the encapsulation position of the attribute parameter set is the geometry component track. (3) The attribute parameter set can be copied multiple times and encapsulated in each attribute component track. In this case, the attribute parameter set indication information is used to indicate that the attribute parameter set is encapsulated in each attribute component track respectively. (4) The index of the attribute parameter set can be encapsulated in each attribute component track. The attribute parameter set indication information is used to indicate that the attribute parameter set is encapsulated in each attribute component track respectively, and can further indicate that the attribute parameter set can be found through the index encapsulated in any attribute component track, or the data unit formed after the attribute parameter set is reorganized can be found.

[0130] The above-mentioned data processing scheme has the following beneficial effects: through the attribute parameter set indication information contained in the media file of the point cloud media, the encapsulation position of the attribute parameter set can be indicated, thereby guiding the transmission, decoding and presentation of the point cloud media; at the same time, the codec independence between the various components of the point cloud media can be effectively utilized to solve the problem of unnecessary codec dependency caused by encapsulating different point cloud data into different file tracks in order to support partial transmission and partial decoding. For example, geometric data is encapsulated into a geometric component track; color attribute data and reflectance attribute data are respectively encapsulated into an attribute component track. At this time, if the color attribute data and the reflectance attribute data share an attribute parameter set, no matter which attribute component track this attribute parameter set is encapsulated in, a codec dependency relationship will be generated; thereby saving the use of transmission resources and computing resources.

[0131] The point cloud media data processing solution provided in the embodiments of this application can also be combined with vehicle networking technology. Specifically, the point cloud media data processing solution can collect buildings, traffic signs, and other objects in the environment and construct a point cloud map in the vehicle for positioning, or use this point cloud map for automatic navigation.

[0132] Based on the above description, the following Figure 2a The data processing system suitable for implementing the point cloud media provided in the embodiment of the present application is introduced. Figure 2a As shown, the data processing system 20 of point cloud media may include a content production device 201 and a media processing device 202. The content production device 201 is located on the encoding side of the point cloud media; the content production device 201 may be a terminal device or a server. The media processing device 202 is located on the decoding side of the point cloud media, and the media processing device 202 may be a terminal device or a server. A communication connection may be established between the content production device 201 and the media processing device 202. The terminal may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a car terminal, a smart TV, etc., but is not limited thereto. The server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0133] In one embodiment, the specific process of the content production device 201 and the media processing device 202 performing data processing of the point cloud media is as follows: For the content production device 201, the data processing mainly includes the following processes: (1) the acquisition process of the point cloud media; (2) the encoding and file packaging process of the point cloud media. For the media processing device 201, the data processing mainly includes the following processes: (3) the file decapsulation and decoding process of the point cloud media; (4) the rendering process of the point cloud media.

[0134] In addition, the transmission process of point cloud media between the content production device 201 and the media processing device 202 can be carried out based on various transmission protocols (or transmission signaling). The transmission protocols here may include but are not limited to: DASH (Dynamic Adaptive Streaming over HTTP, dynamic adaptive streaming media transmission) protocol, HLS (HTTP Live Streaming, dynamic bit rate adaptive transmission) protocol, SMTP (Smart Media Transport Protocol, smart media transmission protocol), TCP (Transmission Control Protocol, transmission control protocol), etc.

[0135] The following is a detailed description of the data processing process of point cloud media:

[0136] (1) The process of acquiring point cloud media.

[0137] Content production device 201 can acquire point cloud media. Point cloud media can be acquired through scene capture or device generation. Scene capture of point cloud media refers to point cloud media acquired by capturing real-world visual scenes using a capture device associated with content production device 201. The capture device is used to provide point cloud media acquisition services for content production device 201. The capture device may include, but is not limited to, any of the following: a camera, a sensor, or a scanning device. Cameras may include standard cameras, stereo cameras, and light field cameras. Sensors may include lasers and radars. Scanning devices may include 3D laser scanning devices. The capture device associated with content production device 201 may refer to a hardware component within content production device 201, such as a terminal camera or sensor. It may also refer to a hardware device connected to content production device 201, such as a camera connected to content production device 201. Device generation of point cloud media refers to the content production device 201 generating point cloud media based on virtual objects (e.g., virtual 3D objects and virtual 3D scenes obtained through 3D modeling).

[0138] (2) The process of encoding and file packaging of point cloud media.

[0139] The content production device 201 can encode the geometric data in the acquired point cloud media based on the geometric parameter set to obtain a geometric code stream. The content production device 201 can also encode the attribute data in the acquired point cloud media based on the attribute parameter set to obtain an attribute code stream. The geometric code stream and the attribute code stream form the point cloud code stream of the point cloud media. Furthermore, the point cloud code stream can include both the attribute parameter set and the geometric parameter set. After obtaining the point cloud code stream, the point cloud code stream can be packaged to obtain a media file for the point cloud media. Furthermore, the media file can also include attribute parameter set indication information, which indicates the location where the attribute parameter set is packaged.

[0140] When a single track is used to encapsulate a point cloud codestream into a single track, the single track may include one or more samples, each of which may contain all the data of a point cloud frame in the point cloud media, namely, geometric data, a geometric parameter set, attribute data, and an attribute parameter set; each sample is divided into one or more subsamples. In one embodiment, the attribute parameter set may be encapsulated in a subsample, and the attribute parameter set indication information may be used to indicate that the attribute parameter set is encapsulated in a subsample. In this case, a subsample may contain a parameter set in the point cloud codestream, or a subsample may contain attribute data in the point cloud codestream, or a subsample may contain geometric data in the point cloud codestream.

[0141] When a point cloud code stream is encapsulated to obtain multiple tracks using a type-based multi-track method, any track may include one or more samples, and each sample may contain a type of data of a point cloud frame in the point cloud media, such as a sample in track 1 (geometry component track) may contain the geometry data of the corresponding point cloud frame, a sample in track 2 (color component track) may contain a set of color attribute data in the corresponding point cloud frame; a sample in track 3 (parameter component track) may contain a set of attribute parameters in the point cloud frame. In this case, the attribute parameter set indication information may be used to indicate that the attribute parameter set is independently encapsulated in the parameter component track. In addition, when a point cloud code stream is encapsulated to obtain multiple tracks using a type-based multi-track method, the multiple tracks include a geometry component track and one or more attribute component tracks. In this case, the attribute parameter set may be directly encapsulated into the geometry component track, and the attribute parameter set indication information may be used to indicate that the attribute parameter set is encapsulated in the geometry component track.

[0142] After obtaining the media file, the content production device 201 can transmit the media file to the media processing device 202, so that the media processing device 202 can obtain the encapsulation position of the attribute parameter set according to the attribute parameter set indication information, and decode the attribute data of the point cloud code stream based on the attribute parameter set.

[0143] Optionally, when point cloud media is transmitted using a streaming transmission method and the attribute parameter set is independently encapsulated in a parameter component track, stream indication information may be included in the transmission signaling, and the stream indication information is used to indicate the data stream corresponding to the parameter component track, and transmit the above-mentioned media file to the media processing device 202 through the transmission signaling.

[0144] (3) The process of decapsulating and decoding point cloud media files;

[0145] The media processing device 202 can obtain the media files and corresponding media presentation description information of the point cloud media through the content production device 201. The media files and media presentation description information of the point cloud media are transmitted from the content production device 201 to the media processing device 202 through transmission signaling (such as DASH, SMT). The file decapsulation process of the media processing device 202 is the opposite of the file encapsulation process of the media processing device 202. The media processing device 202 decapsulates the media file resources according to the file format requirements of the point cloud media to obtain a point cloud code stream. The decoding process of the media processing device 202 is the opposite of the encoding process of the content production device 201. The media processing device 202 decodes the point cloud code stream to restore the point cloud media.

[0146] In the decoding process, the media processing device 202 can obtain attribute parameter set indication information from the media file, and determine the encapsulation position of the attribute parameter set based on the attribute parameter indication information, and obtain the attribute parameter set from the encapsulation position, and then decode the attribute data in the media file based on the attribute parameter set, and finally restore the point cloud media.

[0147] Optionally, the attribute parameter indication information can be used to indicate the relationship between the attribute parameter set and the attribute data in the point cloud codestream. The relationship can be that the attribute parameter set is only effective for a certain group of attribute data in the point cloud codestream, or that the attribute parameter set is only effective for a certain type of attribute data in the point cloud codestream, or that the attribute parameter set is effective for all attribute data in the point cloud codestream. The so-called effectiveness refers to validity, that is, the attribute data in the point cloud codestream can be decoded according to the effective attribute parameter set.

[0148] Among them, the point cloud code stream may include multiple groups of attribute data, for example: multiple groups of attribute data include: a group of yellow attribute data, a group of red attribute data, and a group of reflectivity attribute data; then, the attribute parameter set is only effective for a certain group of attribute data in the point cloud code stream, which may mean that it is only effective for a group of yellow attribute data; and the attribute parameter set is only effective for a certain type of attribute element data in the point cloud code stream, which may mean that it is effective for color attribute data. Since both yellow attribute data and red attribute data belong to color attribute data, the attribute parameter set is only effective for a certain type of attribute element data in the point cloud code stream, which may include being effective for both yellow attribute data and red attribute data.

[0149] (4) Rendering process of point cloud media.

[0150] The media processing device 202 renders the decoded point cloud media according to the metadata related to rendering and viewport in the media presentation description information to obtain the point cloud frame of the point cloud media, and presents the point cloud media according to the presentation time of the point cloud frame.

[0151] In one embodiment, see Figure 2b, content production device side: first, the real-world visual scene A is sampled through the acquisition device to obtain point cloud source data B of the point cloud media corresponding to the real-world visual scene, where the point cloud source data B is a frame sequence composed of a large number of point cloud frames; then, the acquired point cloud media is encoded according to a parameter set (i.e., a geometric parameter set and an attribute parameter set) to obtain a point cloud code stream E (including an encoded geometric code stream and an attribute code stream). Optionally, the point cloud code stream E may also include a parameter set; then, the point cloud code stream E is encapsulated to obtain a media file corresponding to the point cloud media. Specifically, the content production device 201 synthesizes one or more encoded code streams into a media file F for file playback, or an initialization segment and a sequence of media segments (FS) for streaming according to a specific media container file format; wherein the media container file format may refer to the ISO basic media file format specified in the International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC) 14496-12. The media file may also include parameter set indication information, which is used to indicate the encapsulation location of the parameter set. In one embodiment, the content production device also encapsulates the metadata into the media file or the sequence of initialization / media segments, and transmits the sequence of initialization / media segments to the media processing device 202 via transmission signaling (such as a dynamic adaptive streaming media transmission interface).

[0152] At the media processing device end: first, the media file sent by the content production device 201 is received, and the media file may include: a media file F' for file playback, or an initialization segment and a sequence of media segments Fs' for streaming; then the media file is decapsulated to obtain a point cloud code stream E'; then the parameter set is obtained according to the encapsulation position indicated by the parameter set indication information contained in the media file, and the point cloud code stream is decoded according to the parameter set (that is, the attribute data in the point cloud code stream is decoded according to the attribute parameter set in the parameter set and the geometric data in the point cloud code stream is decoded according to the geometric parameter set in the parameter set) to obtain point cloud media D'; in a specific implementation, the media processing device determines the media file or media segment sequence required to present the point cloud media based on the viewing position / viewing direction of the current object; and decodes the media file or media segment sequence required to present the point cloud media to obtain the point cloud media required for presentation. Finally, based on the viewing (viewing window) direction of the current object, the decoded point cloud media is rendered to obtain the point cloud frame A' of the point cloud media, and the point cloud media is presented on the screen of the head-mounted display carried by the media processing device or any other display device according to the presentation time of the point cloud frame. It should be noted that the viewing position / viewing direction of the current object is determined by the head following and possibly the visual following function. In addition to the renderer used to render the point cloud media of the viewing position / viewing direction of the current object, the audio of the viewing (viewing window) direction of the current object can also be decoded and optimized by the audio decoder. Among them, in the window-based transmission process, the current viewing position and viewing direction are also passed to the strategy module to determine the track to be received.

[0153] It is understood that the data processing technology for point cloud media involved in this application can be implemented using cloud technology; for example, using a cloud server as a content production device. Cloud technology refers to a hosting technology that unifies hardware, software, network and other resources within a wide area network or local area network to achieve data computing, storage, processing and sharing.

[0154] In an embodiment of the present application, a media file of point cloud media is obtained, and the media file includes a point cloud code stream and attribute parameter set indication information of the point cloud media. The point cloud code stream includes an attribute parameter set, and the attribute parameter set includes parameter information required for decoding attribute data in the point cloud code stream; the attribute parameter set indication information is used to indicate the encapsulation position of the attribute parameter set; according to the attribute parameter set indication information, the encapsulation position of the attribute parameter set can be directly known, and the point cloud code stream can be decoded based on the attribute parameter set to present the point cloud media, which guides the transmission, decoding and presentation of the point cloud media, can reduce unnecessary encoding and decoding dependencies, and save transmission resources and computing resources.

[0155] Next, the data processing method of point cloud media in the embodiment of this application is described. Figure 3 , Figure 3 This is a flow chart of a method for processing point cloud media data provided in an embodiment of the present application. This method can be executed by the above-mentioned media processing device. The method for processing point cloud media data described in this embodiment can include the following steps S301-S302:

[0156] S301. Obtain a media file of the point cloud media, where the media file includes a point cloud code stream of the point cloud media. The point cloud code stream may include an attribute parameter set, where the attribute parameter set includes parameter information required for decoding attribute data in the point cloud code stream; the media file also includes attribute parameter set indication information, where the attribute parameter set indication information is used to indicate the encapsulation location of the attribute parameter set.

[0157] In one embodiment, the attribute parameter set indication information can also be used to indicate the relationship between the attribute parameter set and the attribute data in the point cloud code stream, that is, the attribute parameter set indication information can indicate the attribute parameter set and the attribute data corresponding to the attribute parameter set. For example, the attribute parameter set indication information indicates that the attribute parameter set is effective for a specific type of attribute data (such as color) in the point cloud code stream; for another example, the attribute parameter set indication information indicates that the attribute parameter set is effective for a specific set of attribute data (such as a set of yellow attribute data) in the point cloud code stream; for another example, the attribute parameter set indication information indicates that the attribute parameter set is effective for all attribute data in the point cloud code stream.

[0158] The media file is obtained by encapsulating the point cloud code stream of the point cloud media. In the embodiment of the present application, starting from the single-track mode and the multi-track mode, there are several implementation methods (1)-(2) for the encapsulation position of the attribute parameter set:

[0159] (1) When the point cloud code stream of the point cloud media is encapsulated into the same track in a single-track manner, the attribute parameter set can be encapsulated in a subsample information data box.

[0160] When a point cloud codestream of a point cloud media is encapsulated into the same track using a single track, the media file includes a track, which may include one or more samples, where one sample corresponds to one point cloud frame in the point cloud media; a sample may be divided into one or more subsamples, each of which may include any type of point cloud data in the point cloud codestream, such as a subsample may include a parameter set, or a subsample may include attribute data or geometric data; the media file may further include a subsample information data box, where the attribute parameter set may be encapsulated in the subsample information data box, and in this case, the attribute parameter set indication information is used to indicate that the attribute parameter set is encapsulated in the subsample information data box. The attribute parameter set indication information may be represented as a subsample information data box, which may include at least one of the following fields: a payload type field, a reference attribute information flag field, a reference attribute type flag field, a reference attribute identification field, and a reference attribute type field.

[0161] Specifically, a subsample information data box can be used when encapsulating a point cloud code stream; when a point cloud frame can be divided into one or more point cloud slices, the attribute data and / or geometric data contained in the subsample belongs to the one or more point cloud slices; the subsample information data box can include a flag field of the subsample information data. A subsample can be defined according to the value of the flag field (flag) of the subsample information data, and the flag field specifies the type of subsample in the subsample information data box: if the flag field is a first preset value (such as 0), then the subsample refers to a subsample based on the data type carried by the point cloud slice. In this case, a subsample only contains one data type and related data, such as a subsample only contains a geometric data type and geometric data. If the flag field is a second preset value (such as 1), then the subsample refers to a subsample based on the point cloud slice. In this case, a subsample only contains all relevant data of a point cloud slice, that is, a geometric slice header and geometric data, an attribute slice header and attribute data. Of course, the flag field can retain other flag values.

[0162] At this time, the codec_specific_parameters field of the subsample information data box may be defined as shown in Table 1:

[0163] Table 1

[0164]

[0165]

[0166] The meanings of the fields in the subsample information data box are as follows:

[0167] Payload Type field (payloadType): The payload type field is used to indicate the type of point cloud data contained in the subsample. If the payload type field is a first value (such as 2), it indicates that the type of point cloud data contained in the subsample is an attribute parameter set. If the payload type field is a second value (such as 3), it indicates that the type of point cloud data contained in the subsample is a geometry parameter set. If the payload type field is a third value (such as 0), it indicates that the type of point cloud data contained in the subsample is attribute data. If the payload type field is a fourth value (such as 1), it indicates that the type of point cloud data contained in the subsample is geometry data. It should be understood that the first value, the second value, the third value, and the fourth value can be arbitrarily set according to requirements, such as the first value can be 0, the second value can be 1, the third value can be 2, and the fourth value can be 3. This embodiment of the present application does not limit this.

[0168] Attribute Presentation Flag Field (attribute_present_flag): This attribute presentation flag field can be used to indicate whether the subsample contains color attribute data and / or reflectance attribute data. For definitions, please refer to AVS (China National Audio and Video Coding Standard AVS) - PCC. If the value of the attribute presentation flag field is a first setting value (such as 0), it indicates whether the subsample contains color attributes; if the value of the attribute presentation flag field is a second setting value (such as 1), it indicates whether the subsample contains reflectance attributes.

[0169] Slice data field (slice_data): This field indicates whether the subsample contains slice data. If the value of the slice data field is the first setting value (such as 1), it indicates that the subsample contains slice geometry and / or attribute type data; if the value of the slice data field is the second setting value (such as 0), it indicates that the subsample does not contain point cloud parameter information.

[0170] Point cloud slice identification field (slice_id): This field indicates the ID of the point cloud slice corresponding to the data contained in the subsample.

[0171] Reference attribute information flag field (ref_attr_info_flag): This reference attribute information flag field is used to indicate the scope of validity of the current attribute parameter set for attribute data. If the reference attribute information flag field is a first preset value (e.g., 1), it indicates that the current attribute parameter set is valid only for a specific group (i.e., a certain group) or a specific type (i.e., a certain class) of attribute data in the current sample. If the reference attribute information flag field is a second preset value (e.g., 0), it indicates that the current attribute parameter set is valid for all attribute data in the current sample.

[0172] Reference attribute type flag field (ref_attr_type_flag): The reference attribute type flag field is used to indicate the type of attribute data to which the current attribute parameter set is effective; if the reference attribute type flag field is a first value (such as 1), it indicates that the current attribute parameter set is only effective for a specific type of attribute data in the current sample; if the reference attribute type flag field is a second value (0), it indicates that the current attribute parameter set is only effective for a specific set of attribute data in the current sample.

[0173] Reference attribute identification field (ref_attr_id): This field is used to indicate the identifier of the attribute data corresponding to the current attribute parameter set, which can be understood as the identifier of the attribute data for which the current attribute parameter set is effective.

[0174] Reference attribute type field (ref_attr_type): This field is used to indicate the type of attribute data corresponding to the current attribute parameter set; that is, it can be understood as the type of attribute data in which the current attribute parameter set is effective; for example, the reference attribute type field can be used to indicate that the type of attribute data corresponding to the current attribute parameter set is color; for another example, the reference attribute type field can be used to indicate that the type of attribute data corresponding to the current attribute parameter set is reflectivity.

[0175] Reserved field: The value of this reserved field can be 0.

[0176] The current attribute parameter set refers to the attribute parameter set being used, and the current sample refers to the sample being decoded.

[0177] (2) When the point cloud stream of the point cloud media is encapsulated into multiple tracks in a multi-track manner, the encapsulation position of the attribute parameter set can be as follows:

[0178] a. Attribute parameter collections are independently encapsulated in parameter component tracks.

[0179] When a point cloud stream is encapsulated into a media file using a multi-track approach, embodiments of the present application can define a parameter component track. The point cloud stream is encapsulated into the media file using a type-based multi-track approach. In this case, the media file can include a parameter component track, in which the attribute parameter set is independently encapsulated. The parameter component track can have a sample entry type of 'apcp' or 'gpp1'; be included in a sample description data box (SampleDescriptionBox); be mandatory: no; and have one or more parameters.

[0180] In this case, the attribute parameter set indication information can be used to indicate that the attribute parameter set is independently encapsulated into a parameter component track. The parameter component track contains one or more samples, each of which corresponds to a point cloud frame of the point cloud media and contains the parameter information required to decode the attribute data of the corresponding point cloud frame.

[0181] In one embodiment, the media file includes one or more parameter component tracks; in a feasible implementation, when the media file includes only one parameter component track, the attribute parameter set in the point cloud code stream is encapsulated into the parameter component track. It can be understood that under this implementation, the geometric parameter set in the point cloud code stream can also be encapsulated in the parameter component track, that is, the geometric parameter set and the attribute parameter set of the point cloud code stream can both be encapsulated in the parameter component track. In another feasible implementation, when the media file includes multiple parameter component tracks, the geometric parameter set in the point cloud code stream can be encapsulated into a separate parameter component track, and the attribute parameter set in the point cloud code stream can be encapsulated into another separate parameter component track. In another feasible implementation, when the media file includes multiple parameter component tracks, the geometric parameter set in the point cloud code stream can be encapsulated into a separate parameter component track, and the attribute parameter set in the point cloud code stream can be encapsulated into multiple parameter component tracks based on type. For example, the attribute parameter set includes a color attribute parameter set and a reflectivity attribute parameter set. When the media file includes multiple parameter component tracks, the color attribute parameter set can be encapsulated into a separate parameter component track, and the reflectivity attribute parameter set can be encapsulated into another separate parameter component track.

[0182] In multi-track mode, the attribute parameter set in the point cloud code stream is encapsulated as a parameter component track. The parameter component track can contain a track data box (TrackBox), and the track data box can contain a track reference type data box (TrackReferenceTypeBox), and the reference type (reference_type) of the track reference type data box is 'appr'. The track reference type data box is used to indicate that the parameter component track is associated with one or more attribute component tracks (i.e., point cloud data tracks) corresponding to the attribute parameter set in the parameter component track. It should be understood that: the track reference type data box is used to reference or associate one or more attribute component tracks corresponding to the attribute parameter set in the parameter component track. For example, if the attribute parameter set in the parameter component track includes color parameters and reflectance parameters, then the track reference type data box is used to reference or associate the color attribute component track and the reflectance attribute component track corresponding to the attribute parameter set in the parameter component track.

[0183] In one embodiment, when the media file complies with a first coding standard (e.g., the AVS coding standard), the parameter component track includes a component information data box (AVSPCCComponentInfoBox), which is set in the header sample entry (AvsPccParaHeaderSampleEntry) of the parameter component track; the component information data box includes at least one of the following fields: a component type field, a header type field, a number of attribute headers field, a reference attribute type flag field, a reference attribute identifier field, and a reference attribute type field; the syntax of the component information data box can be seen in Table 2:

[0184] Table 2

[0185]

[0186] The meanings of the fields in the component information data box are as follows:

[0187] Component type field (avs_pcc_type): The component type field is used to indicate the component type in the track; when the component type field is set to the target preset value (such as 5), it indicates that the component type in the parameter component track is a parameter set. The corresponding relationship between the value of the component type field and the component type can be shown in Table 3:

[0188] Table 3

[0189] avs_pcc_type value describe 1 reserve 2 Geometric data 3 reserve 4 Attribute data 5 Parameter Collection 6..31 reserve

[0190] Attribute type field (attr_type): This attribute type field is used to indicate the type of attribute components contained in the track. If the attribute type field is the first value (such as 0), it indicates that the track only contains color attribute data; if the attribute type field is the second value (such as 1), it indicates that it only contains reflectance attribute data; if the attribute type field is the third value (such as 2), it indicates that it contains color attribute data and reflectance attribute data.

[0191] Header type field (header_type): This header type field is used to indicate the parameter set type contained in the parameter component track; if the header type field is the first setting value (such as 0), it indicates that the parameter set type contained in the parameter component track is an attribute parameter set; if the header type field is the second setting value (such as 1), it indicates that the parameter set type contained in the parameter component track is a geometry parameter set; if the header type field is the third setting value (such as 2), it indicates that the parameter set types contained in the parameter component track are a geometry parameter set and an attribute parameter set. Among them, when the attribute parameter set in the point cloud codestream is encapsulated into multiple parameter component tracks, such as the geometry parameter set is encapsulated into a separate parameter component track and the attribute parameter set is encapsulated into a separate parameter component track, the value of the header type field can only be the first setting value and the second setting value.

[0192] Attribute header number field (attr_header_num): used to indicate the number of attribute parameter sets contained in the sample in the parameter component track, for example, the number of attribute parameter sets can be 1, 2, etc.; wherein, the order of the attribute parameter sets contained in the sample is consistent with the order of the attribute parameter set identifiers or attribute parameter set types indicated in the component information data box; for example, the order of the attribute parameter sets contained in the sample is attribute parameter set 1 and attribute parameter set 2, then the order of the attribute parameter set identifiers indicated in the component information data should also be the identifier of attribute parameter set 1 and the identifier of attribute parameter set 2.

[0193] Reference attribute type field (ref_attr_type_flag): The reference attribute type field is used to indicate the type of attribute data to which the current attribute parameter set is effective; if the reference attribute type flag field is the first preset value (such as 0), it means that the current attribute parameter set is only effective for a specific type of attribute data in the current sample; if the reference attribute type flag field is the second preset value (such as 1), it means that the current attribute parameter set is effective for a specific set of attribute data in the current sample.

[0194] Reference attribute identification field (ref_attr_id): The reference attribute identification field is used to indicate the identifier of the attribute data corresponding to the current attribute parameter set.

[0195] Reference attribute type (ref_attr_type): This reference attribute type is used to indicate the type of attribute data corresponding to the current attribute parameter set;

[0196] The current attribute parameter set refers to the attribute parameter set being used, and the current sample refers to the sample being decoded.

[0197] In another embodiment, the media file may comply with a second coding standard (such as the MPEG coding standard); the parameter component track includes a component information data box (GPCCComponentInfoBox) and a configuration data box (GPCCConfigurationBox); wherein the syntax of the configuration data box may be as shown in Table 4:

[0198] Table 4

[0199]

[0200]

[0201] The meanings of the fields included in the configuration data box are as follows:

[0202] Configuration Version field (configurationVersion): The configuration version field is used to record the version information of the configuration. If the version information of the configuration version field in the corresponding track cannot be recognized by the player / decoder, the player / decoder should not attempt to decode the corresponding track.

[0203] Profile field (profile_idc): The profile field is used to indicate the profile information of the code stream corresponding to the current track.

[0204] Profile compatibility field (profile_compatibility_flags): The profile compatibility field is used to indicate the profile compatibility information of the code stream corresponding to the current track.

[0205] Number of setup units field (num_of_setup_unit_arrays): The number of setup units field is used to indicate the number of parameter set groups included in the current decoder configuration record. Each parameter set group corresponds to a type of parameter set.

[0206] Setup unit type field (setup_unit_type): The setup unit type field is used to indicate the parameter set type corresponding to the current parameter set group.

[0207] Parameter Set Indication Field (array_completeness): This parameter set indication field is used to indicate whether the parameter set of the current type contained in the corresponding point cloud codestream is indicated in the decoder configuration record. When the parameter set indication field is the first value (such as 1), it indicates that all parameter sets of the current type contained in the corresponding codestream are indicated in the decoder configuration record. When the parameter set indication field is the first value (such as 0), it indicates that part of the parameter set of the current type contained in the corresponding codestream is indicated in the decoder configuration record and part is indicated in the codestream.

[0208] The number of setup units field (num_of_setup_units): The number of setup units field is used to indicate the number of parameter sets of the current type. A setup unit (setupUnit) contains a data unit of SPS (sequence parameter set), GPS (geometry parameter set), APS (attribute parameter set), or point cloud block information.

[0209] In one embodiment, the parameter set type corresponding to the setup unit included in the configuration data box is the same as the parameter set type indicated by the component information data box; the setup unit (setupUnit) may include a sequence parameter set (SPS), a geometric parameter set (i.e., GPS), an attribute parameter set (i.e., APS), or a data unit of point cloud segmentation information. It should be understood that: when the parameter set type corresponding to the setup unit is an attribute parameter set, the parameter set type indicated by the component information data box is also an attribute parameter set; when the parameter set type corresponding to the setup unit included in the configuration data box is a geometric parameter set, the parameter set type indicated by the component information data box is also a geometric parameter set.

[0210] The component information data box contains at least one of the following fields: component type field, parameter setting type quantity field, and parameter setting type field. The syntax of the component information data box is shown in Table 5:

[0211] Table 5

[0212]

[0213]

[0214] The meanings of the fields in the component information data box are as follows:

[0215] Component type field (gpcc_type): This field is used to indicate the type of component in the track. If the component type field is a target setting value (such as 5), it indicates that the type of component in the parameter component track is a parameter set. The correspondence between the value of the component type field and the component type can be shown in Table 6:

[0216] Table 6

[0217] gpcc_type value describe 1 reserve 2 Geometric data 3 reserve 4 Attribute data 5 Parameter Collection 6..31 reserve

[0218] Parameter Set Type Number Field (num_para_set_type): This field is used to indicate the number of parameter sets contained in the parameter component track. For example, the number of parameter sets contained in a parameter component track can be 1, 2, and so on. Optionally, when a media file includes multiple parameter component tracks, such as when a geometry parameter set is separately encapsulated into a parameter component track and an attribute parameter set is separately encapsulated into another parameter component track, each parameter component track contains only one type of parameter set, and the Parameter Set Type Number field is not required.

[0219] Parameter setting type field (parameter_set_type): The parameter setting type field is used to indicate the type of parameter set contained in the parameter component track; if the parameter setting type field is the first preset value (such as 0), it means that the type of parameter set contained in the parameter component track is an attribute parameter set; if the parameter setting type field is the second preset value (such as 1), it means that the type of parameter set contained in the parameter component track is a geometric parameter set; if the parameter setting type field is the third preset value (such as 2), it means that the type of parameter set contained in the parameter component track is a sequence parameter set.

[0220] Attribute index field (attr_index): The attribute index field is used to indicate the sequence number of the attribute parameter indicated in the SPS (ie, attribute parameter set).

[0221] The attribute type present field (attr_type_present_flag) is used to indicate whether the attribute data type information is indicated in the component information data box. If the attribute type present field is the first value (such as 1), it indicates that the attribute data type information is indicated in the component information data box. If the attribute type present field is the second value (such as 0), it indicates that the attribute data type information is indicated in the component information data box. The relationship between the four bytes and the attribute data type is shown in Table 7:

[0222] Table 7 Attribute label four bytes (attribute_label_four_bytes)

[0223] Attribute tag four bytes [i] Attribute Type 0 Colour 1 Reflectancestry 2 Frame index 3 Material ID 4 Transparency 5 Normals 6..255 Reserved 256..0xffffffff Unspecified

[0224] Attribute type field (attr_type): The attribute type field is used to indicate the type of the attribute component. Its value refers to the definition in the ISO / IEC 23090-9 [GPCC] standard.

[0225] Attribute name field (attr_name): The attribute name field is used to indicate human-readable attribute component type information.

[0226] b. The attribute parameter collection is encapsulated in the geometry parameter component track.

[0227] When encapsulating point cloud streams into multiple tracks using a multi-track approach, in addition to encapsulating attribute parameter sets into independent parameter component tracks, you can also encapsulate all attribute parameter sets into the geometry component track contained in the media file. Because the attribute component track is always dependent on the geometry component track, even if an attribute parameter set is referenced by different attribute components, when the attribute parameter set is in the geometry component track, no additional decoding dependencies are introduced.

[0228] In a multi-track mode, the above media file may include a geometry component track and at least one attribute component track; wherein, the geometry component track and the attribute component track in the AVS PCC need to use the track reference tool defined in ISO / IEC14496-12 [ISOBMFF] for associated indexing.

[0229] Among them, the geometry component track can include one or more samples. In the sample entry of the geometry component track, there is a decoder configuration data box, which contains parameter sets such as SPS (Sequence Parameter Set), GPS, APS, etc., while the attribute component track should not contain a decoder configuration data box.

[0230] When the point cloud codestream contains an attribute parameter set, the attribute parameter set should be included in the corresponding sample of the geometry component track. The attribute component track should not contain any type of attribute parameter set, only the corresponding type of attribute data. Based on this, the relevant fields of the component data information box are modified. The modified fields are as follows: Component type field (gpcc_type): The component type field is used to indicate the type of component in the track; the value of the component type field is shown in Table 8 below:

[0231] Table 8

[0232] gpcc_type value describe 1 reserve 2 Geometry data and parameter sets 3 reserve 4 Attribute data (excluding parameter sets) 5…31 reserve

[0233] When a media file includes a geometry component track and an attribute component track, the attribute parameter set is encapsulated in the geometry component track. The geometry component track may include a component information data box. The component information data box includes a component type field. The component type field is used to indicate the type of the component in the geometry component track. If the component type field is a first setting value (such as 2), it indicates that the type of the component in the geometry component track is geometry data and a parameter set. The parameter set may include an attribute parameter set and a geometry parameter set. In the attribute component track, the component type field included in the component information data box is a second setting value (such as 4), indicating that the types of the components in the attribute component track are all attribute data and do not include a parameter set.

[0234] c. A collection of attribute parameters can be encapsulated into each attribute component track.

[0235] A media file may include one or more attribute component tracks, and an attribute parameter set may be copied in each attribute component track so that each attribute component track contains a corresponding attribute parameter set. At this time, the attribute parameter set is respectively encapsulated in each attribute track; the attribute parameter set indication information may be used to indicate that the attribute parameter set is respectively encapsulated in each attribute track. Among them, the attribute parameter set encapsulated in each attribute component track may be obtained by copying the same attribute parameter set. That is to say, when the number of attribute component tracks is N, the attribute parameter set may be directly copied N times, and one attribute parameter set may be placed in each attribute component track.

[0236] d. The index of the attribute parameter collection can be encapsulated into each attribute component track.

[0237] The attribute parameter set includes an index, and the attribute parameter set can be repeatedly indexed in each attribute component track. That is, the index of the attribute parameter set is set in each attribute component track, and the attribute parameter set indication information is used to indicate that the index of the attribute parameter set is encapsulated in each attribute component track. The attribute parameter set, or the data unit formed by reorganizing the attribute parameter set, can be found through the index encapsulated in any attribute component track.

[0238] In one embodiment, the point cloud code stream can be reorganized to index samples of different types of attribute component tracks and corresponding attribute parameter sets into the same data unit. Figure 4aAs shown, the point cloud code stream includes a geometry parameter set (GPS), geometry data (Geometry), an attribute parameter set (APS), attribute data 1 (Attribute1), and attribute data 2 (Attribute2); after reorganizing the point cloud code stream, the samples in the geometry component track to which the geometry parameter set and the geometry data belong can be indexed to the same data unit 41; the samples in the attribute component track to which the attribute parameter set and the attribute data 1 belong can be indexed to the same data unit 42; and the samples in the attribute component track to which the attribute parameter set and the attribute data 2 belong can be indexed to the same data unit 43. For another example, Figure 4b As shown, after the point cloud code stream is reorganized, the geometric parameter set and the samples in the geometric component track to which the geometric data belong can be directed to the same data unit 44, and the attribute parameter set, the samples in the attribute component track to which the attribute data 1 belongs, and the samples in the attribute component track to which the attribute data 2 belongs can be indexed to the same data unit 45; in the data unit 45, the index encapsulated in the attribute component track to which the attribute data 1 belongs and the index encapsulated in the attribute component track to which the attribute data 2 belongs can both be indexed into the data unit 45.

[0239] In one embodiment, when point cloud media is transmitted using a streaming transmission method; obtaining the media file of the point cloud media may include: obtaining the transmission signaling of the point cloud media, and then obtaining the media file of the point cloud media according to the transmission signaling. When transmitting media files through transmission signaling, since the media file is transmitted in a streaming transmission method, when the attribute data needs to be decoded, it is necessary to request the data stream corresponding to the parameter component track to obtain the corresponding attribute parameter set, so as to implement the transmission guidance of the media file corresponding to the point cloud media. It should be noted that in the streaming transmission method, if the attribute parameter set is independently encapsulated into the parameter component track, then the transmission signaling also includes stream indication information, and the stream indication information is used to indicate the data stream corresponding to the parameter component track.

[0240] Among them, the data stream can be a representation (Representation), the transmission signaling can be DASH signaling, and the stream indication information can refer to a component descriptor (GPCCComponent descriptor), which is set in the data stream corresponding to the parameter component track; the component descriptor includes a composition type element and a parameter type element, and the composition type element is used to indicate the type of the component in the parameter component track; if the composition type element (component@component_type) is a target character (such as 'prms'), the type of the component in the parameter component track is a parameter component. The parameter type element is used to indicate the type of the parameter set contained in the parameter component track. Among them, the attributes and elements of the component descriptor can be shown in Table 9:

[0241] Table 9

[0242]

[0243]

[0244] Among them, M: Mandatory mandatory field; CM: Conditional Mandatory; O: Optional field.

[0245] S302: Decode the point cloud code stream according to the attribute parameter set indication information to present point cloud media.

[0246] In one embodiment, decoding the point cloud code stream according to the attribute parameter set indication information to present the point cloud media may include: obtaining the attribute parameter set according to the encapsulation position indicated by the attribute parameter set indication information; and then decoding the attribute data in the point cloud code stream using the attribute parameter set to present the point cloud media. Specifically, the encapsulation position indicated by the attribute parameter set indication information may be first decoded to obtain the attribute parameter set. For example, the encapsulation position indicated by the attribute parameter set indication information is a parameter component track, and the parameter component track needs to be decoded to obtain the attribute parameter set. Then, the attribute data that needs to be decoded is determined according to the needs, and the attribute data is decoded using the attribute parameter set. In this way, the relevant point cloud media can be partially decoded, saving decoding resources.

[0247] In an embodiment of the present application, a media file of point cloud media is obtained, and the media file includes a point cloud code stream and attribute parameter set indication information of the point cloud media. The point cloud code stream includes an attribute parameter set, and the attribute parameter set includes parameter information required for decoding attribute data in the point cloud code stream; the attribute parameter set indication information is used to indicate the encapsulation position of the attribute parameter set; according to the attribute parameter set indication information, the encapsulation position of the attribute parameter set can be directly known, and the point cloud code stream can be decoded based on the attribute parameter set to present the point cloud media, which guides the transmission, decoding and presentation of the point cloud media, can reduce unnecessary encoding and decoding dependencies, and save transmission resources and computing resources.

[0248] See Figure 5 , Figure 5 This is a flow chart of a method for processing point cloud media data provided by an embodiment of the present application. The method for processing point cloud media data can be executed by a content production device. The method for processing point cloud media data can include the following steps S501-S503:

[0249] S501: Obtain point cloud media.

[0250] S502: Encode the point cloud media to obtain a point cloud code stream. The point cloud code stream includes an attribute parameter set, which includes parameter information required for decoding attribute data in the point cloud code stream.

[0251] The specific implementation method of encoding the point cloud media can be found in the description of the corresponding part above, which will not be repeated here.

[0252] S503: Encapsulate the point cloud code stream to obtain a media file of the point cloud media; the media file also includes attribute parameter set indication information.

[0253] Among them, attribute parameter indication information can be added to the media file, and the attribute parameter set indication information can be used to indicate the encapsulation position of the attribute parameter set; in addition, the attribute parameter set indication information can also be used to indicate the relationship between the attribute parameter set and the attribute data in the point cloud code stream, such as the attribute parameter set is effective for a certain type of attribute data or a certain group in the point cloud code stream.

[0254] There are several ways to encapsulate the point cloud stream and obtain the point cloud media file:

[0255] (1) The media file includes a track, in which the point cloud code stream is encapsulated. The track contains one or more samples, and one sample corresponds to a point cloud frame in the point cloud media. A sample is divided into one or more subsamples. Each subsample contains a type of point cloud data. The media file also includes a subsample information data box, in which a set of attribute parameters is encapsulated.

[0256] As an implementation method, the content production device can use a single-track method to encapsulate the point cloud code stream into a track, which contains one or more samples, and one sample corresponds to a point cloud frame in the point cloud media; then each sample is divided into one or more sub-samples; when the point cloud code stream of the point cloud media is encapsulated into the same track in a single-track method, a sub-sample information data box can be used when encapsulating the point cloud code stream, and the attribute parameter set in the point cloud code stream can be set in the sub-sample information data box to form a media file of the point cloud media; wherein, the attribute parameter set indication information is used to indicate that the attribute parameter set is set in the sub-sample information data box.

[0257] Among them, the subsample information data box may include a flag field of subsample information data. When a point cloud frame can be divided into one or more point cloud slices, the attribute data and / or geometric data contained in the subsample belongs to the one or more point cloud slices; the subsample is defined according to the value of the flag field (flag) of the subsample information data, and the flag field specifies the type of subsample in the subsample information data box: if the flag field is a first preset value (such as 0), then the subsample refers to a subsample based on the data type carried by the point cloud slice. At this time, a subsample only contains one data type and related data, such as a subsample only contains a geometric data type and geometric data. If the flag field is a second preset value (such as 1), then the subsample refers to a subsample based on the point cloud slice. At this time, a subsample only contains all relevant data of a point cloud slice, that is, a geometric slice header and geometric data, an attribute slice header and attribute data.

[0258] The subsample information data box may include at least one of the following fields: a payload type field, a reference attribute information flag field, a reference attribute type flag field, a reference attribute identifier field, and a reference attribute type field. The payload type field is used to indicate the type of point cloud data contained in the subsample packet; the reference attribute information flag field is used to indicate the scope of validity of the current attribute parameter set for the attribute data; the reference attribute type flag field is used to indicate the type of attribute data valid for the current attribute parameter set; the reference attribute identifier field is used to indicate the identifier of the attribute data corresponding to the current attribute parameter set; and the reference attribute type field is used to indicate the type of attribute data corresponding to the current attribute parameter set.

[0259] At this time, setting the attribute parameter set in the point cloud codestream in the subsample information data box may include: if the type of point cloud data contained in the subsample is an attribute parameter set, setting the payload field to a first value (e.g., 2); if the type of point cloud data contained in the subsample is a geometry parameter set, setting the payload field to a second value (e.g., 3); if the current attribute parameter set is only valid for a specific group (i.e., a certain group) or a specific type (i.e., a certain class) of attribute data in the current sample, setting the reference attribute information flag field to a first preset value (e.g., 1); if the current attribute parameter set is valid for all attribute data in the current sample, setting the reference attribute information flag field to a second preset value (e.g., 0); if the current attribute parameter set is only valid for a specific type of attribute data in the current sample, setting the reference attribute type flag field to a first value (e.g., 1); if the current attribute parameter set is only valid for a specific group of attribute data in the current sample, setting the reference attribute type flag field to a second value (e.g., 0). Wherein, the current attribute parameter set refers to the attribute parameter set being used, and the current sample refers to the sample being encoded.

[0260] (2) The point cloud code stream is encapsulated into a media file in a multi-track manner; the media file includes a parameter component track, and the attribute parameter set is independently encapsulated in the parameter component track; the parameter component track contains one or more samples, each sample corresponds to a point cloud frame of the point cloud media, and each sample contains the parameter information required for decoding the attribute data of the corresponding point cloud frame. The attribute parameter set indication information is used to indicate that the attribute parameter set is independently encapsulated into the parameter component track

[0261] In which, the media file includes one or more parameter component tracks; in a feasible embodiment, when the media file includes only one parameter component track, the attribute parameter set in the point cloud code stream is encapsulated into the parameter component track; it can be understood that under this embodiment, the geometric parameter set in the point cloud code stream can also be encapsulated in the parameter component track, that is, the geometric parameter set and attribute parameter set of the point cloud code stream can both be encapsulated in the parameter component track. In another feasible embodiment, when the media file includes multiple parameter component tracks, the geometric parameter set in the point cloud code stream can be encapsulated into a separate parameter component track, and the attribute parameter set in the point cloud code stream can also be encapsulated into a separate parameter component track. In yet another feasible embodiment, when the media file includes multiple parameter component tracks, the geometric parameter set in the point cloud code stream can be encapsulated into a separate parameter component track, and the attribute parameter set in the point cloud code stream can be encapsulated into multiple parameter component tracks based on type.

[0262] Specifically, a type-based multi-track approach can be used to encapsulate the attribute parameter set in a point cloud codestream into a parameter component track, the geometric data in the point cloud codestream into a geometry component track, and the attribute data in the point cloud codestream into an attribute component track, ultimately forming a media file for point cloud media. This media file may include a parameter component track, an attribute component track, and a geometry component track. The parameter component track contains one or more samples, each corresponding to a point cloud frame of the point cloud media and containing parameter information required for decoding the attribute data of the corresponding point cloud frame. The attribute component track may contain one or more samples, each corresponding to a point cloud frame of the point cloud media and containing attribute data for the corresponding point cloud frame. The geometry component track may contain one or more samples, each corresponding to a point cloud frame of the point cloud media and containing geometric data for the corresponding point cloud frame. The parameter component track includes a track data box, which includes a track reference type data box, which indicates that the parameter component track is associated with one or more attribute component tracks corresponding to the attribute parameter set in the parameter component track.

[0263] a. When the media file complies with the first coding standard (such as AVS), the parameter component track may include a component information data box, which includes at least one of the following fields: a component type field, a header type field, a number of attribute headers field, a reference attribute type flag field, a reference attribute identifier field, and a reference attribute type field. The component type field is used to indicate the component type in the parameter component track; the header type field is used to indicate the parameter set type included in the parameter component track; the number of attribute headers field is used to indicate the number of attribute parameter sets included in the sample in the parameter component track, wherein the order of the attribute parameter sets included in the sample is consistent with the order of the attribute parameter set identifiers or attribute parameter set types indicated in the component information data box; the reference attribute type field is used to indicate the type of attribute data to which the current attribute parameter set is effective; the reference attribute identifier field is used to indicate the identifier of the attribute data corresponding to the current attribute parameter set; the reference attribute type is used to indicate the type of attribute data corresponding to the current attribute parameter set; the current attribute parameter set refers to the attribute parameter set being used, and the current sample refers to the sample being encoded.

[0264] At this time, multi-track encapsulation of the attribute parameter set in the point cloud code stream into the parameter component track may include: if the component type in the parameter component track is a parameter set, setting the component type field to the target preset value; if the parameter set type contained in the parameter component track is an attribute parameter set, setting the header type field to the first set value; if the parameter set type contained in the parameter component track is a geometry parameter set, setting the header type field to the second set value; if the parameter set type contained in the parameter component track is a geometry parameter set and an attribute parameter set, setting the header type field to the third set value. If the current attribute parameter set is only valid for a specific type of attribute data in the current sample, setting the reference attribute type flag field to the first preset value (such as 1); if the current attribute parameter set is valid for a specific set of attribute data in the current sample, setting the reference attribute type flag field to the second preset value (such as 0).

[0265] b. When the media file complies with the second coding standard (such as MPEG), the parameter component track contains a component information data box and a configuration data box; the parameter set type corresponding to the setup unit contained in the configuration data box is the same as the parameter set type indicated by the component information data box; the setup unit (setupUnit) can contain a sequence parameter set (SPS), a geometry parameter set (i.e., GPS), an attribute parameter set (i.e., APS), or a data unit of point cloud segmentation information. The component information data box contains at least one of the following fields: a component type field, a parameter setting type quantity field, and a parameter setting type field. Among them, the component type field is used to indicate the type of component in the parameter component track; the parameter setting type quantity field is used to indicate the number of parameter sets contained in the parameter component track; and the parameter setting type field is used to indicate the type of parameter set contained in the parameter component track.

[0266] At this time, encapsulating the attribute parameter set in the point cloud code stream into the parameter component track may include: if the type of the component in the parameter component track is a parameter set, setting the component type field to the target setting value; if the type of the parameter set contained in the parameter component track is an attribute parameter set, setting the parameter setting type field to the first preset value (such as 0); if the type of the parameter set contained in the parameter component track is a geometric parameter set, setting the parameter setting type field to the second preset value (such as 1); if the type of the parameter set contained in the parameter component track is a sequence parameter set, setting the parameter setting type field to the third preset value (such as 2).

[0267] (3) The media file includes a geometry component track, and the attribute parameter set is encapsulated in the geometry component track.

[0268] When the point cloud code stream is encapsulated into multiple tracks in a multi-track manner, the attribute parameter set can be encapsulated into the geometry parameter component track. Specifically, the attribute parameter set and the geometry data in the point cloud code stream are encapsulated into the geometry component track, and the attribute data in the point cloud code stream is encapsulated into the attribute component track to form a media file of the point cloud media. At this time, the media file may include a geometry component track and an attribute component track. Among them, the attribute parameter set indication information is used to indicate that the attribute parameter set is encapsulated in the geometry component track. The geometry component track includes one or more samples, each sample corresponds to a point cloud frame, and each sample contains the parameter information required for decoding the attribute data of a corresponding point cloud frame.

[0269] In one embodiment, the geometry component track includes a component information data box, which includes a component type field, and the component type field is used to indicate the type of the component in the geometry component track; if the component type in the geometry component track is a set of geometry data and parameters, the component type field can be set to a first set value (such as 2).

[0270] (4) The media file includes one or more attribute component tracks, and the attribute parameter set is encapsulated in each attribute component track; wherein the attribute parameter set encapsulated in each attribute component track is obtained by copying the same attribute parameter set.

[0271] Specifically, the attribute data in the point cloud code stream can be encapsulated into one or more attribute component tracks in a multi-track manner, and the attribute parameter set can be copied, and then a copy of the attribute parameter set can be encapsulated in each attribute component track to form a media file of the point cloud media; wherein the attribute parameter set indication information is used to indicate that the attribute parameter set is encapsulated in each attribute component track respectively.

[0272] (5) The media file includes one or more attribute component tracks, the attribute parameter set includes an index, and the index of the attribute parameter set is encapsulated in each attribute component track.

[0273] Specifically, a multi-track approach is used to encapsulate the attribute data in the point cloud codestream into one or more attribute component tracks, and an index of the attribute parameter set in the point cloud codestream is encapsulated into each attribute component track to form a media file of the point cloud media; wherein, the attribute parameter set can be found through the index encapsulated in any attribute component track, or the data unit formed by reorganizing the attribute parameter set can be found. The data unit formed by reorganizing the attribute parameter set refers to the process of reorganizing the point cloud codestream by indexing samples of different types of attribute component tracks and corresponding attribute parameter sets into the same data unit.

[0274] In one embodiment, in a streaming transmission mode, a media file of a point cloud media is sent via transmission signaling. If a set of attribute parameters is independently encapsulated into a parameter component track, the transmission signaling includes stream indication information, and the stream indication information is used to indicate the data stream corresponding to the parameter component track. The transmission signaling may be a DASH signaling; the stream indication information may refer to a component descriptor, which is set in the data stream corresponding to the parameter component track; the component descriptor includes a composition type element and a parameter type element, and the composition type element is used to indicate the type of component in the parameter component track; if the type of the component in the parameter component track is a parameter set component, the composition type element in the component descriptor is set to the target character. The parameter type element is used to indicate the type of parameter set contained in the parameter component track.

[0275] In an embodiment of the present application, point cloud media is acquired, and the point cloud media is encoded to obtain a point cloud code stream; the point cloud code stream includes an attribute parameter set, and the attribute parameter set includes parameter information required for decoding attribute data in the point cloud code stream. The point cloud code stream is encapsulated to obtain a media file of the point cloud media; the media file also includes attribute parameter set indication information, and the attribute parameter set indication information is used to indicate the encapsulation location of the attribute parameter set; adding the attribute parameter set indication information to the media file can facilitate the media processing device to directly obtain the attribute parameter set according to the attribute parameter set indication information, thereby guiding the transmission, decoding and presentation of the point cloud media, and reducing certain encoding and decoding dependencies.

[0276] The following three complete examples illustrate the data processing method of point cloud media provided by this application in detail:

[0277] Example 1:

[0278] 1. The content production device can collect point cloud media and encode the attribute data in the point cloud media according to the attribute parameter set to obtain an attribute code stream; encode the geometric data in the point cloud media according to the geometric parameter set to obtain a geometric code stream, and the attribute code stream and the geometric code stream are combined to generate a point cloud code stream, which also includes the attribute parameter set; then, when the point cloud code stream is encapsulated into a file, a multi-track method is used to encapsulate the attribute parameter set in the point cloud code stream into the parameter component track, the attribute data in the point cloud code stream into the attribute component track, and the geometric data in the point cloud code stream into the geometry component track to form a media file of the point cloud media; at this time, the media file can contain attribute parameter set indication information, and the attribute parameter set indication information is used to indicate that the attribute parameter set is encapsulated in the parameter component track. Among them, the information of each track after encapsulation is as follows:

[0279] Track 1: geometry component track {GPCCComponentInfoBox: gpcc_type=2}, wherein the geometry component track contains a component information data box GPCCComponentInfoBox, which includes a component type field gpcc_type. gpcc_type=2 indicates that the component type in the geometry component track is geometry data.

[0280] Track2: Attribute component track - color {GPCCComponentInfoBox: gpcc_type=4, attr_type=0}; wherein, the attribute component track contains the component information data box GPCCComponentInfoBox, which includes the component type field gpcc_type and the attribute type field attr_type, gpcc_type=4 indicates that the component type in the geometric component track is attribute data; attr_type=0 indicates that the type of the attribute component is color.

[0281] Track3: Attribute component track - reflectivity {GPCCComponentInfoBox: gpcc_type=4, attr_type=1}; wherein, the attribute component track contains the component information data box GPCCComponentInfoBox, which includes the component type field gpcc_type and the attribute type field attr_type, gpcc_type=4 indicates that the component type in the attribute component track is attribute data; attr_type=1 indicates that the type of the attribute component is reflectivity.

[0282] Track4: parameter component track {GPCCComponentInfoBox: gpcc_type=5, parameter_set_type=0}; {TrackReferenceTypeBox: reference_type='appr', track_id=(2,3)}; wherein, the parameter component track contains the component information data box GPCCComponentInfoBox, which includes the component type field gpcc_type and the parameter setting type field parameter_set_type, and gpcc_type=5 indicates that the component type in the parameter component track is an attribute parameter set; the parameter component track also contains a track reference type data box TrackReferenceTypeBox, and the reference type reference_type of the track reference type data box is 'appr'; the track reference type data box is used to associate the parameter component track with the attribute component track corresponding to the attribute parameter set in the parameter component track, and track_id=(2,3) indicates that the attribute parameter set in the parameter component track (tranck4) is associated with track 2 and track 3.

[0283] 2. The content production device can transmit the media file to the media processing device using transmission signaling. The transmission signaling includes stream indication information, which is used to indicate the data stream corresponding to the parameter component track component@component_type='prms'. The component type element component_type='prms' is used to indicate that the component type in the parameter component track is a parameter set component.

[0284] Assume that Track1 to Track4 correspond to Representation1 to Representation4 respectively.

[0285] 3. The media processing device can request the corresponding media file based on the stream indication information contained in the transmission signaling. At this time, as long as the media processing device needs to request attribute data, it will definitely request the data stream corresponding to the parameter component track, that is, Representation4.

[0286] 4. After obtaining the data stream corresponding to the parameter component track, the media processing device can decapsulate and decode the media file. It can obtain the attribute parameter set from the parameter component track track4 according to the attribute parameter set indication information, and partially decode the related point cloud media according to the attribute parameter set, thereby maximizing the saving of decoding resources. For example, if the media processing file only needs to decode the color attribute, track4+track2 can be decoded. Specifically, the parameter component track track4 can be decoded first to obtain the attribute parameter set, and then the attribute component track corresponding to the color attribute data can be decoded according to the attribute parameter set to obtain the color attribute data in the point cloud media.

[0287] 5. The media processing device can render the color attribute data in the point cloud media to present the point cloud media.

[0288] Example 2:

[0289] 1. The content production device can collect point cloud media and encode the attribute data in the point cloud media according to the attribute parameter set to obtain an attribute code stream; and encode the geometric data in the point cloud media according to the geometric parameter set to obtain a geometric code stream. The attribute code stream and the geometric code stream are combined to generate a point cloud code stream, which also includes the attribute parameter set. Then, when encapsulating the point cloud code stream, a multi-track method is used to encapsulate the attribute parameter set and geometric data in the point cloud code stream into a geometric component track, and the attribute data in the point cloud code stream into an attribute component track to form a media file of the point cloud media. At this time, the media file can contain attribute parameter set indication information, which is used to indicate that the attribute parameter set is encapsulated into the geometric component track. The information of each track after encapsulation is as follows:

[0290] Track 1: Geometry component track {GPCCComponentInfoBox: gpcc_type=2}, wherein the geometry component track contains a component information data box GPCCComponentInfoBox, which includes a component type field gpcc_type. gpcc_type=2 indicates that the component type in the geometry component track is geometry data and parameter set, which includes an attribute parameter set.

[0291] Track2: Attribute component track - color {GPCCComponentInfoBox: gpcc_type=4, attr_type=0}; wherein, the attribute component track contains the component information data box GPCCComponentInfoBox, which includes the component type field gpcc_type and the attribute type field attr_type, gpcc_type=4 indicates that the component type in the geometric component track is attribute data; attr_type=0 indicates that the type of the attribute component is color.

[0292] Track3: Attribute component track - reflectivity {GPCCComponentInfoBox: gpcc_type=4, attr_type=1}; wherein, the attribute component track contains the component information data box GPCCComponentInfoBox, which includes the component type field gpcc_type and the attribute type field attr_type, gpcc_type=4 indicates that the component type in the attribute component track is attribute data; attr_type=1 indicates that the type of the attribute component is reflectivity.

[0293] 2. The content production device can transfer the media files to the media processing device.

[0294] 3. When decapsulating and decoding a media file, the media processing device can obtain an attribute parameter set from the geometry component track track1 according to the attribute parameter set indication information, and partially decode the related point cloud media according to the attribute parameter set, thereby maximizing the savings in decoding resources. For example, when the media processing device only needs to decode color attribute data, it decodes track1+track2. Specifically, the geometry component track track1 can be decoded first to obtain the attribute parameter set, and then the attribute component track truck2 corresponding to the color attribute data can be decoded according to the attribute parameter set to obtain the color attribute data in the point cloud media.

[0295] 5. The media processing device can render the color attribute data in the point cloud media to present the point cloud media.

[0296] Example 3:

[0297] 1. A content production device may capture point cloud media. The attribute data in the point cloud media is then encoded according to an attribute parameter set to obtain an attribute code stream. The geometric data in the point cloud media is then encoded according to a geometric parameter set to obtain a geometric code stream. The attribute code stream and the geometric code stream are combined to generate a point cloud code stream, which also includes an attribute parameter set. When encapsulating the point cloud code stream, a multi-track approach is used to encapsulate the color attribute data in the point cloud code stream into an attribute component track (truck2) corresponding to the color attribute data, and the reflectivity attribute data in the point cloud code stream into an attribute component track (truck3) corresponding to the reflectivity attribute data. Furthermore, the geometric data in the point cloud code stream is encapsulated into a geometry component track, and the attribute parameter set is respectively encapsulated into an attribute component track corresponding to the color attribute data and an attribute component track corresponding to the reflectivity attribute data, thereby forming a media file for the point cloud media. At this point, the media file may include attribute parameter set indication information, which is used to indicate that the attribute parameter set is to be encapsulated into each attribute component track. The information for each track is as follows:

[0298] Track 1: geometry component track {GPCCComponentInfoBox: gpcc_type=2}, wherein the geometry component track contains a component information data box GPCCComponentInfoBox, which includes a component type field gpcc_type. gpcc_type=2 indicates that the component type in the geometry component track is geometry data.

[0299] Track 2: Attribute Component Track - Color {GPCCComponentInfoBox: gpcc_type=4, attr_type=0}; The attribute component track contains the component information data box GPCCComponentInfoBox, which includes the component type field gpcc_type and the attribute type field attr_type. gpcc_type=4 indicates that the component type in this geometry component track is attribute data; attr_type=0 indicates that the attribute component type is color. This attribute component track contains a set of attribute parameters.

[0300] Track3: Attribute component track - reflectivity {GPCCComponentInfoBox: gpcc_type=4, attr_type=1}; wherein, the attribute component track contains the component information data box GPCCComponentInfoBox, which includes the component type field gpcc_type and the attribute type field attr_type, gpcc_type=4 indicates that the component type in the attribute component track is attribute data; attr_type=1 indicates that the type of the attribute component is reflectivity; the attribute component track contains an attribute parameter set.

[0301] 2. The content production device transfers the media file to the media processing device.

[0302] 3. The media processing device can partially decode the relevant point cloud media as needed. For example, if only color attribute data needs to be decoded, track 2 can be directly decoded. Specifically, the attribute component track corresponding to the color attribute data can be directly decoded based on the attribute parameter set indication information to obtain the color attribute data in the point cloud media.

[0303] 4. The media processing device can render the color attribute data in the point cloud media to present the point cloud media.

[0304] It should be understood that the above three embodiments are illustrative examples of the present application, and they can be flexibly selected and used or combined according to the packaging location of the attribute parameter set in accordance with actual conditions. The present application does not limit this.

[0305] In an embodiment of the present application, a media file of point cloud media is obtained, and the media file includes a point cloud code stream and attribute parameter set indication information of the point cloud media. The point cloud code stream includes an attribute parameter set, and the attribute parameter set includes parameter information required for decoding attribute data in the point cloud code stream; the attribute parameter set indication information is used to indicate the encapsulation position of the attribute parameter set; according to the attribute parameter set indication information, the encapsulation position of the attribute parameter set can be directly known, and the point cloud code stream can be decoded based on the attribute parameter set to present the point cloud media, which guides the transmission, decoding and presentation of the point cloud media, can reduce unnecessary encoding and decoding dependencies, and save transmission resources and computing resources.

[0306] See Figure 6 , Figure 6 It is a structural diagram of a point cloud media data processing device provided in an embodiment of the present application. The point cloud media data processing device can be set in the computer device provided in an embodiment of the present application. The computer device can be the media processing device mentioned in the above method embodiment. Figure 6The data processing device of the point cloud media shown can be a computer program (including program code) running on a computer device, and the data processing device of the point cloud media can be used to perform Figure 3 Some or all of the steps in the method embodiment shown. Figure 6 The data processing device of the point cloud media may include the following units:

[0307] An acquisition unit 601 is configured to acquire a media file of point cloud media. The media file includes a point cloud stream of the point cloud media. The point cloud stream includes an attribute parameter set. The attribute parameter set includes parameter information required for decoding attribute data in the point cloud stream. The media file also includes attribute parameter set indication information. The attribute parameter set indication information is used to indicate the encapsulation location of the attribute parameter set.

[0308] The processing unit 602 is configured to decode the point cloud code stream according to the attribute parameter set indication information to present the point cloud media.

[0309] In one embodiment, the media file includes a track, each track includes one or more samples, each sample corresponds to a point cloud frame in the point cloud media; each sample is divided into one or more subsamples, each subsample includes a type of point cloud data;

[0310] The media file also includes a subsample information data box, and the attribute parameter set is encapsulated in the subsample information data box.

[0311] In one embodiment, the attribute parameter set indication information is represented as a subsample information data box, and the attribute parameter set indication information is further used to indicate the relationship between the attribute parameter set and the attribute data in the point cloud code stream;

[0312] The subsample data box includes at least one of the following fields: a payload type field, a reference attribute information flag field, a reference attribute type flag field, a reference attribute identification field, and a reference attribute type field;

[0313] The payload type field is used to indicate the type of point cloud data contained in the subsample. If the payload type field is the first value, it indicates that the type of point cloud data contained in the subsample is an attribute parameter set. If the payload type field is the second value, it indicates that the type of point cloud data contained in the subsample is a geometry parameter set, which contains parameter information required for decoding geometry data.

[0314] The reference attribute information flag field is used to indicate the scope of validity of the current attribute parameter set for attribute data. If the reference attribute information flag field is a first preset value, it indicates that the current attribute parameter set is valid only for a specific group or a specific type of attribute data in the current sample. If the reference attribute information flag field is a second preset value, it indicates that the current attribute parameter set is valid for all attribute data in the current sample.

[0315] The reference attribute type flag field is used to indicate the type of attribute data to which the current attribute parameter set is effective. If the reference attribute type flag field is a first value, it indicates that the current attribute parameter set is only effective for a specific type of attribute data in the current sample. If the reference attribute type flag field is a second value, it indicates that the current attribute parameter set is only effective for a specific set of attribute data in the current sample.

[0316] The reference attribute identification field is used to indicate the identifier of the attribute data corresponding to the current attribute parameter set; the reference attribute type field is used to indicate the type of the attribute data corresponding to the current attribute parameter set;

[0317] The current attribute parameter set refers to the attribute parameter set being used, and the current sample refers to the sample being decoded.

[0318] In one embodiment, the point cloud code stream is encapsulated in a media file in a multi-track manner; the media file includes a parameter component track, and the attribute parameter set is independently encapsulated in the parameter component track;

[0319] The parameter component track contains one or more samples, each sample corresponds to a point cloud frame of the point cloud media, and each sample contains the parameter information required to decode the attribute data of the corresponding point cloud frame.

[0320] In one embodiment, the media file includes one or more parameter component tracks; when the media file includes only one parameter component track, the attribute parameter set in the point cloud code stream is encapsulated into the parameter component track; when the media file includes multiple parameter component tracks, the attribute parameter set in the point cloud code stream is encapsulated into a separate parameter component track or the attribute parameter set in the point cloud code stream is encapsulated into multiple parameter component tracks based on type.

[0321] In one embodiment, the parameter component track includes a track data box;

[0322] The track data box includes a track reference type data box, which is used to indicate that the parameter component track is associated with one or more attribute component tracks corresponding to the attribute parameter set in the parameter component track.

[0323] In one embodiment, the media file complies with a first encoding standard; the parameter component track includes a component information data box, which is set in a header sample entry of the parameter component track; the component information data box includes at least one of the following fields: a component type field, a header type field, a number of attribute headers field, a reference attribute type flag field, a reference attribute identifier field, and a reference attribute type field;

[0324] The component type field is used to indicate the component type in the parameter component track; when the component type field is set to the target preset value, it indicates that the component type in the parameter component track is a parameter set;

[0325] The header type field is used to indicate the parameter set type contained in the parameter component track; if the header type field is the first setting value, it indicates that the parameter set type contained in the parameter component track is an attribute parameter set; if the header type field is the second setting value, it indicates that the parameter set type contained in the parameter component track is a geometry parameter set; if the header type field is the third setting value, it indicates that the parameter set type contained in the parameter component track is a geometry parameter set and an attribute parameter set;

[0326] The attribute header quantity field is used to indicate the number of attribute parameter sets contained in the sample in the parameter component track; wherein the order of the attribute parameter sets contained in the sample is consistent with the order of the attribute parameter set identifiers or attribute parameter set types indicated in the component information data box;

[0327] The reference attribute type field is used to indicate the type of attribute data to which the current attribute parameter set is effective. If the reference attribute type flag field is the first preset value, it indicates that the current attribute parameter set is only effective for a specific type of attribute data in the current sample. If the reference attribute type flag field is the second preset value, it indicates that the current attribute parameter set is effective for a specific set of attribute data in the current sample.

[0328] The reference attribute identifier field is used to indicate the identifier of the attribute data corresponding to the current attribute parameter set;

[0329] The reference attribute type field is used to indicate the type of attribute data corresponding to the current attribute parameter set;

[0330] The current attribute parameter set refers to the attribute parameter set being used, and the current sample refers to the sample being decoded.

[0331] In one embodiment, the media file complies with the second encoding standard; the parameter component track includes a component information data box and a configuration data box; the parameter set type corresponding to the setting unit included in the configuration data box is the same as the parameter set type indicated by the component information data box;

[0332] The component information data box includes at least one of the following fields: a component type field, a parameter setting type quantity field, and a parameter setting type field;

[0333] The component type field is used to indicate the type of the component in the parameter component track; if the component type field is a target setting value, it indicates that the type of the component in the parameter component track is a parameter set;

[0334] The parameter set type quantity field is used to indicate the number of parameter sets contained in the parameter component track;

[0335] The parameter setting type field is used to indicate the type of parameter set contained in the parameter component track; if the parameter setting type field is the first preset value, it means that the type of parameter set contained in the parameter component track is an attribute parameter set; if the parameter setting type field is the second preset value, it means that the type of parameter set contained in the parameter component track is a geometric parameter set; if the parameter setting type field is the third preset value, it means that the type of parameter set contained in the parameter component track is a sequence parameter set.

[0336] In one embodiment, the media file includes a geometry component track, and the attribute parameter set is encapsulated in the geometry component track.

[0337] In one embodiment, the geometry component track includes a component information data box, and the attribute parameter set indication information is represented as the component information data box;

[0338] The component information data box includes a component type field, which is used to indicate the type of component in the geometry component track; if the component type field is the first set value, it indicates that the type of component in the geometry component track is geometry data and parameter set, and the parameter set includes an attribute parameter set and a geometry parameter set.

[0339] In one embodiment, the media file includes one or more attribute component tracks, and the attribute parameter set is encapsulated in each attribute component track;

[0340] The attribute parameter set encapsulated in each attribute component track is obtained by copying the same attribute parameter set.

[0341] In one embodiment, the media file includes one or more attribute component tracks, the attribute parameter set includes an index, and the index of the attribute parameter set is encapsulated in each attribute component track;

[0342] The attribute parameter set or the data unit formed by reorganizing the attribute parameter set can be found through the index encapsulated in any attribute component track.

[0343] In one embodiment, the point cloud media is transmitted in a streaming manner. When acquiring the media file of the point cloud media, the acquisition unit 601 may be specifically configured to:

[0344] Obtaining transmission signaling of the point cloud media, which includes stream indication information. The stream indication information is used to indicate the data stream corresponding to the parameter component track;

[0345] Get the media file of the point cloud media according to the transmission signaling.

[0346] In one embodiment, the transmission signaling is dynamic adaptive stream signaling, and the stream indication information refers to a component descriptor, which is set in the data stream corresponding to the parameter component track; the component descriptor includes a component type element and a parameter type element, the component type element is used to indicate the type of the component in the parameter component track; the parameter type element is used to indicate the type of the parameter set included in the parameter component track;

[0347] If the component type element is a target character, the type of the component in the parameter component track is a parameter collection component.

[0348] In one embodiment, when the processing unit 602 decodes the point cloud stream according to the attribute parameter set indication information to present the point cloud media, it can be specifically configured to:

[0349] Obtaining the attribute parameter set according to the encapsulation position indicated by the attribute parameter set indication information;

[0350] The attribute data in the point cloud code stream is decoded using the parameter information in the attribute parameter set to present the point cloud media.

[0351] In an embodiment of the present application, a media file of point cloud media is obtained, and the media file includes a point cloud code stream and attribute parameter set indication information of the point cloud media. The point cloud code stream includes an attribute parameter set, and the attribute parameter set includes parameter information required for decoding attribute data in the point cloud code stream; the attribute parameter set indication information is used to indicate the encapsulation position of the attribute parameter set; according to the attribute parameter set indication information, the encapsulation position of the attribute parameter set can be directly known, and the point cloud code stream can be decoded based on the attribute parameter set to present the point cloud media, which guides the transmission, decoding and presentation of the point cloud media, can reduce unnecessary encoding and decoding dependencies, and save transmission resources and computing resources.

[0352] See Figure 7 , Figure 7 It is a structural diagram of a point cloud media data processing device provided in an embodiment of the present application. The point cloud media data processing device can be set in the computer device provided in the embodiment of the present application. The computer device can be the content production device mentioned in the above method embodiment. Figure 7The data processing device of the point cloud media shown can be a computer program (including program code) running on a computer device, and the data processing device of the point cloud media can be used to perform Figure 5 Some or all of the steps in the method embodiment shown. Figure 7 The data processing device of the point cloud media may include the following units:

[0353] An acquisition unit 701 is used to acquire point cloud media;

[0354] The processing unit 702 is configured to encode the point cloud media to obtain a point cloud stream; the point cloud stream includes an attribute parameter set, and the attribute parameter set includes parameter information required for decoding attribute data in the point cloud stream;

[0355] The processing unit 702 is further configured to encapsulate the point cloud code stream to obtain a media file of the point cloud media; the media file further includes attribute parameter set indication information, and the attribute parameter set indication information is used to indicate the encapsulation position of the attribute parameter set.

[0356] In one embodiment, the media file includes a track, the point cloud code stream is encapsulated in the track, the track includes one or more samples, and one sample corresponds to one point cloud frame in the point cloud media; one sample is divided into one or more sub-samples, and each sub-sample includes one type of point cloud data;

[0357] The media file also includes a subsample information data box, and the attribute parameter set is encapsulated in the subsample information data box.

[0358] In one embodiment, the point cloud stream is encapsulated into a media file in a multi-track manner; the media file includes a parameter component track, and the attribute parameter set is independently encapsulated in the parameter component track;

[0359] The parameter component track contains one or more samples, each sample corresponds to a point cloud frame of the point cloud media, and each sample contains the parameter information required to decode the attribute data of the corresponding point cloud frame.

[0360] In one embodiment, the media file includes a geometry component track, and the attribute parameter set is encapsulated in the geometry component track.

[0361] In one embodiment, the media file includes one or more attribute component tracks, and the attribute parameter set is encapsulated in each attribute component track;

[0362] The attribute parameter set encapsulated in each attribute component track is obtained by copying the same attribute parameter set.

[0363] In one embodiment, the media file includes one or more attribute component tracks, the attribute parameter set includes an index, and the index of the attribute parameter set is encapsulated in each attribute component track;

[0364] The attribute parameter set or the data unit formed by reorganizing the attribute parameter set can be found through the index encapsulated in any attribute component track.

[0365] In one embodiment, the processing unit 702 is further configured to:

[0366] The media file of the point cloud media is sent through transmission signaling, which includes stream indication information. The stream indication information is used to indicate the data stream corresponding to the parameter component track.

[0367] In an embodiment of the present application, point cloud media is acquired, and the point cloud media is encoded to obtain a point cloud code stream; the point cloud code stream includes an attribute parameter set, and the attribute parameter set includes parameter information required for decoding attribute data in the point cloud code stream. The point cloud code stream is encapsulated to obtain a media file of the point cloud media; the media file also includes attribute parameter set indication information, and the attribute parameter set indication information is used to indicate the encapsulation location of the attribute parameter set; adding the attribute parameter set indication information to the media file can facilitate the media processing device to directly obtain the attribute parameter set according to the attribute parameter set indication information, thereby guiding the transmission, decoding and presentation of the point cloud media, and reducing certain encoding and decoding dependencies.

[0368] Furthermore, the present invention also provides a schematic diagram of the structure of a computer device. Figure 8 The computer device may be the aforementioned media processing device or content production device. The computer device may include a processor 801, an input device 802, an output device 803, and a memory 804. The processor 801, input device 802, output device 803, and memory 804 are connected via a bus. The memory 804 is configured to store a computer program, which includes program instructions. The processor 801 is configured to execute the program instructions stored in the memory 804.

[0369] When the computer device is the above-mentioned media processing device, in the embodiment of the present application, the processor 801 performs the following operations by running the executable program code in the memory 804:

[0370] Obtaining a media file of the point cloud media, the media file including a point cloud code stream of the point cloud media, the point cloud code stream including an attribute parameter set, the attribute parameter set including parameter information required for decoding attribute data in the point cloud code stream; the media file also includes attribute parameter set indication information, the attribute parameter set indication information being used to indicate a packaging location of the attribute parameter set;

[0371] According to the attribute parameter set indication information, the point cloud code stream is decoded to present the point cloud media.

[0372] In one embodiment, the media file includes a track, each track includes one or more samples, each sample corresponds to a point cloud frame in the point cloud media; each sample is divided into one or more subsamples, each subsample includes a type of point cloud data;

[0373] The media file also includes a subsample information data box, and the attribute parameter set is encapsulated in the subsample information data box.

[0374] In one embodiment, the attribute parameter set indication information is represented as a subsample information data box, and the attribute parameter set indication information is further used to indicate the relationship between the attribute parameter set and the attribute data in the point cloud code stream;

[0375] The subsample data box includes at least one of the following fields: a payload type field, a reference attribute information flag field, a reference attribute type flag field, a reference attribute identification field, and a reference attribute type field;

[0376] The payload type field is used to indicate the type of point cloud data contained in the subsample. If the payload type field is the first value, it indicates that the type of point cloud data contained in the subsample is an attribute parameter set. If the payload type field is the second value, it indicates that the type of point cloud data contained in the subsample is a geometry parameter set, which contains parameter information required for decoding geometry data.

[0377] The reference attribute information flag field is used to indicate the scope of validity of the current attribute parameter set for attribute data. If the reference attribute information flag field is a first preset value, it indicates that the current attribute parameter set is valid only for a specific group or a specific type of attribute data in the current sample. If the reference attribute information flag field is a second preset value, it indicates that the current attribute parameter set is valid for all attribute data in the current sample.

[0378] The reference attribute type flag field is used to indicate the type of attribute data to which the current attribute parameter set is effective. If the reference attribute type flag field is a first value, it indicates that the current attribute parameter set is only effective for a specific type of attribute data in the current sample. If the reference attribute type flag field is a second value, it indicates that the current attribute parameter set is only effective for a specific set of attribute data in the current sample.

[0379] The reference attribute identification field is used to indicate the identifier of the attribute data corresponding to the current attribute parameter set; the reference attribute type field is used to indicate the type of the attribute data corresponding to the current attribute parameter set;

[0380] The current attribute parameter set refers to the attribute parameter set being used, and the current sample refers to the sample being decoded.

[0381] In one embodiment, the point cloud code stream is encapsulated in a media file in a multi-track manner; the media file includes a parameter component track, and the attribute parameter set is independently encapsulated in the parameter component track;

[0382] The parameter component track contains one or more samples, each sample corresponds to a point cloud frame of the point cloud media, and each sample contains the parameter information required to decode the attribute data of the corresponding point cloud frame.

[0383] In one embodiment, the media file includes one or more parameter component tracks; when the media file includes only one parameter component track, the attribute parameter set in the point cloud code stream is encapsulated into the parameter component track; when the media file includes multiple parameter component tracks, the attribute parameter set in the point cloud code stream is encapsulated into a separate parameter component track or the attribute parameter set in the point cloud code stream is encapsulated into multiple parameter component tracks based on type.

[0384] In one embodiment, the parameter component track includes a track data box;

[0385] The track data box includes a track reference type data box, which is used to indicate that the parameter component track is associated with one or more attribute component tracks corresponding to the attribute parameter set in the parameter component track.

[0386] In one embodiment, the media file complies with a first encoding standard; the parameter component track includes a component information data box, which is set in a header sample entry of the parameter component track; the component information data box includes at least one of the following fields: a component type field, a header type field, a number of attribute headers field, a reference attribute type flag field, a reference attribute identifier field, and a reference attribute type field;

[0387] The component type field is used to indicate the component type in the parameter component track; when the component type field is set to the target preset value, it indicates that the component type in the parameter component track is a parameter set;

[0388] The header type field is used to indicate the parameter set type contained in the parameter component track; if the header type field is the first setting value, it indicates that the parameter set type contained in the parameter component track is an attribute parameter set; if the header type field is the second setting value, it indicates that the parameter set type contained in the parameter component track is a geometry parameter set; if the header type field is the third setting value, it indicates that the parameter set type contained in the parameter component track is a geometry parameter set and an attribute parameter set;

[0389] The attribute header quantity field is used to indicate the number of attribute parameter sets contained in the sample in the parameter component track; wherein the order of the attribute parameter sets contained in the sample is consistent with the order of the attribute parameter set identifiers or attribute parameter set types indicated in the component information data box;

[0390] The reference attribute type field is used to indicate the type of attribute data to which the current attribute parameter set is effective. If the reference attribute type flag field is the first preset value, it indicates that the current attribute parameter set is only effective for a specific type of attribute data in the current sample. If the reference attribute type flag field is the second preset value, it indicates that the current attribute parameter set is effective for a specific set of attribute data in the current sample.

[0391] The reference attribute identifier field is used to indicate the identifier of the attribute data corresponding to the current attribute parameter set;

[0392] The reference attribute type field is used to indicate the type of attribute data corresponding to the current attribute parameter set;

[0393] The current attribute parameter set refers to the attribute parameter set being used, and the current sample refers to the sample being decoded.

[0394] In one embodiment, the media file complies with the second encoding standard; the parameter component track includes a component information data box and a configuration data box; the parameter set type corresponding to the setting unit included in the configuration data box is the same as the parameter set type indicated by the component information data box;

[0395] The component information data box includes at least one of the following fields: a component type field, a parameter setting type quantity field, and a parameter setting type field;

[0396] The component type field is used to indicate the type of the component in the parameter component track; if the component type field is a target setting value, it indicates that the type of the component in the parameter component track is a parameter set;

[0397] The parameter set type quantity field is used to indicate the number of parameter sets contained in the parameter component track;

[0398] The parameter setting type field is used to indicate the type of parameter set contained in the parameter component track; if the parameter setting type field is the first preset value, it means that the type of parameter set contained in the parameter component track is an attribute parameter set; if the parameter setting type field is the second preset value, it means that the type of parameter set contained in the parameter component track is a geometric parameter set; if the parameter setting type field is the third preset value, it means that the type of parameter set contained in the parameter component track is a sequence parameter set.

[0399] In one embodiment, the media file includes a geometry component track, and the attribute parameter set is encapsulated in the geometry component track.

[0400] In one embodiment, the geometry component track includes a component information data box, and the attribute parameter set indication information is represented as the component information data box;

[0401] The component information data box includes a component type field, which is used to indicate the type of component in the geometry component track; if the component type field is the first set value, it indicates that the type of component in the geometry component track is geometry data and parameter set, and the parameter set includes an attribute parameter set and a geometry parameter set.

[0402] In one embodiment, the media file includes one or more attribute component tracks, and the attribute parameter set is encapsulated in each attribute component track;

[0403] The attribute parameter set encapsulated in each attribute component track is obtained by copying the same attribute parameter set.

[0404] In one embodiment, the media file includes one or more attribute component tracks, the attribute parameter set includes an index, and the index of the attribute parameter set is encapsulated in each attribute component track;

[0405] The attribute parameter set or the data unit formed by reorganizing the attribute parameter set can be found through the index encapsulated in any attribute component track.

[0406] In one embodiment, the point cloud media is transmitted in a streaming manner. When the processor 801 obtains the media file of the point cloud media, it can be specifically configured to:

[0407] Obtaining transmission signaling of the point cloud media, which includes stream indication information. The stream indication information is used to indicate the data stream corresponding to the parameter component track;

[0408] Get the media file of the point cloud media according to the transmission signaling.

[0409] In one embodiment, the transmission signaling is dynamic adaptive stream signaling, and the stream indication information refers to a component descriptor, which is set in the data stream corresponding to the parameter component track; the component descriptor includes a component type element and a parameter type element, the component type element is used to indicate the type of the component in the parameter component track; the parameter type element is used to indicate the type of the parameter set included in the parameter component track;

[0410] If the component type element is a target character, the type of the component in the parameter component track is a parameter collection component.

[0411] In one embodiment, when the processor 801 decodes the point cloud stream according to the attribute parameter set indication information to present the point cloud media, it can be specifically configured to:

[0412] Obtaining the attribute parameter set according to the encapsulation position indicated by the attribute parameter set indication information;

[0413] The attribute data in the point cloud code stream is decoded using the parameter information in the attribute parameter set to present the point cloud media.

[0414] In an embodiment of the present application, a media file of point cloud media is obtained, and the media file includes a point cloud code stream and attribute parameter set indication information of the point cloud media. The point cloud code stream includes an attribute parameter set, and the attribute parameter set includes parameter information required for decoding attribute data in the point cloud code stream; the attribute parameter set indication information is used to indicate the encapsulation position of the attribute parameter set; according to the attribute parameter set indication information, the encapsulation position of the attribute parameter set can be directly known, and the point cloud code stream can be decoded based on the attribute parameter set to present the point cloud media, which guides the transmission, decoding and presentation of the point cloud media, can reduce unnecessary encoding and decoding dependencies, and save transmission resources and computing resources.

[0415] Optionally, when the computer device is the above-mentioned content production device, in the embodiment of the present application, the processor 801 performs the following operations by running the executable program code in the memory 804:

[0416] Get point cloud media;

[0417] Encoding the point cloud media to obtain a point cloud code stream; the point cloud code stream includes an attribute parameter set, and the attribute parameter set includes parameter information required for decoding the attribute data in the point cloud code stream;

[0418] The point cloud code stream is encapsulated to obtain a media file of the point cloud media; the media file also includes attribute parameter set indication information, and the attribute parameter set indication information is used to indicate the encapsulation position of the attribute parameter set.

[0419] In one embodiment, the media file includes a track, the point cloud code stream is encapsulated in the track, the track includes one or more samples, and one sample corresponds to one point cloud frame in the point cloud media; one sample is divided into one or more sub-samples, and each sub-sample includes one type of point cloud data;

[0420] The media file also includes a subsample information data box, and the attribute parameter set is encapsulated in the subsample information data box.

[0421] In one embodiment, the point cloud stream is encapsulated into a media file in a multi-track manner; the media file includes a parameter component track, and the attribute parameter set is independently encapsulated in the parameter component track;

[0422] The parameter component track contains one or more samples, each sample corresponds to a point cloud frame of the point cloud media, and each sample contains the parameter information required to decode the attribute data of the corresponding point cloud frame.

[0423] In one embodiment, the media file includes a geometry component track, and the attribute parameter set is encapsulated in the geometry component track.

[0424] In one embodiment, the media file includes one or more attribute component tracks, and the attribute parameter set is encapsulated in each attribute component track;

[0425] The attribute parameter set encapsulated in each attribute component track is obtained by copying the same attribute parameter set.

[0426] In one embodiment, the media file includes one or more attribute component tracks, the attribute parameter set includes an index, and the index of the attribute parameter set is encapsulated in each attribute component track;

[0427] The attribute parameter set or the data unit formed by reorganizing the attribute parameter set can be found through the index encapsulated in any attribute component track.

[0428] In one embodiment, the processor 801 is further configured to:

[0429] The media file of the point cloud media is sent through transmission signaling, which includes stream indication information. The stream indication information is used to indicate the data stream corresponding to the parameter component track.

[0430] In an embodiment of the present application, point cloud media is acquired, and the point cloud media is encoded to obtain a point cloud code stream; the point cloud code stream includes an attribute parameter set, and the attribute parameter set includes parameter information required for decoding attribute data in the point cloud code stream. The point cloud code stream is encapsulated to obtain a media file of the point cloud media; the media file also includes attribute parameter set indication information, and the attribute parameter set indication information is used to indicate the encapsulation location of the attribute parameter set; adding the attribute parameter set indication information to the media file can facilitate the media processing device to directly obtain the attribute parameter set according to the attribute parameter set indication information, thereby guiding the transmission, decoding and presentation of the point cloud media, and reducing certain encoding and decoding dependencies.

[0431] In addition, it should be noted that the present invention also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program, and the computer program includes program instructions. When the processor executes the above program instructions, it can execute the above Figure 3 and Figure 5The method in the corresponding embodiment will therefore not be described in detail here. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application. As an example, the program instructions can be deployed on a computer device, or executed on multiple computer devices located in one location, or executed on multiple computer devices distributed in multiple locations and interconnected by a communication network.

[0432] According to one aspect of the present application, a computer program product is provided, the computer program product comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, so that the computer device can perform the above-mentioned Figure 3 and Figure 5 The method in the corresponding embodiment will therefore not be described in detail here.

[0433] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0434] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Ordinary technicians in this field can understand that implementing all or part of the processes of the above embodiment and making equivalent changes in accordance with the claims of this application still fall within the scope of the invention.

Claims

1. A method for processing point cloud media data, characterized in that: include: Acquire a media file of point cloud media, the media file including a point cloud codestream of the point cloud media, the point cloud codestream including an attribute parameter set, the attribute parameter set including parameter information required for decoding attribute data in the point cloud codestream; the media file also including attribute parameter set indication information, the attribute parameter set indication information being used to indicate a packaging location of the attribute parameter set; when the point cloud codestream is packaged into the same track of the media file using a single track, the attribute parameter set indication information is represented as a subsample information data box; The subsample information data box includes at least one of the following fields: a payload type field, a reference attribute information flag field, a reference attribute type flag field, a reference attribute identification field, and a reference attribute type field; Decoding the point cloud stream according to the attribute parameter set indication information to present point cloud media; The payload type field is used to indicate the type of point cloud data contained in the subsample; if the payload type field is a first value, it indicates that the type of point cloud data contained in the subsample is an attribute parameter set; if the payload type field is a second value, it indicates that the type of point cloud data contained in the subsample is a geometry parameter set, and the geometry parameter set includes parameter information required for decoding geometry data; The reference attribute information flag field is used to indicate the scope of validity of the current attribute parameter set for attribute data; if the reference attribute information flag field is a first preset value, it indicates that the current attribute parameter set is valid only for a specific group or a specific type of attribute data in the current sample; if the reference attribute information flag field is a second preset value, it indicates that the current attribute parameter set is valid for all attribute data in the current sample; The reference attribute type flag field is used to indicate the type of attribute data to which the current attribute parameter set is effective; if the reference attribute type flag field is a first value, it indicates that the current attribute parameter set is only effective for a specific type of attribute data in the current sample; if the reference attribute type flag field is a second value, it indicates that the current attribute parameter set is only effective for a specific set of attribute data in the current sample; The reference attribute identification field is used to indicate an identifier of the attribute data corresponding to the current attribute parameter set; the reference attribute type field is used to indicate a type of the attribute data corresponding to the current attribute parameter set.

2. The method according to claim 1, wherein When the point cloud code stream is encapsulated into the same track in the media file in a single-track manner, the track contains one or more samples, one sample corresponds to a point cloud frame in the point cloud media; one sample is divided into one or more sub-samples, each sub-sample contains one type of point cloud data.

3. The method according to claim 2, wherein If the attribute parameter set indication information is represented by the subsample information data box, the attribute parameter set indication information is further used to indicate the relationship between the attribute parameter set and the attribute data in the point cloud code stream; The current attribute parameter set refers to the attribute parameter set being used, and the current sample refers to the sample being decoded.

4. The method according to claim 1, wherein When the point cloud code stream is encapsulated in multiple tracks of the media file in a multi-track manner; if the media file includes a parameter component track, and the attribute parameter set is independently encapsulated in the parameter component track, the attribute parameter set indication information is used to indicate that the encapsulation position of the attribute parameter set is the parameter component track; The parameter component track includes one or more samples, each sample corresponds to a point cloud frame of the point cloud media, and each sample includes parameter information required for decoding attribute data of the corresponding point cloud frame.

5. The method according to claim 4, wherein The media file includes one or more parameter component tracks; when the media file includes only one parameter component track, the attribute parameter set in the point cloud stream is encapsulated into the parameter component track; When the media file includes multiple parameter component tracks, the attribute parameter set in the point cloud codestream is encapsulated into a separate parameter component track or the attribute parameter set in the point cloud codestream is encapsulated into multiple parameter component tracks based on type.

6. The method according to claim 4, wherein The parameter component track includes a track data box; The track data box includes a track reference type data box, and the track reference type data box is used to indicate that the parameter component track is associated with one or more attribute component tracks corresponding to the attribute parameter set in the parameter component track.

7. The method according to claim 4, wherein The media file complies with a first encoding standard; the parameter component track includes a component information data box, the component information data box is set in a header sample entry of the parameter component track; the component information data box includes at least one of the following fields: a component type field, a header type field, a number of attribute headers field, a reference attribute type flag field, a reference attribute identifier field, and a reference attribute type field; The component type field is used to indicate the component type in the parameter component track; When the component type field is set to a target preset value, it indicates that the component type in the parameter component track is a parameter set; The header type field is used to indicate the parameter set type contained in the parameter component track; if the header type field is a first setting value, it indicates that the parameter set type contained in the parameter component track is an attribute parameter set; if the header type field is a second setting value, it indicates that the parameter set type contained in the parameter component track is a geometry parameter set; if the header type field is a third setting value, it indicates that the parameter set type contained in the parameter component track is a geometry parameter set and an attribute parameter set; The attribute header quantity field is used to indicate the number of attribute parameter sets contained in the sample in the parameter component track; wherein the order of the attribute parameter sets contained in the sample is consistent with the order of the attribute parameter set identifiers or attribute parameter set types indicated in the component information data box; The reference attribute type field is used to indicate the type of attribute data to which the current attribute parameter set is effective; if the reference attribute type flag field is a first preset value, it indicates that the current attribute parameter set is only effective for a specific type of attribute data in the current sample; if the reference attribute type flag field is a second preset value, it indicates that the current attribute parameter set is effective for a specific set of attribute data in the current sample; The reference attribute identification field is used to indicate an identifier of the attribute data corresponding to the current attribute parameter set; The reference attribute type field is used to indicate the type of attribute data corresponding to the current attribute parameter set; The current attribute parameter set refers to the attribute parameter set being used, and the current sample refers to the sample being decoded.

8. The method according to claim 4, wherein The media file complies with the second encoding standard; the parameter component track includes a component information data box and a configuration data box; the parameter set type corresponding to the setting unit included in the configuration data box is the same as the parameter set type indicated by the component information data box; The component information data box includes at least one of the following fields: a component type field, a parameter setting type quantity field, and a parameter setting type field; The component type field is used to indicate the type of the component in the parameter component track; if the component type field is a target setting value, it indicates that the type of the component in the parameter component track is a parameter set; The parameter setting type quantity field is used to indicate the number of parameter sets contained in the parameter component track; The parameter setting type field is used to indicate the type of parameter set contained in the parameter component track; if the parameter setting type field is a first preset value, it indicates that the type of parameter set contained in the parameter component track is an attribute parameter set; if the parameter setting type field is a second preset value, it indicates that the type of parameter set contained in the parameter component track is a geometric parameter set; if the parameter setting type field is a third preset value, it indicates that the type of parameter set contained in the parameter component track is a sequence parameter set.

9. The method according to claim 1, wherein When the point cloud code stream is encapsulated in multiple tracks in the media file in a multi-track manner, if the media file includes a geometry component track and the attribute parameter set is encapsulated in the geometry component track, then the attribute parameter set indication information is used to indicate that the encapsulation position of the attribute parameter set is the geometry component track.

10. The method according to claim 9, wherein The geometric component track includes a component information data box, and the attribute parameter set indication information is represented as the component information data box; The component information data box includes a component type field, which is used to indicate the type of the component in the geometric component track; if the component type field is a first set value, it indicates that the type of the component in the geometric component track is a geometric data and parameter set, and the parameter set includes an attribute parameter set and a geometric parameter set.

11. The method according to claim 1, wherein When the point cloud code stream is encapsulated in multiple tracks of the media file in a multi-track manner, if the media file includes one or more attribute component tracks, and the attribute parameter set is encapsulated in each attribute component track, then the attribute parameter set encapsulated in each attribute component track is obtained by copying the same attribute parameter set.

12. The method according to claim 1, wherein When the point cloud code stream is encapsulated in multiple tracks in the media file in a multi-track manner, if the media file includes one or more attribute component tracks, and the attribute parameter set includes an index, and the index of the attribute parameter set is encapsulated in each attribute component track, then the attribute parameter set indication information is used to indicate that the attribute parameter set is encapsulated in each attribute component track respectively, and indicates that the attribute parameter set can be found through the index encapsulated in any attribute component track, or that the data unit formed by the reorganization of the attribute parameter set can be found.

13. The method according to any one of claims 4 to 7, wherein: The point cloud media is transmitted in a streaming manner; the media file of the point cloud media is obtained, including: Acquire transmission signaling of the point cloud media, where the transmission signaling includes stream indication information, where the stream indication information is used to indicate a data stream corresponding to the parameter component track; Acquire a media file of the point cloud media according to the transmission signaling.

14. The method according to claim 13, wherein The transmission signaling is dynamic adaptive stream signaling, and the stream indication information refers to a component descriptor, which is set in the data stream corresponding to the parameter component track; the component descriptor includes a component type element and a parameter type element, the component type element is used to indicate the type of the component in the parameter component track; the parameter type element is used to indicate the type of the parameter set contained in the parameter component track; If the component type element is a target character, the type of the component in the parameter component track is a parameter set component.

15. The method according to claim 1, wherein The decoding of the point cloud code stream according to the attribute parameter set indication information to present point cloud media includes: Acquire the attribute parameter set according to the encapsulation position indicated by the attribute parameter set indication information; Attribute data in the point cloud code stream is decoded using parameter information in the attribute parameter set to present the point cloud media.

16. A method for processing point cloud media data, characterized in that: include: Get point cloud media; Encoding the point cloud media to obtain a point cloud code stream; The point cloud code stream includes an attribute parameter set, and the attribute parameter set includes parameter information required for decoding attribute data in the point cloud code stream; Encapsulating the point cloud code stream to obtain a media file of the point cloud media; the media file further includes attribute parameter set indication information, the attribute parameter set indication information being used to indicate an encapsulation location of the attribute parameter set; when the point cloud code stream is encapsulated into the same track of the media file in a single-track manner, the attribute parameter set indication information is represented as a subsample information data box; The subsample information data box includes at least one of the following fields: a payload type field, a reference attribute information flag field, a reference attribute type flag field, a reference attribute identification field, and a reference attribute type field; The payload type field is used to indicate the type of point cloud data contained in the subsample; if the payload type field is a first value, it indicates that the type of point cloud data contained in the subsample is an attribute parameter set; if the payload type field is a second value, it indicates that the type of point cloud data contained in the subsample is a geometry parameter set, and the geometry parameter set includes parameter information required for decoding geometry data; The reference attribute information flag field is used to indicate the scope of validity of the current attribute parameter set for attribute data; if the reference attribute information flag field is a first preset value, it indicates that the current attribute parameter set is valid only for a specific group or a specific type of attribute data in the current sample; if the reference attribute information flag field is a second preset value, it indicates that the current attribute parameter set is valid for all attribute data in the current sample; The reference attribute type flag field is used to indicate the type of attribute data to which the current attribute parameter set is effective; if the reference attribute type flag field is a first value, it indicates that the current attribute parameter set is only effective for a specific type of attribute data in the current sample; if the reference attribute type flag field is a second value, it indicates that the current attribute parameter set is only effective for a specific set of attribute data in the current sample; The reference attribute identification field is used to indicate an identifier of the attribute data corresponding to the current attribute parameter set; the reference attribute type field is used to indicate a type of the attribute data corresponding to the current attribute parameter set.

17. A point cloud media data processing device, characterized in that: include: an acquisition unit, configured to acquire a media file of point cloud media, wherein the media file includes a point cloud code stream of the point cloud media, the point cloud code stream includes an attribute parameter set, and the attribute parameter set includes parameter information required for decoding attribute data in the point cloud code stream; the media file also includes attribute parameter set indication information, and the attribute parameter set indication information is used to indicate the encapsulation location of the attribute parameter set; when the point cloud code stream is encapsulated into the same track of the media file using a single track, the attribute parameter set indication information is represented as a subsample information data box; The subsample information data box includes at least one of the following fields: a payload type field, a reference attribute information flag field, a reference attribute type flag field, a reference attribute identification field, and a reference attribute type field; a processing unit, configured to decode the point cloud code stream according to the attribute parameter set indication information to present point cloud media; The payload type field is used to indicate the type of point cloud data contained in the subsample; if the payload type field is a first value, it indicates that the type of point cloud data contained in the subsample is an attribute parameter set; if the payload type field is a second value, it indicates that the type of point cloud data contained in the subsample is a geometry parameter set, and the geometry parameter set includes parameter information required for decoding geometry data; The reference attribute information flag field is used to indicate the scope of validity of the current attribute parameter set for attribute data; if the reference attribute information flag field is a first preset value, it indicates that the current attribute parameter set is valid only for a specific group or a specific type of attribute data in the current sample; if the reference attribute information flag field is a second preset value, it indicates that the current attribute parameter set is valid for all attribute data in the current sample; The reference attribute type flag field is used to indicate the type of attribute data to which the current attribute parameter set is effective; if the reference attribute type flag field is a first value, it indicates that the current attribute parameter set is only effective for a specific type of attribute data in the current sample; if the reference attribute type flag field is a second value, it indicates that the current attribute parameter set is only effective for a specific set of attribute data in the current sample; The reference attribute identification field is used to indicate an identifier of the attribute data corresponding to the current attribute parameter set; the reference attribute type field is used to indicate a type of the attribute data corresponding to the current attribute parameter set.

18. A computer device, characterized in that: include: a processor suitable for executing a computer program; A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the point cloud media data processing method according to any one of claims 1 to 16 is executed.

19. A computer-readable storage medium, characterized in that The computer storage medium stores a computer program, and when the computer program is executed by the processor, the point cloud media data processing method according to any one of claims 1 to 16 is executed.

20. A computer program product, characterized in that The computer program product includes a computer program, which is stored in a computer-readable storage medium; the processor of the computer device reads and executes the computer program from the computer-readable storage medium, so that the computer device executes the point cloud media data processing method according to any one of claims 1 to 16.

Images