Data Processing Method, Apparatus, Device, and Storage Medium for Point Cloud Media

Abstract

Embodiments of the present application disclose a data processing method, apparatus, device, and storage medium for point cloud media. The method includes: obtaining a media file of the point cloud media, the media file including bitstream data of the point cloud media and parameter indication information, the bitstream data containing at least two parameter sets belonging to the same type, and the parameter indication information being used to indicate the association relationship between the at least two parameter sets and the point cloud frames in the point cloud media, and decoding the bitstream data according to the parameter indication information to present the point cloud media. It can be seen that the association relationship between the parameter set and the point cloud frames in the point cloud media can be described through the parameter indication information (such as indicating the parameter set required for decoding the currently decoded point cloud frame), so as to facilitate the parsing and presentation of the point cloud media.

Description

Technical Field

[0001] The present application relates to the field of computer technologies, and particularly to a data processing method for point cloud media, a data processing apparatus for point cloud media, a computer device, and a computer-readable storage medium. Background Art

[0002] With the progress of scientific and technological research, it is currently possible to obtain a large amount of high-precision point clouds at a relatively low cost and within a relatively short time period. Content production devices can encode these point clouds to obtain bitstream data. It has been found in practice that the bitstream data can carry multiple parameter sets of the same type. How to indicate the association relationship between the parameter sets and the point cloud frames in the point cloud media, so as to facilitate the parsing and presentation of the point cloud media, has become a hot issue in current research. Summary of the Invention

[0003] Embodiments of the present invention provide a data processing method, apparatus, device, and computer-readable storage medium for point cloud media, which can describe the association relationship between parameter sets and point cloud frames in the point cloud media.

[0004] On the one hand, an embodiment of the present application provides a data processing method for point cloud media, including:

[0005] Obtain a media file of the point cloud media, where the media file includes the bitstream data of the point cloud media and parameter indication information; the bitstream data contains at least two parameter sets of the same type; the parameter indication information is used to indicate the association relationship between the at least two parameter sets and the point cloud frames in the point cloud media;

[0006] Decode the bitstream data according to the parameter indication information to present the point cloud media.

[0007] In an embodiment of the present application, a media file of the point cloud media is obtained. The media file includes the bitstream data of the point cloud media and parameter indication information. The bitstream data contains at least two parameter sets of the same type. The parameter indication information is used to indicate the association relationship between the at least two parameter sets and the point cloud frames in the point cloud media. The bitstream data is decoded according to the parameter indication information to present the point cloud media. It can be seen that the association relationship between the parameter sets and the point cloud frames in the point cloud media can be described through the parameter indication information (such as indicating the parameter set required for decoding the currently decoded point cloud frame), so as to facilitate the parsing and presentation of the point cloud media.

[0008] On the one hand, an embodiment of the present application provides a data processing method for point cloud media, including:

[0009] Obtain the point cloud data of the point cloud media, and perform encoding processing on the point cloud data to obtain the bitstream data of the point cloud media. The bitstream data of the point cloud media contains at least two groups of parameter sets of the same type;

[0010] Generate parameter indication information according to the association relationship between the point cloud frames in the point cloud media and at least two parameter sets;

[0011] Encapsulate the bitstream data and the parameter indication information to obtain the media file of the point cloud media.

[0012] In the embodiments of the present application, the point cloud data of the point cloud media is obtained, and the point cloud data is encoded to obtain the bitstream data of the point cloud media. The bitstream data of the point cloud media includes at least two parameter sets of the same type. Parameter indication information is generated according to the association relationship between the point cloud frames in the point cloud media and at least two parameter sets, and the bitstream data and the parameter indication information are encapsulated to obtain the media file of the point cloud media. It can be seen that the association relationship between the parameter set and the point cloud frame in the point cloud media is indicated by the parameter indication information (such as indicating the parameter set required when decoding the currently decoded point cloud frame), so as to facilitate the analysis and presentation of the point cloud media.

[0013] On the one hand, the embodiments of the present application provide a data processing device for point cloud media, and the data processing device for point cloud media includes:

[0014] An acquisition unit, configured to acquire the media file of the point cloud media, where the media file includes the bitstream data of the point cloud media and parameter indication information; the bitstream data includes at least two parameter sets of the same type; the parameter indication information is used to indicate the association relationship between the at least two parameter sets and the point cloud frames in the point cloud media;

[0015] A processing unit, configured to decode the bitstream data according to the parameter indication information to present the point cloud media.

[0016] On the one hand, the embodiments of the present application provide a data processing device for point cloud media, and the data processing device for point cloud media includes:

[0017] An acquisition unit, configured to acquire the point cloud data of the point cloud media, and perform encoding processing on the point cloud data to obtain the bitstream data of the point cloud media, where the bitstream data of the point cloud media includes at least two parameter sets of the same type;

[0018] A processing unit, configured to generate parameter indication information according to the association relationship between the point cloud frames in the point cloud media and at least two parameter sets;

[0019] And for encapsulating the bitstream data and the parameter indication information to obtain the media file of the point cloud media.

[0020] Correspondingly, the present application provides a computer device, and the computer device includes:

[0021] A processor, configured to load and execute a computer program;

[0022] A memory stores a computer program which, when executed by a processor, implements the above-mentioned data processing method for point cloud media.

[0023] Correspondingly, the present application provides a computer-readable storage medium storing a computer program which is adapted to be loaded and executed by a processor to implement the above-mentioned data processing method for point cloud media.

[0024] Correspondingly, the present application provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions so that the computer device implements the above-mentioned data processing method for point cloud media.

[0025] In an embodiment of the present application, a content production device acquires point cloud data of point cloud media, performs encoding processing on the point cloud data to obtain bitstream data of the point cloud media. The bitstream data includes at least two sets of parameter sets of the same type. Parameter indication information is generated according to the association relationship between the point cloud frames in the point cloud media and the at least two parameter sets, and the bitstream data and the parameter indication information are encapsulated to obtain a media file of the point cloud media. A content consumption device acquires the media file of the point cloud media. The media file includes the bitstream data of the point cloud media and the parameter indication information, and decodes the bitstream data according to the parameter indication information to present the point cloud media. It can be seen that the content production device uses the parameter indication information to represent the association relationship between the point cloud frames and the parameter sets, so that the content consumption device can parse and present the point cloud media according to the parameter indication information. Description of the Drawings

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following described drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0027] Figure 1a A schematic diagram of 6DoF provided by an embodiment of the present application;

[0028] Figure 1b A schematic diagram of 3DoF provided by an embodiment of the present application;

[0029] Figure 1c A schematic diagram of 3DoF+ provided by an embodiment of the present application;

[0030] Figure 1dA data processing architecture diagram of point cloud media provided by an embodiment of the present application;

[0031] Figure 2 A flowchart of a data processing method for point cloud media provided by an embodiment of the present application;

[0032] Figure 3 A flowchart of another data processing method for point cloud media provided by an embodiment of the present application;

[0033] Figure 4a A sample schematic diagram provided by an embodiment of the present application;

[0034] Figure 4b A schematic diagram of point cloud segmentation provided by an embodiment of the present application;

[0035] Figure 5 A structural schematic diagram of a data processing device for point cloud media provided by an embodiment of the present application;

[0036] Figure 6 A structural schematic diagram of another data processing device for point cloud media provided by an embodiment of the present application;

[0037] Figure 7 A structural schematic diagram of a content consumption device provided by an embodiment of the present application;

[0038] Figure 8 A structural schematic diagram of a content production device provided by an embodiment of the present application. Detailed implementation manners

[0039] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0040] The embodiments of the present application relate to the data processing technology of point cloud media. Some concepts in the data processing process of point cloud media will be introduced below. In particular, in the subsequent embodiments of the present application, immersive media is taken as an example of point cloud media for illustration.

[0041] I. Immersive media:

[0042] Immersive media refers to media files that can provide immersive media content, allowing viewers immersed in the media content to obtain visual, auditory and other sensory experiences in the real world. Immersive media can be divided into 6DoF (Degree of Freedom) immersive media, 3DoF immersive media, and 3DoF+ immersive media according to the degree of freedom of viewers when consuming media content.

[0043] 2. Point Cloud:

[0044] A point cloud is a set of irregularly distributed discrete points in space that express the spatial structure and surface properties of a three-dimensional object or scene. Each point in a point cloud has at least three-dimensional position information, and may also have color, material, or other information depending on the application scenario. Usually, each point in a point cloud has the same number of additional attributes.

[0045] 3. Point cloud media:

[0046] Point cloud media is a typical 6DoF immersive media. Point cloud media can flexibly and conveniently express the spatial structure and surface properties of three-dimensional objects or scenes. Therefore, it is widely used in virtual reality (VR) games, computer-aided design (CAD), geographic information system (GIS), autonomous navigation system (ANS), digital cultural heritage, free viewpoint broadcasting, three-dimensional immersive telepresence, three-dimensional reconstruction of biological tissues and organs, and other projects.

[0047] 4. Track:

[0048] Tracks are a collection of media data in the process of media file encapsulation. A media file can be composed of one or more tracks. For example, a media file can contain a video track, an audio track, and a subtitle track. In particular, metadata information can also be included in the file as a media type in the form of metadata media tracks.

[0049] 5. Sample:

[0050] A sample is a unit of packaging in the process of media file packaging. A track is composed of many samples. For example, a video track can be composed of many samples. A sample is usually a video frame. In the embodiment of the present application, a sample in a track can be a point cloud frame.

[0051] 6. Sample Number:

[0052] In one embodiment, the serial number of the first sample in the track is 1.

[0053] VII. Sample Entry:

[0054] The sample entry is used to indicate the metadata information related to all samples in the track. For example, in the sample entry of a video track, metadata information related to decoder initialization is usually included.

[0055] VIII. Point Cloud Slice / Strip:

[0056] A point cloud slice / strip refers to a set of a series of syntax elements (such as geometric slices, attribute slices) of a point cloud frame in a partially or fully encoded point cloud media.

[0057] IX. Point Cloud Tile:

[0058] A point cloud tile is also known as a hexahedral space partition region within the boundary space region of a point cloud frame. A point cloud tile is composed of one or more point cloud slices, and there is no codec dependency relationship between point cloud tiles.

[0059] X. Sequence Parameter Set (SPS):

[0060] The sequence parameter set refers to the set of parameters required for decoding a point cloud sequence.

[0061] XI. Geometry Parameter Set (GPS):

[0062] The geometry parameter set refers to the set of parameters required for decoding point cloud geometry data.

[0063] XII. Attribute Parameter Set (APS):

[0064] The attribute parameter set refers to the set of parameters required for decoding point cloud attribute data.

[0065] XIII. Frame-specific attribute property (FSAP) of a specific point cloud frame:

[0066] The frame-specific attribute property of a specific point cloud frame is used to indicate the attributes of a specific point cloud frame.

[0067] XIV. ISOBMFF (ISO Based Media File Format):

[0068] ISOBMFF is a packaging standard for media files. A typical ISOBMFF file is an MP4 file.

[0069] XV. DASH (Dynamic Adaptive Streaming over HTTP):

[0070] DASH is an adaptive bitrate technology that enables high-quality streaming media to be delivered over the Internet via a traditional HTTP web server.

[0071] XVI. MPD (Media Presentation Description, the media presentation description signaling in DASH):

[0072] MPD is used to describe the media segment information in a media file.

[0073] Figure 1a This is a schematic diagram of 6DoF provided by an embodiment of the present application; 6DoF is divided into window 6DoF, omnidirectional 6DoF, and 6DoF. Among them, window 6DoF means that the viewer of immersive media is restricted in rotational movement on the X-axis and Y-axis, and translational movement on the Z-axis; for example, the viewer of immersive media cannot see the scene outside the window frame, and the viewer of immersive media cannot pass through the window. Omnidirectional 6DoF means that the viewer of immersive media is restricted in rotational movement on the X-axis, Y-axis, and Z-axis; for example, the viewer of immersive media cannot freely pass through the three-dimensional 360-degree VR content in the restricted movement area. 6DoF means that the viewer of immersive media can freely translate along the X-axis, Y-axis, and Z-axis; for example, the viewer of immersive media can freely walk in the three-dimensional 360-degree VR content. Similar to 6DoF, there are also 3DoF and 3DoF+ production technologies. Figure 1b This is a schematic diagram of 3DoF provided by an embodiment of the present application; as Figure 1b shown, 3DoF means that the viewer of immersive media is fixed at the center point of a three-dimensional space, and the viewer's head of immersive media rotates along the X-axis, Y-axis, and Z-axis to view the picture provided by the media content. Figure 1c This is a schematic diagram of 3DoF+ provided by an embodiment of the present application, as Figure 1c shown, 3DoF+ means that when the virtual scene provided by immersive media has certain depth information, the viewer's head of immersive media can move within a limited space based on 3DoF to view the picture provided by the media content.

[0074] With the continuous development of technology, it is currently possible to obtain a large amount of highly accurate point cloud data at a relatively low cost within a short time cycle. The acquisition methods of point cloud data include: computer generation, three-dimensional (3-Dimension, 3D) laser scanning, 3D photogrammetry, etc. Specifically, point cloud data can be obtained by a collection device (a set of cameras or a camera device with multiple lenses and sensors) through the acquisition of visual scenes in the real world. Through 3D laser scanning, the point cloud of static real-world three-dimensional objects or scenes can be obtained, and millions of point cloud data can be acquired per second; through 3D photography, the point cloud of dynamic real-world three-dimensional objects or scenes can be obtained, and tens of millions of point cloud data can be acquired per second; in addition, in the medical field, the point cloud data of biological tissue organs can be obtained through magnetic resonance imaging (MRI), computed tomography (CT), and electromagnetic positioning information. For another example, point cloud data can also be directly generated by a computer based on virtual three-dimensional objects and scenes. For example, a computer can generate the point cloud data of virtual three-dimensional objects and scenes. With the continuous accumulation of large-scale point cloud data, the efficient storage, transmission, publishing, sharing, and standardization of point cloud data have become the key to point cloud applications.

[0075] Figure 1d This is a data processing architecture diagram of point cloud media provided by an embodiment of this application. As Figure 1d shown, the data processing process at the content production device end mainly includes: (1) the process of obtaining the media content of point cloud data; (2) the process of encoding and file encapsulation of point cloud data. The data processing process at the content consumption device end mainly includes: (3) the process of file de-encapsulation and decoding of point cloud data; (4) the rendering process of point cloud data. In addition, there is a transmission process of point cloud media between the content production device and the content consumption device, and this transmission process can be carried out based on various transmission protocols. The transmission protocols here may include but are not limited to: DASH (Dynamic Adaptive Streaming over HTTP) protocol, HLS (HTTP Live Streaming) protocol, SMTP (Smart Media Transport Protocol), TCP (Transmission Control Protocol), etc.

[0076] The following will describe the data processing process of point cloud media in detail:

[0077] (1) Obtain the media content of point cloud media.

[0078] From the perspective of the acquisition method of the media content of point cloud media, it can be divided into two methods: one is obtained by capturing the sound-visual scene of the real world through a capture device, and the other is generated by a computer. In one implementation, the capture device may refer to a hardware component provided in the content production device. For example, the capture device refers to the microphone, camera, sensor, etc. of the terminal. In another implementation, the capture device may also be a hardware device connected to the content production device. For example, a camera connected to the server; it is used to provide the acquisition service of the media content of the point cloud data for the content production device. The capture device may include, but is not limited to: audio devices, imaging devices, and sensing devices. Among them, the audio device may include an audio sensor, a microphone, etc. The imaging device may include an ordinary camera, a stereo camera, a light field camera, etc. The sensing device may include a laser device, a radar device, etc. The number of capture devices may be multiple, and these capture devices are deployed at some specific positions in the real space to simultaneously capture the audio content and video content at different angles in this space. The captured audio content and video content are synchronized both in time and space. Due to different acquisition methods, the compression and encoding methods corresponding to the media content of different point cloud data may also be different.

[0079] (2) The process of encoding and file encapsulation of the media content of point cloud media.

[0080] Currently, the Geometry-based point cloud compression (GPCC) encoding method is usually used to encode the acquired point cloud data to obtain a Geometry-based point cloud compression bitstream (including an encoded geometry bitstream and an attribute bitstream). The encapsulation modes of the Geometry-based point cloud compression bitstream include a single-track encapsulation mode and a multi-track encapsulation mode.

[0081] The single-track encapsulation mode means that the bitstream data is encapsulated in the form of a single track. In the single-track encapsulation mode, a sample will contain one or more encoded content units (such as a geometry encoded content unit and multiple attribute encoded content units). The advantage of the single-track encapsulation mode is that: based on the bitstream data, a point cloud file encapsulated in a single track can be obtained without too much processing.

[0082] The multi-track encapsulation mode means that the bitstream data is encapsulated in the form of multiple tracks. In the multi-track encapsulation mode, each track contains a component in the bitstream data, that is, a geometry component track and one or more attribute component tracks. The advantage of multi-track encapsulation is that: encapsulating different components separately is beneficial for the client to select the required components for transmission and decoding consumption according to its own needs.

[0083] (3) The process of file decompression and decoding of point cloud media;

[0084] A content consumption device can obtain the media file resource of point cloud data and the corresponding media presentation description information through a content production device. The media file resource of point cloud data and the media presentation description information are transmitted from the content production device to the content consumption device through a transmission mechanism (such as DASH, SMT). The process of file de-encapsulation at the content consumption device side is the reverse of the file encapsulation process at the content production device side. The content consumption device de-encapsulates the media file resource according to the file format requirements of the point cloud media to obtain an encoded bitstream (GPCC bitstream or VPCC bitstream). The decoding process at the content consumption device side is the reverse of the encoding process at the content production device side. The content consumption device decodes the encoded bitstream to restore the point cloud data.

[0085] (4) Rendering process of point cloud media.

[0086] The content consumption device renders the point cloud data obtained by decoding the GPCC bitstream according to the metadata related to rendering and viewport in the media presentation description information to obtain the point cloud frame of the point cloud media, and presents the point cloud media according to the presentation time of the point cloud frame.

[0087] In one embodiment, at the content production device side: First, the acquisition device samples the visual scene of the real world to obtain the point cloud data corresponding to the visual scene of the real world; then, through geometry-based point cloud compression (GPCC), the obtained point cloud data is encoded to obtain a GPCC bitstream (including the encoded geometry bitstream and attribute bitstream); then, the GPCC bitstream is encapsulated to obtain the media file corresponding to the point cloud data (i.e., the point cloud media). Specifically, the content production device synthesizes one or more encoded bitstreams into a media file for file playback, or a sequence of initialization segments and media segments for streaming transmission according to a specific media container file format; where the media container file format refers to the ISO base media file format specified in International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC) 14496-12. In one implementation, the content production device also encapsulates the metadata into the media file or the sequence of initialization / media segments, and transmits the sequence of initialization / media segments to the content consumption device through a transmission mechanism (such as the dynamic adaptive streaming transport interface).

[0088] On the content consumption device side: First, receive the point cloud media file sent by the content production device, including: a media file for file playback, or a sequence of initialization segments and media segments for streaming; then perform demultiplexing processing on the point cloud media file to obtain the encoded GPCC bitstream; next, parse the encoded GPCC bitstream (i.e., perform decoding processing on the encoded GPCC bitstream to obtain point cloud data); in a specific implementation, the content consumption device determines the media file or media segment sequence required to present the point cloud media based on the viewing position / viewing direction of the current object; and performs decoding processing on the media file or media segment sequence required to present the point cloud media to obtain the point cloud data required for presentation. Finally, based on the viewing (window) direction of the current object, render the decoded point cloud data to obtain the point cloud frame of the point cloud media, and present the point cloud media on the screen of the head-mounted display or any other display device carried by the content consumption device according to the presentation time of the point cloud frame. It should be noted that the viewing position / viewing direction of the current object is determined by head tracking and possibly also visual tracking functions. In addition to using the renderer to render the point cloud data of the viewing position / viewing direction of the current object, the audio decoder can also be used to optimize the decoding of the audio in the viewing (window) direction of the current object.

[0089] Among them, the content production device and the content consumption device can jointly form a point cloud media system. The content production device can refer to the computer device used by the provider of the point cloud media (such as the content producer of the point cloud media), and this computer device can be a terminal (such as a PC (Personal Computer), a smart mobile device (such as a smartphone), etc.) or a server; among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms. The content consumption device can refer to the computer device used by the user of the point cloud media (such as the viewer of the point cloud media), and this computer device can be a terminal (such as a PC (Personal Computer), a smart mobile device (such as a smartphone), a VR device (such as a VR headset, VR glasses, etc.), smart home appliances, in-vehicle terminals, aircraft, etc.).

[0090] It can be understood that the data processing technology of point cloud media involved in this application can be implemented relying on cloud technology; for example, using a cloud server as a content production device. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or a local area network to achieve data calculation, storage, processing, and sharing. The data processing solution for point cloud media provided by the embodiments of this application can also be combined with vehicle networking technology. Specifically, this data processing solution for point cloud media can collect buildings, traffic markers, etc. in the environment and construct a point cloud map in the vehicle for positioning, or use this point cloud map to achieve automatic navigation.

[0091] Based on the above introduction of point cloud media, this application proposes a data processing method for point cloud media. This method can describe the association relationship between the parameter set and the point cloud frame in the media file through parameter indication information, so that the content consumption device can parse and present the point cloud media according to the parameter indication information.

[0092] The parameter indication information can include at least one of a parameter set sample group (GPCCParameterSetInfoEntry) and a decoding configuration record (GPCCDecoderConfigurationRecord). The parameter set sample group is used to indicate the parameter set required for one or more samples in the media file during decoding, and the decoding configuration record is used to indicate the initialization parameter information required for the bitstream data during decoding.

[0093] In one implementation, the parameter indication information includes a parameter set sample group (GPCCParameterSetInfoEntry). The parameter set sample group can be included in the sample group description box (Sample Group Description Box). The data box type corresponding to the parameter set sample group is 'gpsg', the mandatory type is "no", and the quantity is 0 or more.

[0094] Specifically, the parameter set sample group entry defines the parameter set information in the samples containing the shared G-PCC parameter set. When there are multiple sample group data boxes (SampleToGroupBox) with the sample group type of 'gpsg', the version field of these sample group data boxes needs to take the value of 1.

[0095] When there is a sample group description box (Sample Group Description Box) of type 'gpsg' and a sample group data box (SampleToGroupBox), the following conditions need to be met:

[0096] (1) When a parameter set PS_A is updated by a new parameter set PS_B in sample X (i.e., parameter set PS_B is the updated version of parameter set PS_A), in the decoding order, all samples after sample X should not refer to PS_A.

[0097] (2) When a parameter set of a specific type (such as SPS, GPS, APS, and FSAP) included in sample X is referred to by more than one sample, sample X must belong to a SampleToGroupBox of the sample group data box of type 'gpsg' and have a group type parameter field (group_type_parameter) corresponding to the type of the parameter set it contains.

[0098] (3) For any sample that does not contain a parameter set or contains a parameter set of other types, its group description index field (group_description_index) should have a value of 0.

[0099] The syntax of the parameter set sample group (GPCCParameterSetInfoEntry) can be seen in Table 1 below:

[0100] Table 1

[0101]

[0102]

[0103] The semantics of the syntax in Table 1 above are as follows:

[0104] (1) The parameter set ID present field (parameter_set_id_present) is used to indicate whether the parameter set identifier of the parameter set contained in the current sample is indicated in the parameter set sample group. When the value of the parameter set ID present field is the first set value (such as 1), it means that the parameter set identifier of the parameter set contained in the current sample is indicated in the parameter set sample group. When the value of the parameter set ID present field is the second set value (such as 0), it means that the parameter set identifier of the parameter set contained in the current sample is not indicated in the parameter set sample group; in this case, the bitstream data does not contain other parameter sets of the same type as the parameter set contained in the current sample and with different parameter set identifiers; it can be understood that there may be N parameter sets of the same type as the parameter set contained in the current sample and with the same parameter set identifier in the bitstream data. For the content production device, the current sample refers to the sample being encoded, and for the content consumption device, the current sample refers to the sample being decoded.

[0105] (2) The point cloud slice information field (slice_info_present) is used to indicate whether there is an association between the parameter set included in the current sample and the point cloud slice. When the value of the point cloud slice information field is the first set value (such as 1), it indicates that there is an association between the parameter set included in the current sample and the corresponding point cloud slice; when the value of the point cloud slice information field is the second set value (such as 0), it indicates that the association between the parameter set included in the current sample and the point cloud slice is not indicated.

[0106] (3) The point cloud tile information field (tile_info_present) is used to indicate whether there is an association between the parameter set included in the current sample and the point cloud tile. When the value of the point cloud tile information field is the first set value (such as 1), it indicates that there is an association between the parameter set included in the current sample and the corresponding point cloud tile; when the value of the point cloud tile information field is the second set value (such as 0), it indicates that the association between the parameter set included in the current sample and the point cloud tile is not indicated.

[0107] (4) The point cloud slice group information field (slice_tag_info_present) is used to indicate whether there is an association between the parameter set included in the current sample and the point cloud slice group. When the value of the point cloud slice group information field is the first set value (such as 1), it indicates that there is an association between the parameter set included in the current sample and the corresponding point cloud slice group; when the value of the point cloud slice group information field is the second set value (such as 0), it indicates that the association between the parameter set included in the current sample and the point cloud slice group is not indicated.

[0108] (5) The parameter set number field (parameter_set_num) is used to indicate the number of parameter sets included in the current sample.

[0109] (6) The parameter set identification field (parametetSetID) is used to indicate the identifier of the parameter set included in the current sample.

[0110] (7) The point cloud slice number field (associated_slice_num) is used to indicate the number of point cloud slices associated with the parameter set included in the current sample.

[0111] (8) The point cloud slice identification field (slice_id) is used to indicate the identifier of the point cloud slice associated with the parameter set included in the current sample.

[0112] (9) The point cloud tile number field (associated_tile_num) is used to indicate the number of point cloud tiles associated with the parameter set included in the current sample.

[0113] (10) The point cloud tile identification field (tile_id) is used to indicate the identifier of the point cloud tile associated with the parameter set included in the current sample.

[0114] (11) The number of associated point cloud slice groups field (associated_slice_tag_num) is used to indicate the number of point cloud slice groups associated with the parameter set included in the current sample.

[0115] (12) The point cloud slice group identification field (slice_tag) is used to indicate the identifier of the point cloud slice group associated with the parameter set included in the current sample.

[0116] (13) The parameter set type field (parametersetType) is used to indicate the type of the parameter set. For the specific indication method, refer to Table 2:

[0117] Table 2

[0118] Field value Description 0 Reserved 1 Sequence parameter set 2 Geometric parameter set 3 Attribute parameter set 4 Specific frame attribute characteristic information

[0119] 5 to 255 Reserved

[0120] As can be seen from Table 2 above, when the value of the parameter set type field is 1, it indicates that the type of the parameter set corresponding to this field is a sequence parameter set; when the value of the parameter set type field is 2, it indicates that the type of the parameter set corresponding to this field is a set parameter set; when the value of the parameter set type field is 3, it indicates that the type of the parameter set corresponding to this field is an attribute parameter set; when the value of the parameter set type field is 4, it indicates that the parameter set corresponding to this field is specific frame attribute characteristic information.

[0121] In another implementation, the parameter indication information includes a decoding configuration record (GPCCDecoderConfigurationRecord). Some extended fields in the above parameter set sample group (GPCCParameterSetInfoEntry) can also be applied in the decoding configuration record. The syntax of the decoding configuration record can be referred to Table 3 below:

[0122] Table 3

[0123]

[0124]

[0125] The semantics of the syntax in Table 3 above are as follows:

[0126] (1) The parameter set identification field (parameter_set_id_present) is used to indicate whether the parameter set identifier of the parameter set included in the current sample is indicated in the decoding configuration record. When the value of the parameter set identification field is the first set value (such as 1), it indicates that the parameter set identifier of the parameter set included in the current sample is indicated in the decoding configuration record. When the value of the parameter set identification field is the second set value (such as 0), it indicates that the parameter set identifier of the parameter set included in the current sample is not indicated in the decoding configuration record; in this case, the bitstream data does not include other parameter sets that belong to the same type as the parameter set included in the current sample and have different parameter set identifiers; it can be understood that the bitstream data can include N parameter sets that belong to the same type as the parameter set included in the current sample and have the same parameter set identifier, where N is 0 or a positive integer. For the content production device, the current sample refers to the sample being encoded, and for the content consumption device, the current sample refers to the sample being decoded.

[0127] (2) The parameter set identification field (parametetSetID) is used to indicate the identifier of the parameter set included in the current sample.

[0128] (3) The point cloud slice information field (slice_info_present) is used to indicate whether there is an association relationship between the parameter set included in the current sample and the point cloud slice. When the value of the point cloud slice information field is the first set value (such as 1), it indicates that there is an association relationship between the parameter set included in the current sample and the corresponding point cloud slice; when the value of the point cloud slice information field is the second set value (such as 0), it indicates that the association relationship between the parameter set included in the current sample and the point cloud slice is not indicated.

[0129] (4) The point cloud tile information field (tile_info_present) is used to indicate whether there is an association relationship between the parameter set included in the current sample and the point cloud tile. When the value of the point cloud tile information field is the first set value (such as 1), it indicates that there is an association relationship between the parameter set included in the current sample and the corresponding point cloud tile; when the value of the point cloud tile information field is the second set value (such as 0), it indicates that the association relationship between the parameter set included in the current sample and the point cloud tile is not indicated.

[0130] (5) The point cloud slice group information field (slice_tag_info_present) is used to indicate whether there is an association relationship between the parameter set included in the current sample and the point cloud slice group. When the value of the point cloud slice group information field is the first set value (such as 1), it indicates that there is an association relationship between the parameter set included in the current sample and the corresponding point cloud slice group; when the value of the point cloud slice group information field is the second set value (such as 0), it indicates that the association relationship between the parameter set included in the current sample and the point cloud slice group is not indicated.

[0131] (6) The number of point cloud slices field (associated_slice_num) is used to indicate the number of point cloud slices associated with the parameter set included in the current sample.

[0132] (7) The point cloud slice identifier field (slice_id) is used to indicate the identifier of the point cloud slice associated with the parameter set included in the current sample.

[0133] (8) The number of point cloud tiles field (associated_tile_num) is used to indicate the number of point cloud tiles associated with the parameter set included in the current sample.

[0134] (9) The point cloud tile identifier field (tile_id) is used to indicate the identifier of the point cloud tile associated with the parameter set included in the current sample.

[0135] (10) The number of point cloud slice groups field (associated_slice_tag_num) is used to indicate the number of point cloud slice groups associated with the parameter set included in the current sample.

[0136] (11) The point cloud slice group identifier field (slice_tag) is used to indicate the identifier of the point cloud slice group associated with the parameter set included in the current sample.

[0137] (12) The configuration version field (configurationVersion) is used to configure the record version information. If the content consumption device cannot recognize the version information of this field in the corresponding track, the content consumption device should not attempt to decode the corresponding track.

[0138] (13) The profile field (profile_idc) is used to indicate the profile information of the bitstream data corresponding to the current track; for the content production device, the current track refers to the track being encoded, and for the content consumption device, the current track refers to the sample being decoded.

[0139] (14) The profile compatibility flag field (profile_compatibility_flags) is used to indicate the profile compatibility information of the bitstream data corresponding to the current track.

[0140] (15) The number of parameter set groups field (num_of_setup_unit_arrays) is used to indicate the number of parameter set groups included in the current decoding configuration record. Each parameter set group corresponds to a type of parameter set.

[0141] (16) The parameter set group type field (setup_unit_type) is used to indicate the parameter set type corresponding to the current parameter set group.

[0142] (17) Parameter set integrity field (array_completeness) When the value of the parameter set integrity field is the first set value (e.g., 1), it indicates that all the parameter sets of the current type included in the corresponding bitstream data are indicated in the decoding configuration record. When the value of the parameter set integrity field is the second set value (e.g., 0), it indicates that a part of the parameter sets of the current type included in the corresponding bitstream data is indicated in the decoding configuration record, and the other part is indicated in the bitstream data.

[0143] (18) Parameter set number field (num_of_setup_units) is used to indicate the number of parameter sets of the current type.

[0144] (19) Unit indication field (setupUnit) is used to indicate the data unit containing SPS, GPS, APS, or point cloud chunk information in the decoding configuration record.

[0145] In the embodiments of the present application, the association relationship between the point cloud frame and the parameter set is indicated through the parameter indication information, so that the content consumption device can parse and present the point cloud media according to the parameter indication information.

[0146] Figure 2 It is a flowchart of a data processing method for point cloud media provided by the embodiments of the present application; this method can be executed by a content consumption device in a point cloud media system, and this method includes the following steps S201 and step S202:

[0147] S201. Obtain the media file of the point cloud media.

[0148] The media file includes the bitstream data of the point cloud media and the parameter indication information. The bitstream data includes at least two parameter sets of the same type. The parameter indication information is used to indicate the association relationship between the parameter sets included in the bitstream data and the point cloud frames in the point cloud media.

[0149] In one embodiment, the media file includes one or more tracks, each track includes one or more samples, and one sample corresponds to one point cloud frame of the point cloud media. The parameter indication information is used to indicate the reference samples required for decoding one or more samples in the media file. There is at least one track among the one or more tracks included in the media file that includes one or more reference samples. The so-called reference sample is a sample carrying a parameter set.

[0150] In one implementation, the content consumption device downloads the complete file of the point cloud media. The media file of the point cloud media is obtained by the content consumption device from the complete file of the point cloud media stored locally according to the requirements of the object (such as the presentation frame rate of the point cloud media set by the object) or its own decoding ability.

[0151] In another implementation, the media file of the point cloud media is sliced into multiple media segments. The content consumption device obtains the transmission signaling file of the point cloud media. The content consumption device determines the media segments required to present the point cloud media according to the requirements of an object (such as a viewer) (such as the presentation frame rate of the point cloud media set by the object), or the virtual viewing angle of the viewer during viewing, or its own decoding ability, or the current network condition (such as the network transmission speed), and pulls the media file of the determined point cloud media through a streaming transmission method.

[0152] S202. Decode the bitstream data according to the parameter indication information to present the point cloud media.

[0153] The parameter indication information includes at least one of a parameter set sample group (GPCCParameterSetInfoEntry) and a decoding configuration record (GPCCDecoderConfigurationRecord); the parameter set sample group is used to indicate the parameter set required for decoding one or more samples in the media file, and the decoding configuration record is used to indicate the initialization parameter information required for decoding the bitstream data.

[0154] In one implementation, the bitstream data includes at least two parameter sets of the same type, and at least one of the at least two parameter sets has multiple versions. Multiple versions refer to at least two versions; for example, the version before update and the version after update, or multiple versions corresponding to multiple updates of the parameter set, with one update corresponding to one version. Each version of the parameter set with multiple versions is carried in different reference samples; for example, assume that the bitstream data includes version 1 of the parameter set A before update and version 2 after update, then version 1 and version 2 of the parameter set A are carried in different reference samples, and the serial number of the reference sample carrying version 2 is greater than the serial number of the reference sample carrying version 1. In practical applications, the samples located after the reference sample carrying version 2 will not refer to the reference sample carrying version 1 during decoding.

[0155] The parameter indication information includes a parameter set sample group (GPCCParameterSetInfoEntry). The parameter set sample group entry defines the parameter set information in the samples containing the shared G-PCC parameter set. When there are multiple sample group boxes (SampleToGroupBox) with the sample group type of 'gpsg', the version field of these sample group boxes should take the value of 1.

[0156] When there is a Sample Group Description Box of type 'gpsg' and a SampleToGroupBox, the following conditions shall be met:

[0157] (1) When a parameter set PS_A is updated by a new parameter set PS_B in sample X (i.e., PS_B is the updated version of PS_A), in the decoding order, all samples after sample X should not refer to PS_A.

[0158] (2) When a parameter set of a specific type (such as SPS, GPS, APS, and FSAP) included in sample X is referred to by more than one sample, sample X must belong to a SampleToGroupBox of type 'gpsg' and have a group_type_parameter corresponding to the type of the parameter set it contains.

[0159] (3) For any sample that does not contain a parameter set or contains a parameter set of other types, the value of its group_description_index shall be 0.

[0160] In an embodiment, the parameter set sample group includes a parameter_set_id_present; if the value of the parameter_set_id_present is the first set value (such as 1), the content consumption device determines the parameter set identifier in the parameter set sample group that indicates the parameter set contained in the current sample (the current sample refers to the sample being decoded); if the value of the parameter_set_id_present is the second set value (such as 0), the content consumption device determines the parameter set identifier in the parameter set sample group that does not indicate the parameter set contained in the current sample; when the value of the parameter_set_id_present is the second set value (such as 0), the bitstream data does not contain other parameter sets of the same type as the parameter set contained in the current sample and with different parameter set identifiers; it can be understood that the bitstream data may contain N parameter sets of the same type as the parameter set contained in the current sample and with the same parameter set identifier, where N is 0 or a positive integer.

[0161] In one embodiment, a sample may be divided into one or more point cloud slices; the parameter set sample group includes a point cloud slice information field (slice_info_present); if the value of the point cloud slice information field is a first set value (such as 1), the content consumption device determines that there is an association relationship between the parameter set included in the current sample (the current sample refers to the sample being decoded) and at least one point cloud slice; for example, if the value of the point cloud slice information field is 1, the content consumption device determines that there is an association relationship between the parameter set included in the current sample and the corresponding point cloud slice (such as at least one point cloud slice obtained by dividing the current sample). If the value of the point cloud slice information field is a second set value (such as 0), the content consumption device determines that the association relationship between the parameter set included in the current sample and the point cloud slice is not indicated.

[0162] In one embodiment, a sample may be divided into one or more point cloud tiles; the parameter set sample group includes a point cloud tile information field (tile_info_present); if the value of the point cloud tile information field is a first set value (such as 1), the content consumption device determines that there is an association relationship between the parameter set included in the current sample (the current sample refers to the sample being decoded) and at least one point cloud tile; for example, if the value of the point cloud tile information field is 1, the content consumption device determines that there is an association relationship between the parameter set included in the current sample and the corresponding point cloud tile (such as at least one point cloud tile obtained by dividing the current sample). If the value of the point cloud tile information field is a second set value (such as 0), the content consumption device determines that the association relationship between the parameter set included in the current sample and the point cloud tile is not indicated.

[0163] In one embodiment, a sample may be divided into M point cloud slices, where M is a positive integer; the M point cloud slices may form one or more point cloud slice groups. The parameter set sample group includes a point cloud slice group information field (slice_tag_info_present); if the value of the point cloud slice group information field is a first set value (such as 1), the content consumption device determines that there is an association relationship between the parameter set included in the current sample (the current sample refers to the sample being decoded) and at least one point cloud slice group; for example, if the value of the point cloud slice group information field is 1, the content consumption device determines that there is an association relationship between the parameter set included in the current sample and the corresponding point cloud slice group (such as one or more point cloud slice groups formed by the M point cloud slices obtained by dividing the current sample). If the value of the point cloud slice group information field is a second set value (such as 0), the content consumption device determines that the association relationship between the parameter set included in the current sample and the point cloud slice group is not indicated.

[0164] In one embodiment, the parameter set sample group includes a parameter set number field (parameter_set_num); the content consumption device can determine the number of parameter sets included in the current sample (the current sample refers to the sample being decoded) according to the parameter set number field; for example, if parameter_set_num = 2, the content consumption device determines that the number of parameter sets included in the current sample is 2 according to the parameter set number field.

[0165] In one embodiment, the parameter set sample group includes a parameter set identification field (parametetSetID); the content consumption device can determine the parameter set included in the current sample (the current sample refers to the sample being decoded) according to the parameter set identification field; for example, if parametetSetID = 1, the content consumption device determines that the current sample includes a parameter set with the parameter set identifier "1" according to the parameter set identification field.

[0166] In one embodiment, a sample can be divided into one or more point cloud slices; the parameter set sample group includes a point cloud slice number field (associated_slice_num); the content consumption device can determine the number of point cloud slices associated with the parameter set corresponding to this field according to the point cloud slice number field, and the point cloud slices associated with the parameter set can be understood as the point cloud slices that need to use this parameter set during decoding. For example, assume that the current sample is divided into M (M > 2) point cloud slices, and parameter set A is carried in the current sample. If the value of the point cloud slice number field corresponding to parameter set A is 2 (i.e., associated_slice_num = 2), the content consumption device determines that the number of point cloud slices associated with parameter set A is 2. The 2 point cloud slices associated with parameter set A can belong to the M point cloud slices obtained by dividing the current sample, or can belong to the point cloud slices obtained by dividing other samples.

[0167] In one embodiment, a sample can be divided into one or more point cloud slices; the parameter set sample group includes a point cloud slice identification field (slice_id); the content consumption device can determine the point cloud slice associated with the parameter set corresponding to this field according to the point cloud slice identification field, and the point cloud slices associated with the parameter set can be understood as the point cloud slices that need to use this parameter set during decoding. For example, assume that the point cloud slice identification field slice_id = 1 corresponding to parameter set A, the content consumption device determines that the point cloud slice associated with parameter set A is the point cloud slice with the point cloud slice identifier "1". It can be understood that the values of the above point cloud slice identification field are only for illustration and do not constitute an actual limitation of this application.

[0168] In one embodiment, a sample can be divided into one or more point cloud tiles; the parameter set sample group includes a point cloud tile number field (associated_tile_num); the content consumption device can determine the number of point cloud tiles associated with the parameter set corresponding to this field according to the point cloud tile number field. The point cloud tiles associated with the parameter set can be understood as the point cloud tiles that need to use this parameter set during decoding. For example, assume that the current sample is divided into M (M>2) point cloud tiles, and parameter set A is carried in the current sample. If the value of the point cloud tile number field corresponding to parameter set A is 2 (i.e., associated_tile_num = 2), then the content consumption device determines that the number of point cloud tiles associated with parameter set A is 2, and the 2 point cloud tiles associated with parameter set A can belong to the M point cloud tiles obtained by dividing the current sample.

[0169] In one embodiment, a sample can be divided into one or more point cloud tiles; the parameter set sample group includes a point cloud tile identification field (tile_id); the content consumption device can determine the point cloud tiles associated with the parameter set corresponding to this field according to the point cloud tile identification field. The point cloud tiles associated with the parameter set can be understood as the point cloud tiles that need to use this parameter set during decoding. For example, assume that the point cloud tile identification field tile_id = 1 corresponding to parameter set A, then the content consumption device determines that the point cloud tile associated with parameter set A is the point cloud tile with the point cloud tile identifier "1". It can be understood that the values of the above point cloud tile identification field are only for illustration and do not constitute the actual limitation of this application.

[0170] In one embodiment, a sample can be divided into M point cloud slices, where M is a positive integer; the M point cloud slices can form one or more point cloud slice groups. The parameter set sample group includes a point cloud slice group number field (associated_slice_tag_num); the content consumption device can determine the number of point cloud slice groups associated with the parameter set corresponding to this field according to the point cloud slice group number field. The point cloud slice groups associated with the parameter set can be understood as the point cloud slice groups that need to use this parameter set during decoding. For example, assume that the current sample is divided into M point cloud slices, the M point cloud slices form N (N>2) point cloud slice groups, and parameter set A is carried in the current sample. If the value of the point cloud slice group number field corresponding to parameter set A is 2 (i.e., associated_slice_tag_num = 2), then the content consumption device determines that the number of point cloud slice groups associated with parameter set A is 2, and the 2 point cloud slice groups associated with parameter set A can belong to the N point cloud slice groups formed by the M point cloud slices.

[0171] In one embodiment, a sample can be divided into M point cloud slices, where M is a positive integer; the M point cloud slices can form one or more point cloud slice groups. The parameter set sample group includes a point cloud slice group identification field (slice_tag); the content consumption device can determine, based on the point cloud slice group identification field, the point cloud slice group associated with the parameter set corresponding to this field. The point cloud slice group associated with the parameter set can be understood as the point cloud slice group that needs to use this parameter set during decoding. For example, assume that the point cloud slice group identification field slice_tag = 1 corresponding to parameter set A, then the content consumption device determines that the point cloud slice group associated with parameter set A is the point cloud slice group with the point cloud slice group identifier "1". It can be understood that the values of the above point cloud slice group identification field are only for illustration and do not constitute an actual limitation of this application.

[0172] In another implementation, the parameter indication information includes a decoding configuration record (GPCCDecoderConfigurationRecord). The decoding configuration record does not include the parameter set quantity field (parameter_set_num) in the parameter set sample group (GPCCParameterSetInfoEntry). For specific embodiments of the remaining extended fields, reference can be made to the extended fields in the parameter set sample group and the description in Table 3 above, which will not be elaborated here.

[0173] For example, the decoding configuration record includes a parameter set identification field (parameter_set_id_present); if the value of the parameter set identification field is a first set value (such as 1), then the content consumption device determines the parameter set identifier of the parameter set included in the current sample (the current sample refers to the sample being decoded); if the value of the parameter set identification field is a second set value (such as 0), then the content consumption device determines that the parameter set identifier of the parameter set included in the current sample is not indicated in the decoding configuration record; when the value of the parameter set identification field is the second set value (such as 0), the bitstream data does not include other parameter sets that belong to the same type as the parameter set included in the current sample and have different parameter set identifiers; it can be understood that the bitstream data can include N parameter sets that belong to the same type as the parameter set included in the current sample and have the same parameter set identifier, where N is 0 or a positive integer.

[0174] After the content consumption device decodes the bitstream data according to one or more of the above embodiments, it can present the point cloud media based on the decoding result. For specific implementation methods, reference can be made to Figure 1d the implementation methods in (3) and (4) therein, which will not be elaborated here.

[0175] In an embodiment of the present application, a media file of point cloud media is obtained. The media file includes the bitstream data of the point cloud media and parameter indication information. The bitstream data contains at least two parameter sets of the same type. The parameter indication information is used to indicate the association relationship between the at least two parameter sets and the point cloud frames in the point cloud media. The bitstream data is decoded according to the parameter indication information to present the point cloud media. It can be seen that the association relationship between the parameter set and the point cloud frames in the point cloud media can be described through the parameter indication information (such as indicating the parameter set required for decoding the currently decoded point cloud frame), so as to facilitate the parsing and presentation of the point cloud media.

[0176] Figure 3 FIG. 4 is a flowchart of another data processing method for point cloud media provided by an embodiment of the present application; this method can be executed by a content production device in a point cloud media system. This method includes the following steps S301 and step S303:

[0177] S301. Obtain the point cloud data of the point cloud media, and perform encoding processing on the point cloud data to obtain the bitstream data of the point cloud media.

[0178] The bitstream data contains at least two parameter sets of the same type. For the specific acquisition method of the point cloud data of the point cloud media, reference can be made to Figure 1d the implementation manner in (1) in [reference document], and for the specific implementation manner of the content production device performing encoding processing on the point cloud data to obtain the bitstream data of the point cloud media, reference can be made to Figure 1d the implementation manner in (2) in [reference document], which will not be elaborated here.

[0179] S302. Generate parameter indication information according to the association relationship between the point cloud frames in the point cloud media and the at least two parameter sets.

[0180] The parameter indication information is used to indicate the association relationship between the at least two parameter sets and the point cloud frames in the point cloud media. The parameter indication information includes at least one of a parameter set sample group (GPCCParameterSetInfoEntry) and a decoding configuration record (GPCCDecoderConfigurationRecord); the parameter set sample group is used to indicate the parameter sets required for decoding one or more samples in the media file, and the decoding configuration record is used to indicate the initialization parameter information required for decoding the bitstream data.

[0181] The content production device can encapsulate the bitstream data in one or more tracks. Each track contains one or more samples, and one sample corresponds to one point cloud frame of the point cloud media. The parameter indication information is used to indicate the reference samples that one or more samples in the media file need to refer to during decoding. At least one of the one or more tracks contains one or more reference samples. A so-called reference sample is a sample carrying a parameter set.

[0182] In one embodiment, the bitstream data includes at least two parameter sets of the same type, and at least one of the at least two parameter sets has multiple versions, where multiple versions means at least two versions; for example, the version before update and the version after update, or multiple versions corresponding to multiple updates of the parameter set, with one update corresponding to one version. Each version of the parameter set with multiple versions is carried in a different reference sample; for example, assume that the bitstream data includes version 1 of parameter set A before update and version 2 after update, then version 1 and version 2 of parameter set A are carried in different reference samples, and the sequence number of the reference sample carrying version 2 is greater than that of the reference sample carrying version 1. In practical applications, samples after the reference sample carrying version 2 will not refer to the reference sample carrying version 1 during decoding.

[0183] The parameter indication information includes a parameter set sample group (GPCCParameterSetInfoEntry), and the parameter set sample group is used to indicate the parameter set required for one or more samples in the media file during decoding; for example, the parameter set sample group can indicate the parameter set required for the current sample during decoding by indicating the reference sample (the sample carrying the parameter set) that the current sample needs to refer to during decoding.

[0184] Specifically, the parameter set sample group entry defines the parameter set information in the sample containing the shared G-PCC parameter set. When there are multiple sample group data boxes (SampleToGroupBox) with the sample group type 'gpsg', the version field of these sample group data boxes should have a value of 1.

[0185] When there is a sample group description data box (Sample Group Description Box) of type 'gpsg' and a sample group data box (SampleToGroupBox), the following conditions should be met:

[0186] (1) When a parameter set PS_A is updated by a new parameter set PS_B in sample X (i.e., parameter set PS_B is the version after the update of parameter set PS_A), in the decoding order, all samples after sample X should not refer to PS_A.

[0187] (2) When a parameter set of a specific type (such as SPS, GPS, APS, and FSAP) included in sample X is referenced by more than one sample, sample X must belong to a SampleToGroupBox of sample group data of type 'gpsg' and have a group type parameter field (group_type_parameter) corresponding to the type of the parameter set it contains.

[0188] (3) For any sample that does not contain a parameter set or contains a parameter set of other types, the value of its group description index field (group_description_index) shall be 0.

[0189] In one embodiment, the parameter set sample group includes a parameter set identifier field (parameter_set_id_present); when the value of the parameter set identifier field is a first set value (such as 1), it indicates the parameter set identifier of the parameter set included in the current sample (i.e., the sample being encoded) in the parameter set sample group; when the value of the parameter set identifier field is a second set value (such as 0), it indicates the parameter set identifier of the parameter set not included in the current sample in the parameter set sample group. When the value of the parameter set identifier field is the second set value, the bitstream data does not contain other parameter sets of the same type as the parameter set included in the current sample and with different parameter set identifiers; it can be understood that the bitstream data can contain N parameter sets of the same type as the parameter set included in the current sample and with the same parameter set identifier, where N is 0 or a positive integer. For a content production device, the current sample refers to the sample being encoded, and for a content consumption device, the current sample refers to the sample being decoded.

[0190] Specifically, during the process of generating parameter indication information by the content production device, if the parameter set identifier of the parameter set included in the current sample is indicated in the parameter set sample group, the content production device sets the value of the parameter set identifier field corresponding to the current sample to the first set value; if the parameter set identifier of the parameter set included in the current sample is not indicated in the parameter set sample group, the content production device determines that the bitstream data does not contain other parameter sets of the same type as the parameter set included in the current sample and with different parameter set identifiers, and sets the value of the parameter set identifier field corresponding to the current sample to the second set value.

[0191] In one embodiment, during the encoding process, the content production device may divide a sample into one or more point cloud slices; the parameter set sample group includes a point cloud slice information field (slice_info_present); when the value of the point cloud slice information field is a first set value (such as 1), it indicates that there is an association relationship between the parameter set included in the current sample (i.e., the sample being encoded) and at least one point cloud slice; for example, assuming the current sample is divided into M point cloud slices, if the point cloud slice information field slice_info_present = 1, it indicates that there is an association relationship between the parameter set included in the current sample and at least one of the M point cloud slices. When the value of the point cloud slice information field is a second set value (such as 0), it indicates that the association relationship between the parameter set included in the current sample and the point cloud slice is not indicated.

[0192] Specifically, during the process of generating parameter indication information, if there is an association relationship between the parameter set included in the current sample and at least one point cloud slice, the content production device sets the value of the point cloud slice information field corresponding to the current sample to the first set value; if the association relationship between the parameter set included in the current sample and the point cloud slice is not indicated, the content production device sets the value of the point cloud slice information field corresponding to the current sample to the second set value.

[0193] In one embodiment, during the encoding process, the content production device may divide a sample into one or more point cloud tiles; the parameter set sample group includes a point cloud tile information field (tile_info_present); when the value of the point cloud tile information field is a first set value (such as 1), it indicates that there is an association relationship between the parameter set included in the current sample (i.e., the sample being encoded) and at least one point cloud tile; for example, assuming the current sample is divided into M point cloud tiles, if the point cloud tile information field tile_info_present = 1, it indicates that there is an association relationship between the parameter set included in the current sample and at least one of the M point cloud tiles. When the value of the point cloud tile information field is a second set value (such as 0), it indicates that the association relationship between the parameter set included in the current sample and the point cloud tile is not indicated.

[0194] Specifically, during the process of generating parameter indication information, if there is an association relationship between the parameter set included in the current sample and at least one point cloud tile, the content production device sets the value of the point cloud tile information field corresponding to the current sample to the first set value; if the association relationship between the parameter set included in the current sample and the point cloud tile is not indicated, the content production device sets the value of the point cloud tile information field corresponding to the current sample to the second set value.

[0195] In one embodiment, during the encoding process, the content production device may divide a sample into M point cloud slices, where M is a positive integer; the M point cloud slices may form one or more point cloud slice groups. The parameter set sample group includes a point cloud slice group information field (slice_tag_info_present); when the value of the point cloud slice group information field is a first set value (such as 1), it indicates that there is an association relationship between the parameter set included in the current sample (i.e., the sample being encoded) and at least one point cloud slice group; for example, assume that the current sample is divided into M point cloud slices, and the M point cloud slices form N point cloud slice groups. If the slice_tag_info_present of the point cloud slice group information field = 1, it indicates that there is an association relationship between the parameter set included in the current sample and at least one of the N point cloud slice groups. When the value of the point cloud slice group information field is a second set value (such as 0), it indicates that the association relationship between the parameter set included in the current sample and the point cloud slice group is not indicated.

[0196] Specifically, during the process of generating parameter indication information, if there is an association relationship between the parameter set included in the current sample and at least one of one or more point cloud slice groups, the content production device sets the value of the point cloud slice group information field corresponding to the current sample to the first set value; if the association relationship between the parameter set included in the current sample and the point cloud slice group is not indicated, the content production device sets the value of the point cloud slice group information field corresponding to the current sample to the second set value.

[0197] In one embodiment, the parameter set sample group includes a parameter set quantity field (parameter_set_num); the parameter set quantity field is used to indicate the quantity of parameter sets included in the current sample (i.e., the sample being encoded).

[0198] Specifically, during the process of generating parameter indication information, the content production device sets the value of the parameter set quantity field according to the quantity of parameter sets included in the current sample; for example, if the quantity of parameter sets included in the current sample is 2, the content production device sets parameter_set_num = 2.

[0199] In one embodiment, the parameter set sample group includes a parameter set identifier field (parametetSetID); the parameter set identifier field is used to indicate the identifier of the parameter set included in the current sample (i.e., the sample being encoded).

[0200] Specifically, during the process of generating parameter indication information, the content production device sets the value of the parameter set identifier field according to the parameter set included in the current sample; for example, assume that the identifier of parameter set A is 1 and the identifier of parameter set B is 2. If the current sample includes parameter set B, the content production device sets parametetSetID = 2.

[0201] In one embodiment, during the encoding process, the content production device may divide a sample into one or more point cloud slices; the parameter set sample group includes a point cloud slice number field (associated_slice_num), and the point cloud slice number field is used to indicate the number of point cloud slices associated with the parameter set included in the current sample (i.e., the sample being encoded). The point cloud slices associated with the parameter set can be understood as the point cloud slices that need to use the parameter set during decoding.

[0202] Specifically, during the process of generating the parameter indication information, the content production device sets the value of the point cloud slice number field corresponding to the parameter set according to the number of point cloud slices associated with the parameter set included in the current sample; for example, assume that the current sample includes parameter set A and parameter set B. If the number of point cloud slices associated with parameter set A is 2, and the number of point cloud slices associated with parameter set B is 3, then the content production device sets the point cloud slice number field corresponding to parameter set A to associated_slice_num = 2, and sets the point cloud slice number field corresponding to parameter set B to associated_slice_num = 3. It can be understood that the above values are only for illustration and do not constitute an actual limitation of this application.

[0203] In one embodiment, during the encoding process, the content production device may divide a sample into one or more point cloud slices; the parameter set sample group includes a point cloud slice identifier field (slice_id), and the point cloud slice identifier field is used to indicate the identifier of the point cloud slice associated with the parameter set included in the current sample (i.e., the sample being encoded). The point cloud slices associated with the parameter set can be understood as the point cloud slices that need to use the parameter set during decoding.

[0204] Specifically, during the process of generating the parameter indication information, the content production device sets the value of the point cloud slice identifier field corresponding to the parameter set according to the point cloud slices associated with the parameter set included in the current sample; for example, assume that the current sample includes parameter set A and parameter set B, and the identifier of point cloud slice 1 is 1, and the identifier of point cloud slice 2 is 2; if point cloud slice 1 is associated with parameter set A, and point cloud slice 2 is associated with parameter set B, then the content production device sets the point cloud slice identifier field corresponding to parameter set A to slice_id = 1, and sets the point cloud slice identifier field corresponding to parameter set B to slice_id = 2. It can be understood that the above values are only for illustration and do not constitute an actual limitation of this application.

[0205] In one embodiment, during the encoding process, the content production device may divide a sample into one or more point cloud tiles; the parameter set sample group includes a point cloud tile quantity field (associated_tile_num), and the point cloud tile quantity field is used to indicate the number of point cloud tiles associated with the parameter set included in the current sample (i.e., the sample being encoded). The point cloud tiles associated with the parameter set can be understood as the point cloud tiles that need to use the parameter set during decoding.

[0206] Specifically, during the process of generating the parameter indication information, the content production device sets the value of the point cloud tile quantity field corresponding to the parameter set according to the number of point cloud tiles associated with the parameter set included in the current sample; for example, assume that the current sample includes parameter set A and parameter set B. If the number of point cloud tiles associated with parameter set A is 2, and the number of point cloud tiles associated with parameter set B is 3, then the content production device sets the point cloud tile quantity field corresponding to parameter set A to associated_tile_num = 2, and sets the point cloud tile quantity field corresponding to parameter set B to associated_tile_num = 3. It can be understood that the above values are only for illustration and do not constitute an actual limitation of this application.

[0207] In one embodiment, during the encoding process, the content production device may divide a sample into one or more point cloud tiles; the parameter set sample group includes a point cloud tile identifier field (tile_id), and the point cloud tile identifier field is used to indicate the identifier of the point cloud tile associated with the parameter set included in the current sample (i.e., the sample being encoded). The point cloud tiles associated with the parameter set can be understood as the point cloud tiles that need to use the parameter set during decoding.

[0208] Specifically, during the process of generating the parameter indication information, the content production device sets the value of the point cloud tile identifier field corresponding to the parameter set according to the point cloud tiles associated with the parameter set included in the current sample; for example, assume that the current sample includes parameter set A and parameter set B, the identifier of point cloud tile 1 is 1, and the identifier of point cloud tile 2 is 2; if point cloud tile 1 is associated with parameter set A, and point cloud tile 2 is associated with parameter set B, then the content production device sets the point cloud tile identifier field corresponding to parameter set A to tile_id = 1, and sets the point cloud tile identifier field corresponding to parameter set B to tile_id = 2. It can be understood that the above values are only for illustration and do not constitute an actual limitation of this application.

[0209] In one embodiment, during the encoding process, the content production device may divide a sample into M point cloud slices, where M is a positive integer; the M point cloud slices may form one or more point cloud slice groups. The parameter set sample group includes a point cloud slice group number field (associated_slice_tag_num), and the point cloud slice group number field is used to indicate the number of point cloud slice groups associated with the parameter set included in the current sample (i.e., the sample being encoded). The point cloud slice groups associated with the parameter set can be understood as the point cloud slice groups that need to use this parameter set during decoding.

[0210] Specifically, during the process of generating the parameter indication information, the content production device sets the value of the point cloud slice group number field corresponding to the parameter set according to the number of point cloud slice groups associated with the parameter set included in the current sample; for example, assume that the current sample includes parameter set A and parameter set B. If the number of point cloud slice groups associated with parameter set A is 2, and the number of point cloud slice groups associated with parameter set B is 3, then the content production device sets the point cloud slice group number field corresponding to parameter set A to associated_slice_tag_num = 2, and sets the point cloud slice group number field corresponding to parameter set B to associated_slice_tag_num = 3. It can be understood that the above values are only for illustration and do not constitute an actual limitation of this application.

[0211] In one embodiment, during the encoding process, the content production device may divide a sample into M point cloud slices, where M is a positive integer; the M point cloud slices may form one or more point cloud slice groups. The parameter set sample group includes a point cloud slice group identifier field (slice_tag), and the point cloud slice group identifier field is used to indicate the identifier of the point cloud slice group associated with the parameter set included in the current sample (i.e., the sample being encoded). The point cloud slice groups associated with the parameter set can be understood as the point cloud slice groups that need to use this parameter set during decoding.

[0212] Specifically, during the process of generating the parameter indication information, the content production device sets the value of the point cloud slice group identifier field corresponding to the parameter set according to the point cloud slice groups associated with the parameter set included in the current sample; for example, assume that the current sample includes parameter set A and parameter set B, the identifier of point cloud slice group 1 is 1, and the identifier of point cloud slice group 2 is 2; if point cloud slice group 1 is associated with parameter set A, and point cloud slice group 2 is associated with parameter set B, then the content production device sets the point cloud slice group identifier field corresponding to parameter set A to slice_tag = 1, and sets the point cloud slice group identifier field corresponding to parameter set B to slice_tag = 2. It can be understood that the above values are only for illustration and do not constitute an actual limitation of this application.

[0213] In another embodiment, the parameter indication information includes a decoding configuration record (GPCCDecoderConfigurationRecord). The decoding configuration record does not include the parameter set number field (parameter_set_num) in the parameter set information entry (GPCCParameterSetInfoEntry). For specific embodiments of the remaining extension fields, reference may be made to the extension fields in the parameter set information entry and the description in Table 3 above, which will not be elaborated here.

[0214] For example, the decoding configuration record includes a parameter set identifier field (parameter_set_id_present); when the value of the parameter set identifier field is a first set value (such as 1), it indicates that the parameter set identifier of the parameter set included in the current sample (i.e., the sample being encoded) is indicated in the decoding configuration record; when the value of the parameter set identifier field is a second set value (such as 0), it indicates that the parameter set identifier of the parameter set included in the current sample is not indicated in the decoding configuration record. When the value of the parameter set identifier field is the second set value, the bitstream data does not include other parameter sets that belong to the same type as the parameter set included in the current sample and have different parameter set identifiers; it can be understood that the bitstream data may include N parameter sets that belong to the same type as the parameter set included in the current sample and have the same parameter set identifier, where N is 0 or a positive integer. For the content production device, the current sample refers to the sample being encoded, and for the content consumption device, the current sample refers to the sample being decoded.

[0215] Specifically, during the process of generating the parameter indication information by the content production device, if the parameter set identifier of the parameter set included in the current sample is indicated in the decoding configuration record, the content production device sets the value of the parameter set identifier field corresponding to the current sample to the first set value; if the parameter set identifier of the parameter set included in the current sample is not indicated in the decoding configuration record, the content production device determines that the bitstream data does not include other parameter sets that belong to the same type as the parameter set included in the current sample and have different parameter set identifiers, and sets the value of the parameter set identifier field corresponding to the current sample to the second set value.

[0216] S303. Package the bitstream data and the parameter indication information to obtain a media file of the point cloud media.

[0217] The specific implementation manner of the content production device packaging the bitstream data and the parameter indication information to obtain a media file of the point cloud media can be referred to Figure 1d the implementation manner in (2) therein, which will not be elaborated here.

[0218] In one implementation, the content production device may also generate a transmission signaling file for point cloud media. The transmission signaling file includes a representation level and an adaptation set level. The adaptation set level includes one or more representation levels. When the description information exists in the target adaptation set level, it is used to describe all the representation levels in the target adaptation set level. When the description information exists in the target representation level, it is used to describe the target representation level.

[0219] The following uses two complete embodiments to detail the data processing method for point cloud media provided in this application.

[0220] In one embodiment, the content production device acquires point cloud data and obtains bitstream data after encoding the point cloud data. When encapsulating the bitstream data into a file, for a point cloud frame containing a parameter set, the content production device identifies the samples corresponding to the point cloud frame containing the parameter set and marks them as reference samples (i.e., samples containing the parameter set) in the corresponding sample group, and:

[0221] a) Indicate the type of the parameter set included in the reference sample;

[0222] b) Indicate the identifier of the parameter set included in the reference sample.

[0223] Figure 4a For a sample schematic diagram provided in an embodiment of this application, as Figure 4a shown, Sample 1 contains Parameter Set 1 (GPS1) of geometric type and Parameter Set 2 (GPS1) of geometric type. Sample 10 contains Parameter Set 1 (GPS1) of geometric type. The GPS1 in Sample 10 is the parameter set updated from the GPS1 in Sample 1. Similarly, Sample 20 contains Parameter Set 2 (GPS2) of geometric type. The GPS2 in Sample 20 is the parameter set updated from the GPS2 in Sample 1. It should be noted that for samples after Sample 10, if GPS1 is needed during decoding, the GPS1 in Sample 1 will not be referenced. Similarly, for samples after Sample 20, if GPS2 is needed during decoding, the GPS2 in Sample 1 will not be referenced.

[0224] According to the Figure 4a samples, the content production device can generate the following parameter indication information:

[0225] Sample1: {parametersetType = 2; parameter_set_id_present = 1; parameter_set_num = 2; parametetSetID = {1,2}}

[0226] Sample10: {parametersetType = 2; parameter_set_id_present = 1; parameter_set_num = 1; parametetSetID = {1}}

[0227] Sample20: {parametersetType = 2; parameter_set_id_present = 1; parameter_set_num = 1; parametetSetID = {2}}

[0228] Among them, in Sample1, parametersetType = 2 indicates that the type of the parameter set included in Sample1 is a geometric type; parameter_set_id_present = 1 indicates the parameter set identifier in the parameter indication information that indicates the parameter set (i.e., GPS1 and GPS2) included in Sample1; parameter_set_num = 2 indicates that the number of parameter sets included in Sample1 is 2; parametetSetID = {1, 2} indicates that the identifiers of the parameter sets included in Sample1 are "1" and "2" respectively; similarly, in Sample10, parametersetType = 2 indicates that the type of the parameter set included in Sample10 is a geometric type; parameter_set_id_present = 1 indicates the parameter set identifier in the parameter indication information that indicates the parameter set (i.e., GPS1) included in Sample10; parameter_set_num = 1 indicates that the number of parameter sets included in Sample10 is 1; parametetSetID = {1} indicates that the identifier of the parameter set included in Sample10 is "1". In Sample20, parametersetType = 2 indicates that the type of the parameter set included in Sample20 is a geometric type; parameter_set_id_present = 1 indicates the parameter set identifier in the parameter indication information that indicates the parameter set (i.e., GPS2) included in Sample20; parameter_set_num = 1 indicates that the number of parameter sets included in Sample20 is 1; parametetSetID = {2} indicates that the identifier of the parameter set included in Sample20 is "2".

[0229] Further, a content production device (such as a server) transmits the point cloud file to a content consumption device (such as a client). When the content production device unpacks and decodes the point cloud file / file segment, according to the above parameter indication information, it can obtain the set of parameters required for decoding when decoding the samples. Specifically, when decoding Sample 1 to Sample 9, the set of parameters in sample1 needs to be referred to; when decoding sample10 to sample19, the sets of parameters in sample1 and sample10 need to be referred to; when decoding sample20 and subsequent samples, the sets of parameters in sample10 and sample20 need to be referred to.

[0230] In another embodiment, the content production device collects point cloud data and obtains bitstream data after encoding the point cloud data. When encapsulating the bitstream data into a file, for the point cloud frame containing the set of parameters, the content production device identifies the samples corresponding to the point cloud frame containing the set of parameters and marks them as reference samples (i.e., samples containing the set of parameters) in the corresponding sample group, and:

[0231] a) Indicates the type of the set of parameters included in the reference sample;

[0232] b) Indicates the identifier of the set of parameters included in the reference sample.

[0233] Figure 4b A schematic diagram of point cloud segmentation provided by an embodiment of the present application is as Figure 4b shown. The samples corresponding to the point cloud frame are encapsulated in the point cloud segmentation base track, point cloud segmentation track 1, and point cloud segmentation track 2. In the file mode based on point cloud segmentation, all the sets of parameters are placed in the point cloud segmentation base track, and the point cloud frame data is placed in point cloud segmentation track 1 and point cloud segmentation track 2. In the point cloud segmentation base track: Sample 1 contains the geometric type parameter set 1 (GPS1-tile1) and the geometric type parameter set 2 (GPS1-tile2), Sample 10 contains the geometric type parameter set 1 (GPS1-tile1), and GPS1-tile1 in Sample 10 is the updated parameter set of GPS1-tile1 in Sample 1; similarly, Sample 20 contains the geometric type parameter set 2 (GPS2-tile2), and GPS2-tile2 in Sample 20 is the updated parameter set of GPS2-tile2 in Sample 1.

[0234] According to Figure 4b the samples in the point cloud segmentation base track in

[0235] Sample1: {parametersetType = 2; parameter_set_id_present = 1; parameter_set_num = 2; tile_info_present = 1; {parametetSetID = 1; associated_tile_num = 1; tile_id = 1}; {parametetSetID = 2; associated_tile_num = 1; tile_id = 2}}

[0236] Sample10: {parametersetType = 2; parameter_set_id_present = 1; parameter_set_num = 1; tile_info_present = 1; {parametetSetID = 1; associated_tile_num = 1; tile_id = 1}}

[0237] Sample20: {parametersetType = 2; parameter_set_id_present = 1; parameter_set_num = 1; tile_info_present = 1; {parametetSetID = 2; associated_tile_num = 1; tile_id = 2}}

[0238] Among them, in Sample1, parametersetType = 2 indicates that the type of the parameter set included in Sample 1 is a geometric type; parameter_set_id_present = 1 indicates that the parameter set identifier of the parameter set included in Sample 1 (i.e., GPS1-tile1 and GPS2-tile2) is indicated in the parameter indication information; parameter_set_num = 2 indicates that the number of parameter sets included in Sample 1 is 2; tile_info_present = 1 indicates that there is an association relationship between the parameter sets included in Sample 1 and the point cloud tiles; {parametetSetID = 1; associated_tile_num = 1; tile_id = 1} indicates that the number of point cloud blocks associated with the parameter set with parameter set identifier 1 is 1, and the identifier of the point cloud block associated with this parameter set is 1; similarly, {parametetSetID = 2; associated_tile_num = 1; tile_id = 2} indicates that the number of point cloud blocks associated with the parameter set with parameter set identifier 2 is 1, and the identifier of the point cloud block associated with this parameter set is 2.

[0239] Similarly, in Sample10, parametersetType = 2, indicating that the type of the parameter set included in Sample10 is a geometric type; parameter_set_id_present = 1, indicating that the parameter set identifier of the parameter set included in Sample10 (i.e., GPS1-tile1) is indicated in the parameter indication information; GPS1-tile1 included in Sample10 can be an updated parameter set of GPS1-tile1 included in Sample1; parameter_set_num = 1, indicating that the number of parameter sets included in Sample10 is 1; tile_info_present = 1, indicating that there is an association relationship between the parameter set included in Sample10 and the point cloud tile; {parametetSetID = 1; associated_tile_num = 1; tile_id = 1} indicates that the number of point cloud tiles associated with the parameter set with parameter set identifier 1 is 1, and the identifier of the point cloud tile associated with this parameter set is 1. In Sample20, parametersetType = 2, indicating that the type of the parameter set included in Sample20 is a geometric type; parameter_set_id_present = 1, indicating that the parameter set identifier of the parameter set included in Sample20 (i.e., GPS2-tile2) is indicated in the parameter indication information; GPS2-tile2 included in Sample20 can be an updated parameter set of GPS2-tile2 included in Sample1; parameter_set_num = 1, indicating that the number of parameter sets included in Sample20 is 1; tile_info_present = 1, indicating that there is an association relationship between the parameter set included in Sample20 and the point cloud tile; {parametetSetID = 2; associated_tile_num = 1; tile_id = 2} indicates that the number of point cloud tiles associated with the parameter set with parameter set identifier 2 is 1, and the identifier of the point cloud tile associated with this parameter set is 2.

[0240] Further, a content production device (such as a server) transmits the point cloud file to a content consumption device (such as a client). When the content production device unpacks and decodes the point cloud file / file segment, according to the above parameter indication information, a set of parameters required for decoding can be obtained when decoding the samples. Specifically, when decoding the point cloud tile track 1 (tile track1), the content consumption device only needs to refer to the set of parameters associated with the point cloud tile track 1 (tile track1) (i.e., GPS1-tile1). Therefore, when the content consumption device decodes samples 1 (sample1) to sample 9 (sample9) of the cloud tile track 1 (tile track1), it refers to GPS1-tile1 in sample 1 (sample1) of the point cloud tile base track (tile basetrack). When decoding sample 10 (sample10) and subsequent samples of the point cloud tile track 1 (tile track1), it refers to GPS1-tile1 in sample 10 (sample10) of the point cloud tile base track (tile basetrack). Similarly, when decoding the point cloud tile track 2 (tile track2), the content consumption device only needs to refer to the set of parameters associated with the point cloud tile track 2 (tile track2) (i.e., GPS2-tile2). Therefore, when the content consumption device decodes samples 1 (sample1) to sample 9 (sample9) of the cloud tile track 2 (tile track2), it refers to GPS2-tile2 in sample 1 (sample1) of the point cloud tile base track (tile basetrack). When decoding sample 10 (sample10) and subsequent samples of the point cloud tile track 2 (tile track2), it refers to GPS2-tile2 in sample 10 (sample10) of the point cloud tile base track (tile basetrack).

[0241] In the embodiments of the present application, a media file of the point cloud media is obtained. The media file includes the bitstream data of the point cloud media and parameter indication information. The bitstream data contains at least two parameter sets of the same type. The parameter indication information is used to indicate the association relationship between the at least two parameter sets and the point cloud frames in the point cloud media. The bitstream data is decoded according to the parameter indication information to present the point cloud media. It can be seen that the association relationship between the parameter set and the point cloud frames in the point cloud media can be described through the parameter indication information (such as indicating the set of parameters required for decoding the point cloud frame currently being decoded), so as to facilitate the parsing and presentation of the point cloud media.

[0242] The above details the method of the embodiments of the present application. To facilitate better implementation of the above solutions of the embodiments of the present application, correspondingly, the device of the embodiments of the present application is provided below.

[0243] Please refer to Figure 5 , Figure 5 , which is a schematic structural diagram of a data processing device for point cloud media provided by an embodiment of the present application; the data processing device for point cloud media may be a computer program (including program code) running in a content consumption device. For example, the data processing device for point cloud media may be an application software in a content consumption device. As Figure 5 shown, the data processing device for point cloud media includes an acquisition unit 501 and a processing unit 502.

[0244] Please refer to Figure 5 , in an exemplary embodiment, the detailed description of each unit is as follows:

[0245] The acquisition unit 501 is configured to acquire a media file of point cloud media, where the media file includes bitstream data of point cloud media and parameter indication information; the bitstream data includes at least two parameter sets of the same type; the parameter indication information is used to indicate the association relationship between the at least two parameter sets and the point cloud frames in the point cloud media;

[0246] The processing unit 502 is configured to decode the bitstream data according to the parameter indication information to present the point cloud media.

[0247] In an implementation manner, the media file includes tracks, and one or more samples are included in the tracks, and one sample corresponds to one point cloud frame of the point cloud media;

[0248] One or more reference samples are included in the tracks, and the reference samples refer to the samples including parameter sets;

[0249] The parameter indication information is used to indicate the reference samples that one or more samples in the media file need to refer to during decoding.

[0250] In an implementation manner, the parameter indication information includes at least one of a parameter set sample group and a decoding configuration record; the parameter set sample group is used to indicate the parameter sets that one or more samples in the media file need during decoding, and the decoding configuration record is used to indicate the initialization parameter information required for decoding the bitstream data.

[0251] In an implementation manner, the parameter indication information includes a parameter set identification field; the processing unit 502 is configured to decode the bitstream data according to the parameter indication information to present the point cloud media, and specifically:

[0252] If the value of the parameter set identification field is a first set value, determine the parameter set identifier that indicates the parameter set included in the current sample in the parameter indication information;

[0253] If the value of the parameter set identification field is the second set value, determine the parameter set identifier in the parameter indication information that does not indicate the parameter set included in the current sample, and the bitstream data does not include other parameter sets that belong to the same type as the parameter set included in the current sample and have different parameter set identifiers;

[0254] Wherein, the current sample refers to the sample being decoded.

[0255] In one implementation, a sample is divided into one or more point cloud patches; the parameter indication information includes a point cloud patch information field; the processing unit 502 is configured to decode the bitstream data according to the parameter indication information to present point cloud media, specifically:

[0256] If the value of the point cloud patch information field is the first set value, determine that there is an association relationship between the parameter set included in the current sample and at least one point cloud patch;

[0257] If the value of the point cloud patch information field is the second set value, determine that the association relationship between the parameter set included in the current sample and one or more point cloud patches is not indicated;

[0258] Wherein, the current sample refers to the sample being decoded.

[0259] In one implementation, a sample is divided into one or more point cloud blocks; the parameter indication information includes a point cloud block information field; the processing unit 502 is configured to decode the bitstream data according to the parameter indication information to present point cloud media, specifically:

[0260] If the value of the point cloud block information field is the first set value, determine that there is an association relationship between the parameter set included in the current sample and at least one point cloud block;

[0261] If the value of the point cloud block information field is the second set value, determine that the association relationship between the parameter set included in the current sample and one or more point cloud blocks is not indicated;

[0262] Wherein, the current sample refers to the sample being decoded.

[0263] In one implementation, a sample is divided into M point cloud patches, where M is a positive integer; one or more of the M point cloud patches form a point cloud patch group; the parameter indication information includes a point cloud patch group information field; the processing unit 502 is configured to decode the bitstream data according to the parameter indication information to present point cloud media, specifically:

[0264] If the value of the point cloud patch group information field is the first set value, determine that there is an association relationship between the parameter set included in the current sample and at least one point cloud patch group;

[0265] If the value of the point cloud slice group information field is the second set value, it is determined that the association relationship between the parameter set included in the current sample and one or more point cloud slice groups is not indicated;

[0266] Wherein, the current sample refers to the sample being decoded.

[0267] In one embodiment, the parameter indication information includes a parameter set identification field; the processing unit 502 is configured to decode the bitstream data according to the parameter indication information to present point cloud media, specifically:

[0268] Determine the parameter set included in the current sample according to the parameter set identification field;

[0269] Wherein, the current sample refers to the sample being decoded.

[0270] In one embodiment, a sample is divided into one or more point cloud slices; the parameter indication information includes a point cloud slice quantity field and a point cloud slice identification field; the processing unit 502 is configured to decode the bitstream data according to the parameter indication information to present point cloud media, specifically:

[0271] Determine the number of point cloud slices associated with the parameter set corresponding to this field according to the point cloud slice quantity field;

[0272] Determine the point cloud slices associated with the parameter set corresponding to this field according to the point cloud slice identification field.

[0273] In one embodiment, a sample is divided into one or more point cloud blocks; the parameter indication information includes a point cloud block quantity field and a point cloud block identification field; the processing unit 502 is configured to decode the bitstream data according to the parameter indication information to present point cloud media, specifically:

[0274] Determine the number of point cloud blocks associated with the parameter set corresponding to this field according to the point cloud block quantity field;

[0275] Determine the point cloud blocks associated with the parameter set corresponding to this field according to the point cloud block identification field.

[0276] In one embodiment, a sample is divided into M point cloud slices, where M is a positive integer; one or more of the M point cloud slices form a point cloud slice group; the parameter indication information includes a point cloud slice group quantity field and a point cloud slice group identification field; the processing unit 502 is configured to decode the bitstream data according to the parameter indication information to present point cloud media, specifically:

[0277] Determine the number of point cloud slice groups associated with the parameter set corresponding to this field according to the point cloud slice group quantity field;

[0278] Determine the point cloud patch group associated with the parameter set corresponding to the point cloud patch group identification field.

[0279] In one embodiment, the parameter indication information includes a parameter sample set group, and the parameter sample set group includes a parameter set quantity field; the processing unit 502 is configured to decode the bitstream data according to the parameter indication information to present the point cloud media, specifically:

[0280] Determine the number of parameter sets included in the current sample according to the parameter set quantity field;

[0281] Wherein, the current sample refers to the sample being decoded.

[0282] According to an embodiment of the present application, Figure 2 Some of the steps involved in the data processing method of the point cloud media shown can be performed by Figure 5 each unit in the data processing device of the point cloud media shown. For example, Figure 2 the step S201 shown in Figure 5 can be performed by the acquisition unit 501 shown, and the step S202 can be performed by Figure 5 the processing unit 502 shown. Figure 5 Each unit in the data processing device of the point cloud media shown can be separately or all combined into one or several other units to form, or a certain one (or some) of the units can be further split into multiple smaller units in terms of function to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above units are divided based on logical functions. In actual applications, the function of one unit can also be realized by multiple units, or the functions of multiple units can be realized by one unit. In other embodiments of the present application, the data processing device of the point cloud media may also include other units. In actual applications, these functions can also be assisted by other units and can be realized by the cooperation of multiple units.

[0283] According to another embodiment of the present application, it is possible to construct, by running a computer program (including program code) capable of executing the respective steps involved in the corresponding method shown in Figure 2 on a general computing device such as a computer including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM), Figure 5 the data processing device of the point cloud media shown, and to implement the data processing method of the point cloud media of the embodiments of the present application. The computer program can be recorded on, for example, a computer-readable recording medium, and be loaded into the above computing device through the computer-readable recording medium and run therein.

[0284] Based on the same inventive concept, the principle of problem-solving and the beneficial effects of the point cloud media data processing device provided in the embodiments of the present application are similar to those of the point cloud media data processing method in the method embodiments of the present application. You can refer to the principle and beneficial effects of the method implementation. For the sake of brevity, they will not be elaborated here.

[0285] Please refer to Figure 6 , Figure 6 FIG. [FIGURE NUMBER] is a schematic structural diagram of another point cloud media data processing device provided in the embodiments of the present application; the point cloud media data processing device may be a computer program (including program code) running in a content production device. For example, the point cloud media data processing device may be an application software in the content production device. As Figure 6 shown, the point cloud media data processing device includes an acquisition unit 601 and a processing unit 602. Please refer to Figure 6 , and the detailed description of each unit is as follows:

[0286] The acquisition unit 601 is configured to acquire the point cloud data of the point cloud media, and perform encoding processing on the point cloud data to obtain the bitstream data of the point cloud media. The bitstream data of the point cloud media includes at least two parameter sets of the same type;

[0287] The processing unit 602 is configured to generate parameter indication information according to the association relationship between the point cloud frames and at least two parameter sets in the point cloud media;

[0288] and is configured to encapsulate the bitstream data and the parameter indication information to obtain the media file of the point cloud media.

[0289] In one implementation, the media file includes tracks, and each track includes one or more samples. One sample corresponds to one point cloud frame of the point cloud media;

[0290] Each track includes one or more reference samples, and a reference sample refers to a sample that includes a parameter set;

[0291] The parameter indication information is used to indicate one or more samples in the media file that need to refer to the reference samples during decoding.

[0292] In one implementation, the parameter indication information includes at least one of a parameter set sample group and a decoding configuration record; the parameter set sample group is used to indicate the parameter sets required by one or more samples in the media file during decoding, and the decoding configuration record is used to indicate the initialization parameter information required by the bitstream data during decoding.

[0293] In one implementation, the parameter indication information includes a parameter set identification field;

[0294] It should be noted that the specific figure number in "FIG. [FIGURE NUMBER]" needs to be filled in according to the actual figure number in the original text. Since it is not provided in the given content, it is left as "FIG. [FIGURE NUMBER]" here.When the value of the parameter set identification field is the first set value, it indicates the parameter set identifier in the parameter indication information that indicates the parameter set included in the current sample;

[0295] When the value of the parameter set identification field is the second set value, it indicates the parameter set identifier in the parameter indication information that does not indicate the parameter set included in the current sample, and the bitstream data does not include a parameter set that belongs to the same type as the parameter set included in the current sample and has a different parameter set identifier;

[0296] Wherein, the current sample refers to the sample being encoded.

[0297] In one embodiment, a sample is divided into one or more point cloud patches; the parameter indication information includes a point cloud patch information field;

[0298] When the value of the point cloud patch information field is the first set value, it indicates that there is an association relationship between the parameter set included in the current sample and at least one point cloud patch;

[0299] When the value of the point cloud patch information field is the second set value, it indicates that the association relationship between the parameter set included in the current sample and one or more point cloud patches is not indicated;

[0300] Wherein, the current sample refers to the sample being encoded.

[0301] In one embodiment, a sample is divided into one or more point cloud blocks; the parameter indication information includes a point cloud block information field;

[0302] When the value of the point cloud block information field is the first set value, it indicates that there is an association relationship between the parameter set included in the current sample and at least one point cloud block;

[0303] When the value of the point cloud block information field is the second set value, it indicates that the association relationship between the parameter set included in the current sample and one or more point cloud blocks is not indicated;

[0304] Wherein, the current sample refers to the sample being encoded.

[0305] In one embodiment, a sample is divided into M point cloud patches, where M is a positive integer; one or more of the M point cloud patches form a point cloud patch group; the parameter indication information includes a point cloud patch group information field;

[0306] When the value of the point cloud patch group information field is the first set value, it indicates that there is an association relationship between the parameter set included in the current sample and at least one point cloud patch group;

[0307] When the value of the point cloud patch group information field is the second set value, it indicates that the association relationship between the parameter set included in the current sample and one or more point cloud patch groups is not indicated;

[0308] Wherein, the current sample refers to the sample being encoded.

[0309] In one embodiment, the parameter indication information includes a parameter set identification field; the parameter set identification field is used to indicate the identifier of the parameter set included in the current sample;

[0310] Wherein, the current sample refers to the sample being encoded.

[0311] In one embodiment, a sample is divided into one or more point cloud patches; the parameter indication information includes a point cloud patch quantity field and a point cloud patch identification field;

[0312] The point cloud patch quantity field is used to indicate the quantity of the point cloud patches associated with the parameter set included in the current sample;

[0313] The point cloud patch identification field is used to indicate the identifier of the point cloud patches associated with the parameter set included in the current sample;

[0314] Wherein, the current sample refers to the sample being encoded.

[0315] In one embodiment, a sample is divided into one or more point cloud blocks; the parameter indication information includes a point cloud block quantity field and a point cloud block identification field;

[0316] The point cloud block quantity field is used to indicate the quantity of the point cloud blocks associated with the parameter set included in the current sample;

[0317] The point cloud block identification field is used to indicate the identifier of the point cloud blocks associated with the parameter set included in the current sample;

[0318] Wherein, the current sample refers to the sample being encoded.

[0319] In one embodiment, a sample is divided into M point cloud patches, where M is a positive integer; one or more of the M point cloud patches form a point cloud patch group; the parameter indication information includes a point cloud patch group quantity field and a point cloud patch group identification field;

[0320] The point cloud patch group quantity field is used to indicate the quantity of the point cloud patch groups associated with the parameter set included in the current sample;

[0321] The point cloud patch group identification field is used to indicate the identifier of the point cloud patch groups associated with the parameter set included in the current sample;

[0322] Wherein, the current sample refers to the sample being encoded.

[0323] In one embodiment, the parameter indication information includes a parameter set sample group, and the parameter set sample group includes a parameter set quantity field; the parameter set quantity field is used to indicate the number of parameter sets included in the current sample;

[0324] wherein, the current sample refers to the sample being encoded.

[0325] According to an embodiment of the present application, Figure 3 some of the steps involved in the data processing method of the point cloud media shown can be performed by Figure 6 each unit in the data processing device of the point cloud media shown. For example, Figure 3 the step S301 shown in Figure 6 can be performed by the obtaining unit 601 shown, and the steps S302 and S303 can be performed by Figure 6 the processing unit 602 shown. Figure 6 Each unit in the data processing device of the point cloud media shown can be separately or all combined into one or several other units to form, or some of the units can be further split into multiple smaller units in terms of function to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above units are divided based on logical functions. In practical applications, the function of one unit can also be realized by multiple units, or the functions of multiple units can be realized by one unit. In other embodiments of the present application, the data processing device of the point cloud media may also include other units. In practical applications, these functions can also be assisted by other units and can be realized by the cooperation of multiple units.

[0326] According to another embodiment of the present application, it can be achieved by running a computer program (including program code) capable of executing the respective steps involved in the corresponding method shown in Figure 3 on a general computing device such as a computer including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM), to construct the data processing device of the point cloud media shown in Figure 6 and to implement the data processing method of the point cloud media in the embodiments of the present application. The computer program can be recorded on, for example, a computer-readable recording medium, and be loaded into the above computing device through the computer-readable recording medium and run therein.

[0327] Based on the same inventive concept, the principle of solving problems and the beneficial effects of the data processing device of the point cloud media provided in the embodiments of the present application are similar to the principle of solving problems and the beneficial effects of the data processing method of the point cloud media in the method embodiments of the present application. The principle and beneficial effects of the method implementation can be referred to. For the sake of brevity, they will not be elaborated here.

[0328] Figure 7 This is a schematic structural diagram of a content consumption device provided by an embodiment of the present application; the content consumption device may be a computer device used by a user of point cloud media, and the computer device may be a terminal (such as a PC, a smart mobile device (such as a smart phone), a VR device (such as a VR helmet, VR glasses, etc.)). As Figure 7 shown, the content consumption device includes a receiver 701, a processor 702, a memory 703, and a display / playback device 704. Among them:

[0329] The receiver 701 is used to implement decoding and transmission interaction with other devices, and specifically used to implement the transmission of point cloud media between the content production device and the content consumption device. That is, the content consumption device receives relevant media resources of the point cloud media transmitted by the content production device through the receiver 701.

[0330] The processor 702 (or CPU (Central Processing Unit)) is the processing core of the content production device. The processor 702 is adapted to implement one or more program instructions, and specifically adapted to load and execute one or more program instructions to implement Figure 2 the process of the data processing method of the point cloud media shown.

[0331] The memory 703 is a memory device in the content consumption device, used to store programs and media resources. It can be understood that the memory 703 here can include both the built-in storage medium in the content consumption device, and of course can also include the extended storage medium supported by the content consumption device. It should be noted that the memory 703 can be a high-speed RAM memory, or a non-volatile memory, such as at least one disk memory; optionally, it can also be at least one memory located far from the aforementioned processor. The memory 703 provides a storage space, which is used to store the operating system of the content consumption device. And, in this storage space, it is also used to store a computer program, the computer program includes program instructions, and the program instructions are adapted to be called and executed by the processor to execute the various steps of the data processing method of the point cloud media. In addition, the memory 703 can also be used to store the three-dimensional image of the point cloud media formed after being processed by the processor, the audio content corresponding to the three-dimensional image, and the information required for rendering the three-dimensional image and audio content, etc.

[0332] The display / playback device 704 is used to output the rendered sound and three-dimensional image.

[0333] Please refer to Figure 7 again, the processor 702 may include a parser 721, a decoder 722, a converter 723, and a renderer 724; among them:

[0334] The parser 721 is used to unpack the encapsulated file of the rendered media from the content production device. Specifically, it unpacks the media file resources according to the file format requirements of the point cloud media to obtain the audio bitstream and the video bitstream, and provides the audio bitstream and the video bitstream to the decoder 722.

[0335] The decoder 722 performs audio decoding on the audio bitstream to obtain the audio content and provides it to the renderer for audio rendering. In addition, the decoder 722 decodes the video bitstream to obtain a 2D image. According to the metadata provided by the media presentation description information, if the metadata indicates that the point cloud media has undergone a region encapsulation process, the 2D image refers to the encapsulated image; if the metadata indicates that the point cloud media has not undergone a region encapsulation process, the planar image refers to the projection image.

[0336] The converter 723 is used to convert the 2D image into a 3D image. If the point cloud media has undergone a region encapsulation process, the converter 723 will first perform region unpacking on the encapsulated image to obtain the projection image, and then perform reconstruction processing on the projection image to obtain the 3D image. If the rendered media has not undergone a region encapsulation process, the converter 723 will directly reconstruct the projection image to obtain the 3D image.

[0337] The renderer 724 is used to render the audio content and the 3D image of the point cloud media. Specifically, it renders the audio content and the 3D image according to the metadata related to rendering and the window in the media presentation description information, and after the rendering is completed, it is handed over to the display / playback device for output.

[0338] In an exemplary embodiment, the processor 702 (specifically, each device included in the processor) executes Figure 2 each step of the data processing method of the point cloud media shown. Specifically, the memory stores one or more first instructions, and the one or more first instructions are adapted to be loaded and executed by the processor 702 to perform the following steps:

[0339] Obtain the media file of the point cloud media, where the media file includes the bitstream data and parameter indication information of the point cloud media; the bitstream data contains at least two parameter sets of the same type; the parameter indication information is used to indicate the association relationship between the at least two parameter sets and the point cloud frames in the point cloud media;

[0340] Decode the bitstream data according to the parameter indication information to present the point cloud media.

[0341] In one implementation, the media file includes tracks, and one or more samples are included in the tracks, and one sample corresponds to one point cloud frame of the point cloud media;

[0342] The track contains one or more reference samples, where a reference sample refers to a sample containing a set of parameters;

[0343] The parameter indication information is used to indicate one or more samples in the media file that need to refer to the reference samples during decoding.

[0344] In one implementation, the parameter indication information includes at least one of a set of parameter samples group and a decoding configuration record; the set of parameter samples group is used to indicate the set of parameters required for one or more samples in the media file during decoding, and the decoding configuration record is used to indicate the initialization parameter information required for the bitstream data during decoding.

[0345] In one implementation, the parameter indication information includes a parameter set identification field; a specific example of the processor 702 decoding the bitstream data according to the parameter indication information to present the point cloud media is as follows:

[0346] If the value of the parameter set identification field is the first set value, then determine the parameter set identifier in the parameter indication information that indicates the set of parameters included in the current sample;

[0347] If the value of the parameter set identification field is the second set value, then determine the parameter set identifier in the parameter indication information that does not indicate the set of parameters included in the current sample, and the bitstream data does not contain other parameter sets that belong to the same type as the set of parameters included in the current sample and have different parameter set identifiers;

[0348] Herein, the current sample refers to the sample being decoded.

[0349] In one implementation, a sample is divided into one or more point cloud patches; the parameter indication information includes a point cloud patch information field; a specific example of the processor 702 decoding the bitstream data according to the parameter indication information to present the point cloud media is as follows:

[0350] If the value of the point cloud patch information field is the first set value, then determine that there is an association relationship between the set of parameters included in the current sample and at least one point cloud patch;

[0351] If the value of the point cloud patch information field is the second set value, then determine that the association relationship between the set of parameters included in the current sample and one or more point cloud patches is not indicated;

[0352] Herein, the current sample refers to the sample being decoded.

[0353] In one implementation, a sample is divided into one or more point cloud blocks; the parameter indication information includes a point cloud block information field; a specific example of the processor 702 decoding the bitstream data according to the parameter indication information to present the point cloud media is as follows:

[0354] If the value of the point cloud block information field is the first set value, it is determined that there is an association relationship between the parameter set included in the current sample and at least one point cloud block;

[0355] If the value of the point cloud block information field is the second set value, it is determined that the association relationship between the parameter set included in the current sample and one or more point cloud blocks is not indicated;

[0356] Wherein, the current sample refers to the sample being decoded.

[0357] In one implementation, a sample is divided into M point cloud slices, where M is a positive integer; one or more of the M point cloud slices form a point cloud slice group; the parameter indication information includes a point cloud slice group information field; a specific embodiment in which the processor 702 decodes the bitstream data according to the parameter indication information to present the point cloud media is as follows:

[0358] If the value of the point cloud slice group information field is the first set value, it is determined that there is an association relationship between the parameter set included in the current sample and at least one point cloud slice group;

[0359] If the value of the point cloud slice group information field is the second set value, it is determined that the association relationship between the parameter set included in the current sample and one or more point cloud slice groups is not indicated;

[0360] Wherein, the current sample refers to the sample being decoded.

[0361] In one implementation, the parameter indication information includes a parameter set identification field; a specific embodiment in which the processor 702 decodes the bitstream data according to the parameter indication information to present the point cloud media is as follows:

[0362] According to the parameter set identification field, determine the parameter set included in the current sample;

[0363] Wherein, the current sample refers to the sample being decoded.

[0364] In one implementation, a sample is divided into one or more point cloud slices; the parameter indication information includes a point cloud slice quantity field and a point cloud slice identification field; a specific embodiment in which the processor 702 decodes the bitstream data according to the parameter indication information to present the point cloud media is as follows:

[0365] According to the point cloud slice quantity field, determine the quantity of the point cloud slices associated with the parameter set corresponding to this field;

[0366] According to the point cloud slice identification field, determine the point cloud slices associated with the parameter set corresponding to this field.

[0367] In one implementation, a sample is divided into one or more point cloud blocks; the parameter indication information includes a point cloud block quantity field and a point cloud block identification field; a specific embodiment in which the processor 702 decodes the bitstream data according to the parameter indication information to present the point cloud media is as follows:

[0368] Determine the quantity of the point cloud blocks associated with the parameter set corresponding to this field according to the point cloud block quantity field;

[0369] Determine the point cloud blocks associated with the parameter set corresponding to this field according to the point cloud block identification field.

[0370] In one implementation, a sample is divided into M point cloud slices, where M is a positive integer; one or more of the M point cloud slices form a point cloud slice group; the parameter indication information includes a point cloud slice group quantity field and a point cloud slice group identification field; a specific embodiment in which the processor 702 decodes the bitstream data according to the parameter indication information to present the point cloud media is as follows:

[0371] Determine the quantity of the point cloud slice groups associated with the parameter set corresponding to this field according to the point cloud slice group quantity field;

[0372] Determine the point cloud slice groups associated with the parameter set corresponding to this field according to the point cloud slice group identification field.

[0373] In one implementation, the parameter indication information includes a parameter sample set group, and the parameter sample set group includes a parameter set quantity field; a specific embodiment in which the processor 702 decodes the bitstream data according to the parameter indication information to present the point cloud media is as follows:

[0374] Determine the quantity of the parameter sets included in the current sample according to the parameter set quantity field;

[0375] Wherein, the current sample refers to the sample being decoded.

[0376] Based on the same inventive concept, the principle of problem-solving and the beneficial effects of the content consumption device provided in the embodiments of the present application are similar to those of the data processing method of the point cloud media in the method embodiments of the present application. The principle and beneficial effects of the method implementation can be referred to. For the sake of brevity of description, they will not be repeated here.

[0377] Figure 8 FIG. is a schematic structural diagram of a content production device provided in an embodiment of the present application; the content production device may refer to the computer device used by the provider of the point cloud media, and the computer device may be a terminal (such as a PC, a smart mobile device (such as a smart phone), etc.) or a server. As Figure 8 shown, the content production device includes a capture device 801, a processor 802, a memory 803, and a transmitter 804. Among them:

[0378] The capture device 801 is used to collect the real-world audio-visual scene to obtain the raw data of the point cloud media (including the audio content and video content that are synchronized in time and space). The capture device 801 may include, but is not limited to: audio devices, camera devices, and sensing devices. Among them, the audio devices may include audio sensors, microphones, etc. The camera devices may include ordinary cameras, stereo cameras, light field cameras, etc. The sensing devices may include laser devices, radar devices, etc.

[0379] The processor 802 (or CPU (Central Processing Unit)) is the processing core of the content production device. The processor 802 is adapted to implement one or more program instructions, and is specifically adapted to load and execute one or more program instructions to implement Figure 3 the process of the data processing method of the point cloud media shown.

[0380] The memory 803 is the memory device in the content production device, and is used to store programs and media resources. It can be understood that the memory 803 here may include both the built-in storage medium in the content production device, and of course may also include the extended storage medium supported by the content production device. It should be noted that the memory may be a high-speed RAM memory, or a non-volatile memory, such as at least one disk memory; optionally, it may also be at least one memory located far from the aforementioned processor. The memory provides a storage space, and this storage space is used to store the operating system of the content production device. And, in this storage space, it is also used to store a computer program, the computer program includes program instructions, and the program instructions are adapted to be called and executed by the processor to be used to execute the steps of the data processing method of the point cloud media. In addition, the memory 803 can also be used to store the point cloud media file formed after being processed by the processor, and the point cloud media file includes media file resources and media presentation description information.

[0381] The transmitter 804 is used to realize the transmission interaction between the content production device and other devices, and is specifically used to realize the transmission of the point cloud media between the content production device and the content playback device. That is, the content production device transmits the relevant media resources of the point cloud media to the content playback device through the transmitter 804.

[0382] Please refer to Figure 8 , the processor 802 may include a converter 821, an encoder 822, and a packager 823; among them:

[0383] The converter 821 is used to perform a series of conversion processes on the captured video content, making the video content suitable for video encoding of the executed point cloud media. The conversion processes may include: splicing and projection. Optionally, the conversion processes may further include region encapsulation. The converter 821 can convert the captured 3D video content into 2D images and provide them to the encoder for video encoding.

[0384] The encoder 822 is used to perform audio encoding on the captured audio content to form an audio bitstream of the point cloud media. It is also used to perform video encoding on the 2D images converted by the converter 821 to obtain a video bitstream.

[0385] The encapsulator 823 is used to encapsulate the audio bitstream and the video bitstream in a file container according to the file format of the point cloud media (such as ISOBMFF) to form a media file resource of the point cloud media. This media file resource can be a media file or a media segment to form a media file of the point cloud media; and record the metadata of the media file resource of the point cloud media by using media presentation description information according to the requirements of the file format of the point cloud media. The encapsulated file of the point cloud media obtained by the encapsulator processing is saved in the memory and provided to the content playback device for presenting the point cloud media as needed.

[0386] The processor 802 (specifically, each device included in the processor) executes Figure 3 each step of the data processing method of the point cloud media shown. Specifically, the memory 803 stores one or more first instructions, and the one or more first instructions are adapted to be loaded and executed by the processor 802 to perform the following steps:

[0387] Obtain the point cloud data of the point cloud media, and perform encoding processing on the point cloud data to obtain the bitstream data of the point cloud media. The bitstream data of the point cloud media contains at least two sets of parameter sets belonging to the same type;

[0388] Generate parameter indication information according to the association relationship between the point cloud frames in the point cloud media and at least two parameter sets;

[0389] Encapsulate the bitstream data and the parameter indication information to obtain the media file of the point cloud media.

[0390] In one implementation, the media file contains tracks, and each track contains one or more samples. One sample corresponds to one point cloud frame of the point cloud media;

[0391] Each track contains one or more reference samples, and a reference sample refers to a sample containing a parameter set;

[0392] The parameter indication information is used to indicate the reference samples that one or more samples in the media file need to refer to during decoding.

[0393] In one implementation, the parameter indication information includes at least one of a parameter set sample group and a decoding configuration record; the parameter set sample group is used to indicate a parameter set required for decoding one or more samples in a media file, and the decoding configuration record is used to indicate initialization parameter information required for decoding bitstream data.

[0394] In one implementation, the parameter indication information includes a parameter set identification field;

[0395] When the value of the parameter set identification field is a first set value, it indicates a parameter set identifier of the parameter set included in the current sample indicated in the parameter indication information;

[0396] When the value of the parameter set identification field is a second set value, it indicates that the parameter indication information does not indicate a parameter set identifier of the parameter set included in the current sample, and the bitstream data does not include a parameter set that belongs to the same type as the parameter set included in the current sample and has a different parameter set identifier;

[0397] Herein, the current sample refers to the sample being encoded.

[0398] In one implementation, a sample is divided into one or more point cloud patches; the parameter indication information includes a point cloud patch information field;

[0399] When the value of the point cloud patch information field is a first set value, it indicates that there is an association relationship between the parameter set included in the current sample and at least one point cloud patch;

[0400] When the value of the point cloud patch information field is a second set value, it indicates that the association relationship between the parameter set included in the current sample and one or more point cloud patches is not indicated;

[0401] Herein, the current sample refers to the sample being encoded.

[0402] In one implementation, a sample is divided into one or more point cloud blocks; the parameter indication information includes a point cloud block information field;

[0403] When the value of the point cloud block information field is a first set value, it indicates that there is an association relationship between the parameter set included in the current sample and at least one point cloud block;

[0404] When the value of the point cloud block information field is a second set value, it indicates that the association relationship between the parameter set included in the current sample and one or more point cloud blocks is not indicated;

[0405] Herein, the current sample refers to the sample being encoded.

[0406] In one implementation, a sample is divided into M point cloud patches, where M is a positive integer; one or more of the M point cloud patches form a point cloud patch group; the parameter indication information includes a point cloud patch group information field;

[0407] When the value of the point cloud patch group information field is a first set value, it indicates that there is an association relationship between the parameter set included in the current sample and at least one point cloud patch group;

[0408] When the value of the point cloud patch group information field is a second set value, it indicates that the association relationship between the parameter set included in the current sample and one or more point cloud patch groups is not indicated;

[0409] Wherein, the current sample refers to the sample being encoded.

[0410] In one implementation, the parameter indication information includes a parameter set identification field; the parameter set identification field is used to indicate the identifier of the parameter set included in the current sample;

[0411] Wherein, the current sample refers to the sample being encoded.

[0412] In one implementation, a sample is divided into one or more point cloud patches; the parameter indication information includes a point cloud patch quantity field and a point cloud patch identification field;

[0413] The point cloud patch quantity field is used to indicate the quantity of the point cloud patches associated with the parameter set included in the current sample;

[0414] The point cloud patch identification field is used to indicate the identifier of the point cloud patches associated with the parameter set included in the current sample;

[0415] Wherein, the current sample refers to the sample being encoded.

[0416] In one implementation, a sample is divided into one or more point cloud blocks; the parameter indication information includes a point cloud block quantity field and a point cloud block identification field;

[0417] The point cloud block quantity field is used to indicate the quantity of the point cloud blocks associated with the parameter set included in the current sample;

[0418] The point cloud block identification field is used to indicate the identifier of the point cloud blocks associated with the parameter set included in the current sample;

[0419] Wherein, the current sample refers to the sample being encoded.

[0420] In one implementation, a sample is divided into M point cloud patches, where M is a positive integer; one or more of the M point cloud patches form a point cloud patch group; the parameter indication information includes a point cloud patch group quantity field and a point cloud patch group identification field;

[0421] The point cloud patch group quantity field is used to indicate the quantity of point cloud patch groups associated with the parameter set included in the current sample;

[0422] The point cloud patch group identification field is used to indicate the identifier of the point cloud patch group associated with the parameter set included in the current sample;

[0423] Wherein, the current sample refers to the sample being encoded.

[0424] In one implementation, the parameter indication information includes a parameter set sample group, and the parameter set sample group includes a parameter set quantity field; the parameter set quantity field is used to indicate the quantity of parameter sets included in the current sample;

[0425] Wherein, the current sample refers to the sample being encoded.

[0426] Based on the same inventive concept, the principle of problem-solving and the beneficial effects of the content production device provided in the embodiments of the present application are similar to those of the data processing method of point cloud media in the method embodiments of the present application. The principle of problem-solving and the beneficial effects of the method can be referred to. For the sake of concise description, they will not be elaborated here.

[0427] The embodiments of the present application further provide a computer-readable storage medium, in which one or more instructions are stored, and the one or more instructions are suitable for being loaded and executed by a processor to execute the data processing method of point cloud media in the above method embodiments.

[0428] The embodiments of the present application further provide a computer program product containing instructions, which when running on a computer, causes the computer to execute the data processing method of point cloud media in the above method embodiments.

[0429] The embodiments of the present application further provide a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above data processing method of point cloud media.

[0430] The steps in the method embodiments of the present application can be adjusted, combined, and deleted according to actual needs.

[0431] The modules in the device embodiments of the present application can be combined, divided, and deleted according to actual needs.

[0432] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. The readable storage medium can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disc, etc.

[0433] The above-disclosed is only a preferred embodiment of the present application. Of course, it cannot be used to limit the scope of rights of the present application. Those of ordinary skill in the art can understand all or part of the processes of implementing the above embodiments, and the equivalent changes made according to the claims of the present application still fall within the scope covered by the application.

Claims

1. A data processing method for point cloud media, characterized in that, The method includes: Obtaining a media file of point cloud media, where the media file includes bitstream data of the point cloud media and parameter indication information; the bitstream data contains at least two parameter sets of the same type; the parameter indication information is used to indicate the association relationship between the at least two parameter sets and point cloud frames in the point cloud media; Decoding the bitstream data according to the parameter indication information to present the point cloud media.

2. The method according to claim 1, wherein The media file contains tracks, and one or more samples are included in the tracks, and one sample corresponds to one point cloud frame of the point cloud media; One or more reference samples are included in the tracks, and the reference sample refers to a sample containing a parameter set; The parameter indication information is used to indicate the reference samples that one or more samples in the media file need to refer to during decoding.

3. The method according to claim 2, wherein The parameter indication information includes at least one of a parameter set sample group and a decoding configuration record; the parameter set sample group is used to indicate the parameter sets that one or more samples in the media file need during decoding, and the decoding configuration record is used to indicate the initialization parameter information required for decoding the bitstream data.

4. The method according to claim 2 or 3, characterized in that, The parameter indication information includes a parameter set identification field; the decoding the bitstream data according to the parameter indication information to present the point cloud media includes: If the value of the parameter set identification field is a first set value, determining the parameter set identifier of the parameter set indicated in the parameter indication information that the current sample contains; If the value of the parameter set identification field is a second set value, determining that the parameter set identifier of the parameter set that the current sample contains is not indicated in the parameter indication information, and the bitstream data does not contain other parameter sets of the same type as the parameter set that the current sample contains and with different parameter set identifiers; Wherein, the current sample refers to the sample being decoded.

5. The method according to claim 2 or 3, characterized in that, One sample is divided into one or more point cloud slices; the parameter indication information includes a point cloud slice information field; the decoding the bitstream data according to the parameter indication information to present the point cloud media includes: If the value of the point cloud slice information field is a first set value, determining that there is an association relationship between the parameter set contained in the current sample and at least one point cloud slice; If the value of the point cloud slice information field is a second set value, determining that the association relationship between the parameter set contained in the current sample and the one or more point cloud slices is not indicated; Wherein, the current sample refers to the sample being decoded.

6. The method according to claim 2 or 3, characterized in that, One sample is divided into one or more point cloud blocks; the parameter indication information includes a point cloud block information field; the decoding the bitstream data according to the parameter indication information to present the point cloud media includes: If the value of the point cloud block information field is a first set value, determining that there is an association relationship between the parameter set contained in the current sample and at least one point cloud block; If the value of the point cloud block information field is a second set value, determining that the association relationship between the parameter set contained in the current sample and the one or more point cloud blocks is not indicated; Wherein, the current sample refers to the sample being decoded.

7. The method according to claim 2 or 3, characterized in that, A sample is divided into M point cloud slices, where M is a positive integer; one or more of the M point cloud slices form a point cloud slice group; the parameter indication information includes a point cloud slice group information field; the decoding of the bitstream data according to the parameter indication information to present the point cloud media includes: If the value of the point cloud slice group information field is a first set value, determining that there is an association relationship between the parameter set included in the current sample and at least one point cloud slice group; If the value of the point cloud slice group information field is a second set value, determining that the association relationship between the parameter set included in the current sample and the one or more point cloud slice groups is not indicated; Wherein, the current sample refers to the sample being decoded.

8. The method according to claim 2 or 3, characterized in that, The parameter indication information includes a parameter set identification field; the decoding of the bitstream data according to the parameter indication information to present the point cloud media includes: Determining the parameter set included in the current sample according to the parameter set identification field; Wherein, the current sample refers to the sample being decoded.

9. The method according to claim 2 or 3, characterized in that, A sample is divided into one or more point cloud slices; the parameter indication information includes a point cloud slice quantity field and a point cloud slice identification field; the decoding of the bitstream data according to the parameter indication information to present the point cloud media includes: Determining the number of point cloud slices associated with the parameter set corresponding to this field according to the point cloud slice quantity field; Determining the point cloud slices associated with the parameter set corresponding to this field according to the point cloud slice identification field.

10. The method according to claim 2 or 3, characterized in that, A sample is divided into one or more point cloud blocks; the parameter indication information includes a point cloud block quantity field and a point cloud block identification field; the decoding of the bitstream data according to the parameter indication information to present the point cloud media includes: Determining the number of point cloud blocks associated with the parameter set corresponding to this field according to the point cloud block quantity field; Determining the point cloud blocks associated with the parameter set corresponding to this field according to the point cloud block identification field.

11. The method according to claim 2 or 3, characterized in that, A sample is divided into M point cloud slices, where M is a positive integer; one or more of the M point cloud slices form a point cloud slice group; the parameter indication information includes a point cloud slice group quantity field and a point cloud slice group identification field; the decoding of the bitstream data according to the parameter indication information to present the point cloud media includes: Determining the number of point cloud slice groups associated with the parameter set corresponding to this field according to the point cloud slice group quantity field; Determining the point cloud slice groups associated with the parameter set corresponding to this field according to the point cloud slice group identification field.

12. The method according to claim 2 or 3, characterized in that, The parameter indication information includes a parameter set sample group, and the parameter set sample group includes a parameter set quantity field; the decoding of the bitstream data according to the parameter indication information to present the point cloud media includes: Determining the number of parameter sets included in the current sample according to the parameter set quantity field; Wherein, the current sample refers to the sample being decoded.

13. A data processing method for point cloud media, characterized in that, The method includes: Obtain the point cloud data of the point cloud media, and perform encoding processing on the point cloud data to obtain the bitstream data of the point cloud media. The bitstream data of the point cloud media includes at least two parameter sets of the same type; Generate parameter indication information according to the association relationship between the point cloud frames in the point cloud media and at least two parameter sets; Package the bitstream data and the parameter indication information to obtain the media file of the point cloud media.

14. The method according to claim 13, wherein The media file includes tracks, and one or more samples are included in the tracks. One sample corresponds to one point cloud frame of the point cloud media; One or more reference samples are included in the tracks, and the reference sample refers to a sample containing a parameter set; The parameter indication information is used to indicate the reference samples required for decoding one or more samples in the media file.

15. The method according to claim 14, wherein The parameter indication information includes at least one of a parameter set sample group and a decoding configuration record; the parameter set sample group is used to indicate the parameter sets required for decoding one or more samples in the media file, and the decoding configuration record is used to indicate the initialization parameter information required for decoding the bitstream data.

16. The method according to claim 14 or 15, characterized in that, The parameter indication information includes a parameter set identification field; When the value of the parameter set identification field is the first set value, it indicates that the parameter set identifier of the parameter set included in the current sample is indicated in the parameter indication information; When the value of the parameter set identification field is the second set value, it indicates that the parameter set identifier of the parameter set included in the current sample is not indicated in the parameter indication information, and the bitstream data does not include other parameter sets of the same type as the parameter set included in the current sample and with different parameter set identifiers; Wherein, the current sample refers to the sample being encoded.

17. The method according to claim 14 or 15, characterized in that, One sample is divided into one or more point cloud slices; the parameter indication information includes a point cloud slice information field; When the value of the point cloud slice information field is the first set value, it indicates that there is an association relationship between the parameter set included in the current sample and at least one point cloud slice; When the value of the point cloud slice information field is the second set value, it indicates that the association relationship between the parameter set included in the current sample and the one or more point cloud slices is not indicated; Wherein, the current sample refers to the sample being encoded.

18. The method according to claim 14 or 15, characterized in that, One sample is divided into one or more point cloud blocks; the parameter indication information includes a point cloud block information field; When the value of the point cloud block information field is the first set value, it indicates that there is an association relationship between the parameter set included in the current sample and at least one point cloud block; When the value of the point cloud block information field is the second set value, it indicates that the association relationship between the parameter set included in the current sample and the one or more point cloud blocks is not indicated; Wherein, the current sample refers to the sample being encoded.

19. The method according to claim 14 or 15, characterized in that, One sample is divided into M point cloud slices, where M is a positive integer; one or more of the M point cloud slices form a point cloud slice group; the parameter indication information includes a point cloud slice group information field; When the value of the point cloud patch group information field is the first set value, it indicates that there is an association relationship between the parameter set included in the current sample and at least one point cloud patch group; When the value of the point cloud patch group information field is the second set value, it indicates that the association relationship between the parameter set included in the current sample and the one or more point cloud patch groups is not indicated; Wherein, the current sample refers to the sample being encoded.

20. The method according to claim 14 or 15, characterized in that, The parameter indication information includes a parameter set identification field; the parameter set identification field is used to indicate the identifier of the parameter set included in the current sample; Wherein, the current sample refers to the sample being encoded.

21. The method according to claim 14 or 15, characterized in that, A sample is divided into one or more point cloud patches; the parameter indication information includes a point cloud patch quantity field and a point cloud patch identification field; The point cloud patch quantity field is used to indicate the quantity of the point cloud patches associated with the parameter set included in the current sample; The point cloud patch identification field is used to indicate the identifier of the point cloud patches associated with the parameter set included in the current sample; Wherein, the current sample refers to the sample being encoded.

22. The method according to claim 14 or 15, characterized in that A sample is divided into one or more point cloud blocks; the parameter indication information includes a point cloud block quantity field and a point cloud block identification field; The point cloud block quantity field is used to indicate the quantity of the point cloud blocks associated with the parameter set included in the current sample; The point cloud block identification field is used to indicate the identifier of the point cloud blocks associated with the parameter set included in the current sample; Wherein, the current sample refers to the sample being encoded.

23. The method according to claim 14 or 15, characterized in that A sample is divided into M point cloud patches, where M is a positive integer; one or more of the M point cloud patches form a point cloud patch group; the parameter indication information includes a point cloud patch group quantity field and a point cloud patch group identification field; The point cloud patch group quantity field is used to indicate the quantity of the point cloud patch groups associated with the parameter set included in the current sample; The point cloud patch group identification field is used to indicate the identifier of the point cloud patch groups associated with the parameter set included in the current sample; Wherein, the current sample refers to the sample being encoded.

24. The method according to claim 14 or 15, characterized in that, The parameter indication information includes a parameter set sample group, and the parameter set sample group includes a parameter set quantity field; the parameter set quantity field is used to indicate the quantity of the parameter sets included in the current sample; Wherein, the current sample refers to the sample being encoded.

25. A data processing device for point cloud media, characterized in that, The data processing device of the point cloud media includes: An acquisition unit, configured to acquire a media file of the point cloud media, where the media file includes the bitstream data of the point cloud media and parameter indication information; the bitstream data contains at least two parameter sets of the same type; the parameter indication information is used to indicate the association relationship between the at least two parameter sets and the point cloud frames in the point cloud media; A processing unit, configured to decode the bitstream data according to the parameter indication information to present the point cloud media.

26. A data processing device for point cloud media, characterized in that The data processing device of the point cloud media includes: An acquisition unit, configured to acquire the point cloud data of the point cloud media, and perform encoding processing on the point cloud data to obtain the bitstream data of the point cloud media, where the bitstream data of the point cloud media contains at least two groups of parameter sets of the same type; A processing unit, configured to generate parameter indication information according to the association relationship between the point cloud frames and at least two sets of parameter sets in the point cloud media; and configured to encapsulate the bitstream data and the parameter indication information to obtain a media file of the point cloud media.

27. A computer device, characterized in that, Comprising: A storage device and a processor; A memory, in which a computer program is stored; A processor, configured to load the computer program to implement the data processing method of the point cloud media according to any one of claims 1-12; or configured to load the computer program to implement the data processing method of the point cloud media according to any one of claims 13-24.

28. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and the computer program is adapted to be loaded and executed by the processor to implement the data processing method of the point cloud media according to any one of claims 1-12; or to be loaded and executed to implement the data processing method of the point cloud media according to any one of claims 13-24.

Images