A data processing method of a point cloud medium and a related device

Abstract

The embodiment of the application provides a kind of point cloud media data processing method and related equipment, wherein the method comprises: obtaining the point cloud code stream of point cloud media, the point cloud code stream includes one or more point cloud frames, and the parameter information of each point cloud frame;According to the parameter information of each point cloud frame, the point cloud code stream is decoded to present the point cloud media.Processing.The embodiment of the application can organize the parameter information in point cloud code stream, guide the decoding and presentation of point cloud media, and optimize the resource overhead of decoding end.

Description

Technical Field

[0001] This application relates to the field of Internet technology, specifically to a data processing method for point cloud media, a data processing device for point cloud media, a computer device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] With the continuous development of point cloud technology, the compression coding of point cloud media has become an important research issue. Although existing point cloud compression coding technologies define parameters such as sequence headers and point cloud patch information, some problems still exist. For example, they cannot organize the parameter information in the point cloud bitstream, easily leading to wasted resources on the decoding side. Therefore, how to improve point cloud coding technology is a hot topic. Summary of the Invention

[0003] This application provides a data processing method and related equipment for point cloud media, which can organize parameter information in point cloud bitstreams, guide the decoding and presentation of point cloud media, and optimize the resource consumption of the decoding end.

[0004] On one hand, embodiments of this application provide a data processing method for point cloud media, the method comprising:

[0005] The point cloud stream of the point cloud media is obtained. The point cloud stream contains one or more point cloud frames and parameter information of each point cloud frame. The parameter information of the point cloud frame is used to indicate at least one of the following: frame sequence number, frame timestamp information, frame spatial information, and information of intra-frame data units.

[0006] Based on the parameter information of each point cloud frame, the point cloud bitstream is decoded to present the point cloud media.

[0007] On one hand, embodiments of this application provide a data processing method for point cloud media, the method comprising:

[0008] Acquire point cloud media;

[0009] The point cloud media is encoded to obtain a point cloud bitstream. The point cloud bitstream contains one or more point cloud frames and parameter information of each point cloud frame. The parameter information of the point cloud frame is used to indicate at least one of the following: frame sequence number, frame timestamp information, frame spatial information, and information of intra-frame data units.

[0010] Transmit point cloud bitstream.

[0011] On one hand, embodiments of this application provide a data processing apparatus for point cloud media, the apparatus comprising:

[0012] The acquisition unit is used to acquire the point cloud bitstream of the point cloud media. The point cloud bitstream contains one or more point cloud frames and parameter information of each point cloud frame. The parameter information of the point cloud frame is used to indicate at least one of the following: frame sequence number, frame timestamp information, frame spatial information, and information of intra-frame data units.

[0013] The processing unit is used to decode the point cloud bitstream according to the parameter information of each point cloud frame to present the point cloud media.

[0014] In one embodiment, the parameter information of the point cloud frame is used to indicate the frame sequence number; one or more point cloud frames in the point cloud bitstream constitute a point cloud sequence in sequence.

[0015] The parameter information of the current point cloud frame includes a frame sequence number field, which indicates the sequence number of the current point cloud frame in the point cloud sequence.

[0016] The current point cloud frame refers to the point cloud frame being decoded in the point cloud bitstream.

[0017] In one embodiment, the parameter information of the point cloud frame is used to indicate the frame timestamp information; the parameter information of the current point cloud frame includes at least one of the following fields: timestamp flag field, timestamp quantity field, timestamp type field, and timestamp field;

[0018] The timestamp flag field is used to indicate whether the current point cloud frame contains a timestamp. When the timestamp flag field is a first preset value, it indicates that the current point cloud frame does not contain a timestamp; when the timestamp flag field is a second preset value, it indicates that the current point cloud frame contains a timestamp.

[0019] The timestamp count field is used to indicate the number of timestamps contained in the current point cloud frame;

[0020] The timestamp type field is used to indicate the type of the current timestamp. If the timestamp type field is the first preset value, it indicates that the current timestamp is the decoding time; if the timestamp type field is the second preset value, it indicates that the current timestamp is the presentation time; if the timestamp type field is the third preset value, it indicates that the current timestamp is the acquisition time.

[0021] The timestamp field is used to indicate any one or more of the acquisition time, presentation time, and decoding time of the current point cloud frame; the timestamp field is used to indicate Coordinated Universal Time, or the timestamp field is indicated according to a preset format, the preset format including at least one of the following fields: year timestamp field, hour timestamp field, second timestamp field, and frame timestamp field;

[0022] Here, the current point cloud frame refers to the point cloud frame being decoded in the point cloud bitstream, and the current timestamp refers to the timestamp of the current point cloud frame being used.

[0023] In one embodiment, the parameter information of the point cloud frame is used to indicate the frame spatial information; the parameter information of the current point cloud frame includes a spatial information flag field.

[0024] The spatial information flag field indicates whether to indicate the spatial information of the current point cloud frame. When the spatial information flag field is the first value, it indicates the spatial information of the current point cloud frame; when the spatial information flag field is the second value, it does not indicate the spatial information of the current point cloud frame.

[0025] Spatial information includes at least one of the following: coordinates of the origin of the bounding box of the current point cloud frame, width of the bounding box of the current point cloud frame, and depth of the bounding box of the current point cloud frame.

[0026] In one embodiment, the parameter information of a point cloud frame is used to indicate the information of the data unit within the frame; a point cloud frame contains one or more data units, and the data unit includes point cloud patches; the parameter information of the current point cloud frame includes a point cloud patch count field or a point cloud patch count minus one field.

[0027] The point cloud patch quantity field is used to indicate the number of point cloud patches contained in the current point cloud frame;

[0028] The "Number of Point Cloud Pieces - One" field is used to indirectly indicate the number of point cloud pieces contained in the current point cloud frame.

[0029] The current point cloud frame refers to the point cloud frame being decoded in the point cloud bitstream.

[0030] In one embodiment, the parameter information of the point cloud frame is also used to indicate whether the point cloud frame contains a parameter set, which includes any of the following: a geometry head parameter set, an attribute head parameter set, a geometry head parameter set, and an attribute head parameter set.

[0031] The parameter information of the current point cloud frame includes any one of the following fields: geometry header flag field, attribute header flag field, and attribute count minus one field;

[0032] The geometry head flag field is used to indicate whether the current point cloud frame contains a geometry head parameter set; if the geometry head flag field is a first preset value, it indicates that the current point cloud frame contains a geometry head parameter set; if the geometry head flag field is a second preset value, it indicates that the current point cloud frame does not contain a geometry head parameter set.

[0033] The attribute header flag field is used to indicate whether the current point cloud frame contains an attribute header parameter set. If the attribute header flag field is a first set value, it indicates that the current point cloud frame contains an attribute header parameter set; if the attribute header flag field is a second set value, it indicates that the current point cloud frame does not contain an attribute header parameter set.

[0034] The "Attribute Count Minus One" field indicates the number of attribute data groups corresponding to the attribute header parameter set contained in the current point cloud frame.

[0035] In one embodiment, one or more point cloud frames in the point cloud bitstream sequentially constitute a point cloud sequence;

[0036] When the current point cloud frame is the first frame in the point cloud sequence, the geometry header flag field is set to the first preset value, and the attribute header flag field is set to the first set value; the attribute quantity minus one field is the maximum number of attribute data groups contained in the point cloud sequence.

[0037] When the current point cloud frame is not the first frame in the point cloud sequence, if the geometry head flag field is set to the first preset value, it indicates that the set of geometry head parameters in the point cloud sequence has been updated.

[0038] If the current point cloud frame is not the first frame in the point cloud sequence, and the attribute header flag field is set to the first set value, it indicates that the set of attribute header parameters in the point cloud sequence has been updated.

[0039] In one embodiment, in point cloud media, a set of attribute data corresponds to a set of attribute header parameters;

[0040] The attribute header parameter set includes an attribute presence flag field, which indicates whether the current attribute header parameter set contains the encoded parameters of the x-th attribute data, where x is an integer greater than or equal to 0 and less than or equal to 15;

[0041] If the attribute exists and the flag field is the first value, it indicates that the current attribute header parameter set contains the encoded parameters of the xth group of attribute data;

[0042] If the attribute exists and the flag field has the second value, it indicates that the current attribute header parameter set does not contain the encoded parameters of the xth attribute data.

[0043] In one embodiment, the attribute header parameter set includes an attribute type field and an attribute data identifier field, which represent the correspondence between the attribute header parameter set and the attribute data.

[0044] In one embodiment, the parameter information of the point cloud frame is used to indicate the information of the data unit within the frame;

[0045] If the data unit includes point cloud frames, the parameter information of the current point cloud frame includes the start code and end code of the current point cloud frame; the current point cloud frame refers to the point cloud frame being decoded in the point cloud bitstream.

[0046] If the data unit includes point cloud slices, the parameter information of the current point cloud frame includes the start code and end code of the current point cloud slice in the current point cloud frame. The current point cloud slice refers to the point cloud slice being decoded in the point cloud bitstream.

[0047] If the data unit includes the geometric data of the point cloud patch, then the parameter information of the current point cloud frame includes the start code and end code of the geometric data of the current point cloud patch;

[0048] If the data unit includes attribute data of a point cloud patch, then the parameter information of the current point cloud frame includes the start code and end code of the current attribute data of the current point cloud patch. The current attribute data of the current point cloud patch refers to the attribute data being decoded in the current point cloud patch.

[0049] In one embodiment, the start code of the current point cloud frame is a preset specific bit string, or the start code of the current point cloud frame is the end code of the previous point cloud frame; the start code of the current point cloud frame is used to indicate the position where the current point cloud frame starts decoding, and the end code of the current point cloud frame is used to indicate the position where the current point cloud frame ends decoding.

[0050] The start code of the current point cloud patch is a preset specific bit string, or the start code of the current point cloud patch is the end code of the previous point cloud patch; the start code of the current point cloud patch is used to indicate the position where the current point cloud patch starts decoding, and the end code of the current point cloud patch is used to indicate the position where the current point cloud patch ends decoding.

[0051] The start code of the geometric data of the current point cloud patch is a preset specific bit string, or the start code of the geometric data of the current point cloud patch is the end code of the previous point cloud patch; the start code of the geometric data of the current point cloud patch is used to indicate the position where the geometric data of the current point cloud patch begins to be decoded, and the end code of the geometric data of the current point cloud patch is used to indicate the position where the geometric data of the current point cloud patch ends to be decoded.

[0052] The start code of the current attribute data of the current point cloud patch is a preset specific bit string, or the start code of the current attribute data of the current point cloud patch is the end code of the geometric data of the current point cloud patch, or the start code of the current attribute data of the current point cloud patch is the end code of the previous attribute data of the current attribute data in the current point cloud patch; the start code of the current attribute data of the current point cloud patch is used to indicate the position where the current attribute data of the current point cloud patch begins to be decoded, and the end code of the current attribute data of the current point cloud patch is used to indicate the position where the current attribute data of the current point cloud patch ends to be decoded.

[0053] In one embodiment, the start code of the current attribute data of the current point cloud patch is also used to indicate the type of the current attribute data of the current point cloud patch.

[0054] In one embodiment, the parameter information of each point cloud frame is set in the frame header of each point cloud frame. A point cloud frame includes one or more data units. When the processing unit decodes the point cloud bitstream according to the parameter information of each point cloud frame to present the point cloud media, it can be specifically used for:

[0055] Determine the target point cloud frame that needs to be decoded from the point cloud bitstream;

[0056] Based on the parameter information of the target point cloud frame, locate the data unit that needs to be decoded in the target point cloud frame;

[0057] Decode and present the data units that need to be decoded in the target point cloud frame.

[0058] On one hand, embodiments of this application provide a data processing apparatus for point cloud media, the apparatus comprising:

[0059] The acquisition unit is used to acquire point cloud media;

[0060] The processing unit is used to encode the point cloud media to obtain a point cloud bitstream. The point cloud bitstream contains one or more point cloud frames and parameter information of each point cloud frame. The parameter information of the point cloud frame is used to indicate at least one of the following: frame sequence number, frame timestamp information, frame spatial information, and information of intra-frame data units.

[0061] The processing unit is also used to transmit point cloud code streams.

[0062] In one embodiment, the parameter information of each point cloud frame is set in the frame header of each point cloud frame;

[0063] The parameter information of the point cloud frame is also used to indicate whether the point cloud frame contains a parameter set, which includes any of the following: geometric head parameter set, attribute head parameter set, geometric head parameter set, and attribute head parameter set.

[0064] In one embodiment, the parameter information of the point cloud frame is used to indicate the information of the data unit within the frame;

[0065] If the data unit includes point cloud frames, the parameter information of the current point cloud frame includes the start code and end code of the current point cloud frame; the current point cloud frame refers to the point cloud frame being encoded in the point cloud bitstream.

[0066] If the data unit includes point cloud slices, the parameter information of the current point cloud frame includes the start code and end code of the current point cloud slice in the current point cloud frame. The current point cloud slice refers to the point cloud slice being encoded in the point cloud bitstream.

[0067] If the data unit includes the geometric data of the point cloud patch, then the parameter information of the current point cloud frame includes the start code and end code of the geometric data of the current point cloud patch;

[0068] If the data unit includes the attribute data of the point cloud patch, then the parameter information of the current point cloud frame includes the start code and end code of the current attribute data of the current point cloud patch. The current attribute data of the current point cloud patch refers to the attribute data being encoded in the current point cloud patch.

[0069] On one hand, embodiments of this application provide a computer device, the computer device comprising:

[0070] A processor is used to execute computer programs;

[0071] A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the data processing method for point cloud media as described above.

[0072] On one hand, embodiments of this application provide a computer-readable storage medium storing a computer program that is loaded by a processor and executes the data processing method for point cloud media as described above.

[0073] On one hand, embodiments of this application provide a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the aforementioned point cloud media data processing method.

[0074] In this embodiment of the application, a point cloud bitstream of point cloud media is obtained. The point cloud bitstream contains one or more point cloud frames and parameter information of each point cloud frame. Based on the parameter information of each point cloud frame, the point cloud bitstream is decoded to present the point cloud media. This method can flexibly organize the parameter information in the point cloud bitstream to guide the decoding and presentation of the point cloud media, thereby optimizing the resource overhead of the decoding end. Attached Figure Description

[0075] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0076] Figure 1a This is a schematic diagram of a 6DoF provided in an exemplary embodiment of this application;

[0077] Figure 1b This is a schematic diagram of a 3DoF implementation provided in an exemplary embodiment of this application;

[0078] Figure 1c This is a schematic diagram of a 3DoF+ embodiment provided in this application;

[0079] Figure 2a This is an architectural diagram of a point cloud media data processing system provided in an exemplary embodiment of this application;

[0080] Figure 2b This is a flowchart illustrating data processing of point cloud media provided in an exemplary embodiment of this application;

[0081] Figure 3 This is a flowchart illustrating a data processing method for point cloud media provided in an exemplary embodiment of this application;

[0082] Figure 4 This is a schematic diagram of a data unit contained in a point cloud frame according to an exemplary embodiment of this application;

[0083] Figure 5 This is a flowchart illustrating a data processing method for point cloud media provided in another exemplary embodiment of this application;

[0084] Figure 6 This is a schematic diagram of the structure of a point cloud media data processing device provided in an exemplary embodiment of this application;

[0085] Figure 7 This is a schematic diagram of the structure of a point cloud media data processing device provided in another exemplary embodiment of this application;

[0086] Figure 8 This is a schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application. Detailed Implementation

[0087] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0088] In this application, the terms "first," "second," etc., are used to distinguish identical or similar items with essentially the same function. It should be understood that there is no logical or temporal dependency between "first," "second," and "nth," nor are there any restrictions on quantity or execution order.

[0089] In this application, the term "at least one" means one or more, and "multiple" means two or more; similarly, the term "at least one set" means one or more sets, and "multiple sets" means two or more sets. For example, if a point in a point cloud has multiple sets of attribute data, it means that the point has two or more sets of attribute data.

[0090] The following describes other technical terms used in this application:

[0091] I. Immersive Media

[0092] Immersive media refers to media files that provide immersive content, allowing viewers to experience visual, auditory, and other sensory sensations reminiscent of the real world. Based on the degree of freedom viewers have when consuming the media content, immersive media can be categorized into: 6DoF (Degree of Freedom) immersive media, 3DoF immersive media, and 3DoF+ immersive media. Among these, for example... Figure 1a As shown, 6DoF refers to the ability of viewers of immersive media to freely translate along the X, Y, and Z axes. For example, viewers of immersive media can move freely within 360-degree VR content. Similar to 6DoF are 3DoF and 3DoF+ production techniques. Figure 1b This is a schematic diagram of a 3DoF implementation provided in an embodiment of this application; as shown... Figure 1b As shown, 3DoF refers to the viewer of immersive media being fixed at the center point in a three-dimensional space, while the viewer's head rotates along the X, Y, and Z axes to view the images provided by the media content. Figure 1c This is a schematic diagram of a 3DoF+ embodiment provided in this application, as shown below. Figure 1c As shown, 3DoF+ refers to the ability of immersive media viewers to move their heads within a limited space based on 3DoF to view the images provided by the media content when the virtual scene provided by the immersive media has a certain depth information.

[0093] II. Point Cloud

[0094] A point cloud is a set of randomly distributed discrete points in space that represent the spatial structure and surface properties of a three-dimensional object or scene. Each point in a point cloud includes at least geometric data, which represents the point's three-dimensional position information. Depending on the application scenario, points in a point cloud may also include one or more sets of attribute data. Each set of attribute data reflects a specific attribute possessed by the point, such as color, material, or other information. Typically, each point in a point cloud has the same number of sets of attribute data.

[0095] Point clouds can flexibly and conveniently represent the spatial structure and surface properties of three-dimensional objects or scenes, and therefore have a wide range of applications. They can be used in virtual reality (VR) games, computer-aided design (CAD), geographic information systems (GIS), autonomous navigation systems (ANS), digital cultural heritage, free-viewpoint broadcasting, 3D immersive remote presentation, and 3D reconstruction of biological tissues and organs.

[0096] Point clouds are primarily acquired through the following methods: computer generation, 3D laser scanning, and 3D photogrammetry. Specifically, point clouds can be obtained by acquiring visual scenes of the real world using acquisition devices (a set of cameras or a camera system with multiple lenses and sensors). 3D laser scanning can obtain point clouds of static 3D objects or scenes in the real world, acquiring millions of point clouds per second; 3D photogrammetry can obtain point clouds of dynamic 3D objects or scenes in the real world, acquiring tens of millions of point clouds per second. Furthermore, in the medical field, point clouds of biological tissues and organs can be obtained through magnetic resonance imaging (MRI), computed tomography (CT), and electromagnetic positioning information. Point clouds can also be directly generated by computers based on virtual 3D objects and scenes. With the continuous accumulation of large-scale point cloud data, the efficient storage, transmission, publication, sharing, and standardization of point cloud data have become crucial for point cloud applications.

[0097] III. Point Cloud Media

[0098] Point cloud media is a typical 6DoF immersive media. Point cloud media comprises a sequence of one or more point cloud frames, each frame consisting of geometric and attribute data of one or more points within the point cloud. The geometric data, also known as 3D positional information, refers to the spatial coordinates (x, y, z) of a point in the point cloud, including its coordinates along each axis of the 3D coordinate system, such as x (x-axis), y (y-axis), and z (z-axis). Each point cloud frame can be enclosed by a bounding box, which can be the smallest cuboid that encloses the frame. Therefore, the spatial information of each point cloud frame can include at least one of the following: the coordinates of the origin of the bounding box, the width of the bounding box, and the depth of the bounding box. The coordinates of the origin of the bounding box can include the x-coordinate, y-coordinate, and z-coordinate of the point cloud frame's bounding box origin.

[0099] A point in a point cloud can include one or more sets of attribute data. Each set of attribute data reflects a specific attribute possessed by the point. For example, a point in a point cloud may have a set of color attribute data, which reflects the point's color attribute (such as red, yellow, etc.). Similarly, a point in a point cloud may have a set of reflectivity attribute data, which reflects the laser reflection intensity attribute of the point. When a point in a point cloud has multiple sets of attribute data, these sets of attribute data can be of the same or different types. For example, a point in a point cloud may have one set of color attribute data and one set of reflectivity attribute data; or a point in a point cloud may have two sets of color attribute data, which respectively reflect the point's color attribute at different times.

[0100] IV. Data Units within a Point Cloud Frame

[0101] A point cloud frame may include one or more data units. In one implementation, when a point cloud frame includes only one data unit, that data unit is the point cloud frame itself. In another implementation, a point cloud frame can be divided into multiple point cloud patches, each of which is a data unit of that point cloud frame. Here, a point cloud patch represents a set of syntactic elements (e.g., geometric point cloud patches, attribute point cloud patches) of data after a point cloud frame has been partially or fully encoded. That is, a point cloud patch may include the geometric data in the point cloud frame, or it may include one or more sets of attribute data in the point cloud frame, or it may include both the geometric data in the point cloud frame and one or more sets of attribute data. In yet another implementation, a type of data contained in a point cloud patch can be considered as an independent data unit. For example, the geometric data of a point cloud patch can be considered as a data unit; a set of attribute data in a point cloud patch can be considered as a data unit.

[0102] V. Point Cloud Compression (PCC)

[0103] Point cloud encoding refers to the process of encoding the geometric and attribute data of each point in a point cloud to obtain a point cloud bitstream. Point cloud encoding can include two main processes: geometric data encoding and attribute data encoding. During the encoding process, G-PCC (Geometry-based Point Cloud Compression) can be used to encode the geometric data of each point in the point cloud media to obtain a geometric bitstream; G-PCC can also be used to encode the attribute data of each point in the point cloud media to obtain an attribute bitstream; the geometric bitstream and the attribute bitstream together constitute the point cloud bitstream of the point cloud media.

[0104] The point cloud bitstream may include metadata information, which may include one or more of the following: a sequence header parameter set, a geometry header parameter set, an attribute header parameter set, and point cloud patch information. The sequence header parameter set, geometry header parameter set, attribute header parameter set, and point cloud patch information will be described in detail below:

[0105] (1) Sequence Header Parameter Set

[0106] This sequence header parameter set contains the decoding parameters required for decoding the point cloud sequence corresponding to the point cloud bitstream. The syntax of this sequence header parameter set is shown in Table 1:

[0107] Table 1

[0108]

[0109]

[0110]

[0111] The meanings of the fields in Table 1 above are as follows:

[0112] Profile ID field: This profile ID field is an 8-bit unsigned integer used to indicate the profile of the bitstream.

[0113] Level label field (level_id): This level label field is an 8-bit unsigned integer used to indicate the level of the bitstream conformance.

[0114] The sequence parameter set representation field (sequence_parameter_set_id) provides an SPS (Sequence Parameter Set) identifier for reference by other syntax elements. This identifier is an integer between 0 and 31.

[0115] The high-order part of the bounding box origin x-coordinate (bounding_box_offset_x_upper): This field is an unsigned integer used to represent the number of bits above 16 bits in the bounding box x-coordinate.

[0116] The lower 16 bits of the bounding box origin x-coordinate field (bounding_box_offset_x_lower): This field is an unsigned integer representing the lower 16 bits of the bounding box origin x-coordinate. Therefore, the bounding box origin x-coordinate is: Bounding_box_offset_x = (bounding_box_offset_x_upper) << 16 + bounding_box_offset_x_lower.

[0117] The high-order part of the bounding box origin y-coordinate field (bounding_box_offset_y_upper) is an unsigned integer used to represent the number of bits above 16 bits in the bounding box y-coordinate.

[0118] The lower 16 bits of the bounding box origin's y-coordinate: This field is an unsigned integer representing the lower 16 bits of the bounding box origin's y-coordinate. Therefore, the bounding box origin's y-coordinate is: Bounding_box_offset_y = (bounding_box_offset_y_upper) << 16 + bounding_box_offset_y_lower.

[0119] The high-order part of the bounding box origin z-coordinate (bounding_box_offset_z_upper): This field is an unsigned integer used to represent the number of bits above 16 bits in the bounding box z-coordinate.

[0120] The lower 16 bits of the bounding box origin's z-coordinate: This field is an unsigned integer representing the lower 16 bits of the bounding box origin's z-coordinate. Therefore, the bounding box origin's z-coordinate is: Bounding_box_offset_z = (bounding_box_offset_z_upper) << 16 + bounding_box_offset_z_lower.

[0121] The upper bounding box width field (bounding_box_size_width_upper): This field is an unsigned integer used to represent the number of bits in the bounding box width that are greater than 16 bits.

[0122] The lower bounding box width field (bounding_box_size_width_lower): This field is an unsigned integer used to represent the lower 16 bits of the bounding box width. The bounding box width is: bounding_box_size_width = (bounding_box_size_width_upper) << 16 + bounding_box_size_width_lower.

[0123] The bounding box height upper part field (bounding_box_size_height_upper): This field is an unsigned integer used to represent the number of bits in the bounding box height that are higher than 16 bits.

[0124] The lower bounding box height field (bounding_box_size_height_lower): This field is an unsigned integer representing the lower 16 bits of the bounding box height. The bounding box height is: bounding_box_size_height = (bounding_box_size_height_upper) << 16 + bounding_box_size_height_lower.

[0125] The bounding box depth upper part field (bounding_box_size_depth_upper): This field is an unsigned integer used to represent the number of bits in the bounding box depth that are greater than 16 bits.

[0126] The lower bounding box depth field (bounding_box_size_depth_lower): This field is an unsigned integer representing the lower 16 bits of the bounding box depth. The bounding box depth is: bounding_box_size_depth = (bounding_box_size_depth_upper) << 16 + bounding_box_size_depth_lower.

[0127] Quantization step size high-order part field (quant_step_lower): This field is a 16-bit number used to represent the high 16 bits of the quantization step size of a 32-bit floating-point number.

[0128] The lower 16 bits of the quantization step size (quant_step_upper): This field is 16 bits and represents the lower 16 bits of the 32-bit floating-point quantization step size. The quantization step size is: quant_step = (float)((quant_step_upper<<16)+quant_step_lower).

[0129] The `geomRemoveDuplicateFlag` field is a binary variable that indicates whether duplicate points are removed before geometric encoding. When the `geomRemoveDuplicateFlag` field is at its first value (e.g., '1'), it means that duplicate points are removed before geometric encoding, i.e., points with the same geometric position are removed; when the `geomRemoveDuplicateFlag` field is at its second value (e.g., '0'), it means that duplicate points are not removed before geometric encoding.

[0130] The attribute present flag field (attribute_present_flag) is a binary variable. When the attribute present flag field is the first value (e.g., '1'), it indicates that this bitstream contains attribute encoding; when the attribute present flag field is the second value (e.g., '0'), it indicates that this bitstream does not contain attribute encoding.

[0131] The `maxNumAttributesMinus1` field is an unsigned integer. Increasing this field by 1 indicates the maximum number of attribute codes supported by this standard's bitstream. The value of `maxNumAttributesMinus1` is an integer between 0 and 15. When `maxNumAttributesMinus1` does not appear in the bitstream, it defaults to 0.

[0132] The attribute-adaptive prediction flag field (attribute_adapt_pred) is a binary variable. When the value of the attribute-adaptive prediction flag field is the first value (e.g., '0'), it indicates that no adaptive prediction method is selected; when the value of the attribute-adaptive prediction flag field is the second value (e.g., '1'), it indicates that switching from a geometrically based prediction method to an attribute-based prediction method is allowed.

[0133] Attribute quantization parameter field (attribute_qp): This attribute quantization parameter field is an unsigned integer used to represent the attribute quantization parameter.

[0134] (2) Set of geometric head parameters

[0135] This geometry header parameter set contains the decoding parameters required for decoding the geometric data of each point cloud frame in the point cloud bitstream. The syntax of this geometry header parameter set is shown in Table 2:

[0136] Table 2

[0137]

[0138]

[0139] The semantics of the syntax in Table 2 above are as follows:

[0140] Geometry parameter set field (geometry_parameter_set_id): This geometry parameter set field provides a geometry parameter identifier for reference by other syntax elements. This geometry parameter identifier is an integer between 0 and 31.

[0141] The geometry_sequence_parameter_set_id field: This field identifies an SPS identifier for use in the current geometry parameter set. This SPS identifier is an integer between 0 and 31, and this value remains consistent across all geometry parameter sets within the same point cloud.

[0142] The node size field of the geometry macroblock (gps_lcu_node_size_log2_minus_one): This field is an unsigned integer used to indicate whether block structure encoding is enabled. When the value of the node size field of the geometry macroblock is the target preset value (e.g., '0'), it indicates that block structure encoding is disabled; when the value of the node size field of the geometry macroblock is greater than the target preset value (e.g., the value of the node size field of the geometry macroblock is greater than '0'), it indicates that block structure encoding is enabled, and the geometry node size of the macroblock is defined, i.e., gps_lcu_node_size_log2 = gps_lcu_node_size_log2_minus_one + 1.

[0143] Geometric Implicit Partition Flag (gps_implicit_geom_partition_flag): This field is a binary variable used to indicate whether geometric implicit partitioning is disabled. If the geometric implicit partition flag field is the first set value (e.g., '0'), it means that geometric implicit partitioning is disabled; if the geometric implicit partition flag field is the second set value (e.g., '1'), it means that geometric implicit partitioning is enabled.

[0144] The field `gps_max_num_implicit_qtbt_before_ot` represents the maximum number of quadtree / binary tree partitions allowed before octree partitioning in geometric implicit partitioning.

[0145] Minimum size field for quadtree / binary tree partitioning (gps_min_size_implicit_qtbt): This field is an unsigned integer that represents the minimum allowed partition size for a quadtree or binary tree partition in geometric implicit partitioning.

[0146] Geometric outlier encoding mode flag field (gps_single_mode_flag): This field is a binary variable. When the geometric outlier encoding mode flag field is the first set value (e.g., '0'), it indicates that the geometric outlier encoding mode is off; when the geometric outlier encoding mode flag field is the second set value (e.g., '1'), it indicates that the geometric outlier encoding mode is on.

[0147] When the aforementioned geometric implicit partitioning flag field is the second set value ('1'), gps_max_num_implicit_qtbt_before_ot and gps_min_size_implicit_qtbt need to be limited according to the logarithmic size of the root node, as shown in Table 3.

[0148] Table 3

[0149]

[0150] The geometry encoding status storage flag field (gps_save_stat_flag) is a binary variable used to indicate whether the encoding status is stored. When the geometry encoding status storage flag field is the first set value (e.g., '0'), it means that the encoding status is not stored, that is, the entropy encoding context and the geometry encoding hash table information are not stored; when the geometry encoding status storage flag field is the second set value (e.g., '1'), it means that the encoding status is stored.

[0151] (3) Attribute header parameter set

[0152] The attribute header parameter set contains the decoding parameters required for decoding the attribute data of each point cloud frame in the point cloud bitstream. The syntax of this attribute header parameter set is shown in Table 4:

[0153] Table 4

[0154]

[0155]

[0156]

[0157] The meanings of the fields in Table 4 above are as follows:

[0158] The attribute presence flag field (attributePresentFlag[attrIdx]) is a binary variable. When the attribute presence flag field takes the first set value (e.g., 1), it indicates that the current point cloud bitstream contains the attrIdx attribute code; when it takes the second set value (e.g., 0), it indicates that the current point cloud bitstream does not contain the attrIdx attribute code. attrIdx is an integer between 0 and 15. The meaning of this attribute presence flag field can be found in Table 5 below.

[0159] Table 5, x attribute encoding mapping table

[0160] 0 Color 1 Reflectance 2..15 reserve

[0161] Attribute Transform Algorithm Flag Field (transform): This attribute transform algorithm flag field is a binary variable. It controls whether wavelet transform is used to encode the attribute data. When the attribute transform algorithm flag field takes the first preset value (e.g., 1), it controls the use of wavelet transform for encoding the attribute data; when it takes the second preset value (e.g., 0), it controls the use of prediction methods for encoding the attribute data.

[0162] Attribute transformation coefficient quantization parameter difference (attrTransformQpDelta): This attribute transformation coefficient quantization parameter difference is an unsigned integer used to represent the difference between the attribute transformation coefficient quantization parameter and the attribute residual quantization parameter. Attribute transformation coefficient quantization parameter attrTransformQp = attrQuantParam(attribute quantization parameter) + attrTransformQpDelta.

[0163] Attribute Transform Points (attrTransformNumPoints): The attribute transform points are unsigned integers used to represent the number of points used for attribute transformation, i.e., wavelet transform using attrTransformNumPoints points; when attrTransformNumPoints is set to the target value (e.g., 0), it means that all points in the chip are used for wavelet transform.

[0164] The maximum search neighbor count logarithm minus seven (maxNumOfNeighbour_log2_minus7): This maximum search neighbor count logarithm minus seven is an unsigned integer used to derive the variable maxNumOfNeighbour (maximum neighbor count), representing the maximum number of encoded neighbors available for searching. This controls the search range of neighbor candidates during attribute prediction and the number of points cached by the hardware. maxNumOfNeighbour is calculated using the following formula:

[0165] maxNumOfNeighbour = 2 (maxNumOfNeighbor_log2_minus7+7) maxNumOfNeighbour_log2_minus7.

[0166] The attribute residual quadratic prediction field (cross_component_pred) is a binary variable used to indicate whether attribute residual quadratic prediction is allowed. When the attribute residual quadratic prediction field takes the first set value (e.g., 1), it means that attribute residual quadratic prediction is allowed. When the attribute residual quadratic prediction field takes the second set value (e.g., 0), it means that attribute residual quadratic prediction is not allowed.

[0167] Residual encoding order switch field (orderSwitch): This residual encoding order switch field is a binary variable; when the value of the residual encoding order switch field is the first set value (such as 1), it indicates that the residual encoding order is UYV / GRB order; when the value of the residual encoding order switch field is the second set value (such as 0), it indicates that the residual encoding order is RGB / YUV order.

[0168] The half-zero runlength_enable flag field is a binary variable. When the half-zero runlength_enable flag field is at the first set value (e.g., 1), it means that the half-zero runlength is used. When the value of the half-zero runlength_enable flag field is at the second set value (e.g., 0), it means that the half-zero runlength is not used.

[0169] Chroma channel Cb quantization parameter offset (chromaQpOffsetCb): This chroma channel Cb quantization parameter offset is a signed integer used to control the quantization parameter of the Cb channel. The value range of this chroma channel Cb quantization parameter offset is -16 to 16. If chromaQpOffsetCb does not exist in the current attribute header information, then the value of chromaQpOffsetCb is 0, that is, chromaQpCb = Clip3(minQP, maxQP, attribute_qp + chromaQpOffsetCb). Note that the quantization parameter lumaQp of the luma channel is attribute_qp, the minimum supported quantization parameter is minQP = 0, and the maximum supported quantization parameter is maxQP = 63.

[0170] Chroma channel quantization parameter offset (chromaQpOffsetCr): This chroma channel quantization parameter offset is a signed integer used to control the quantization parameter of the Cr channel; the value range of this chroma channel quantization parameter offset is -16 to 16. If chromaQpOffsetCr does not exist in the current attribute header information, then the value of chromaQpOffsetCr is 0, that is, chromaQpCr = Clip3(minQP, maxQP, attribute_qp + chromaQpOffsetCr). Note that the quantization parameter lumaQp of the luma channel is attribute_qp, the minimum supported quantization parameter is minQP = 0, and the maximum supported quantization parameter is maxQP = 63.

[0171] Nearest neighbor prediction parameter 1 (nearestPredParam1): This nearest neighbor prediction parameter 1 is an unsigned integer used to control the threshold for nearest neighbor prediction.

[0172] Nearest neighbor prediction parameter 2 (nearestPredParam2): This nearest neighbor prediction parameter 2 is an unsigned integer used to control the threshold for nearest neighbor prediction; the threshold is expressed as attrQuantParam*nearestPredParam1+nearestPredParam1.

[0173] Spatial bias coefficient (axisBias): This spatial bias coefficient is an unsigned integer used to control the offset in the Z direction during the calculation of attribute prediction values.

[0174] `outputBitDepthMinus1`: This is an unsigned integer used to control the attribute output bit depth, ranging from 0 to 15. Specifically, `outputBitDepth = outputBitDepthMinus1 + 1`. If this syntax element is not in the point cloud code stream, the default value is zero.

[0175] LoD Levels (numOflevelOfDetail): This LoD level is an unsigned integer used to control the number of LOD levels divided during attribute prediction. In the bitstream conforming to this section, numOflevelOfDetail should not exceed 32.

[0176] Maximum number of neighbor points selected for prediction (maxNumOfPredictNeighbours): This maximum number of neighbor points selected for prediction is an unsigned integer used to limit the number of neighbor points selected during attribute prediction. In bitstreams conforming to this section, maxNumOfPredictNeighbours should not exceed 16.

[0177] The LoD intra-layer prediction flag field (intraLodFlag) is a binary variable used to control whether intra-layer prediction is enabled. When the LoD intra-layer prediction flag field is set to the first preset value (e.g., 1), intra-layer prediction is enabled; when the LoD intra-layer prediction flag field is set to the second preset value (e.g., 0), intra-layer prediction is disabled.

[0178] The color reorder mode field (colorReorderMode) is an unsigned integer used to represent the reorder mode selected for the current color information. When the color reorder mode field is the first preset value (e.g., 0), it represents the original point cloud input order; when the color reorder mode field is the second preset value (e.g., "1"), it represents Hilbert reordering; and when the color reorder mode field is the third preset value (e.g., "2"), it represents Morton reordering.

[0179] The reflectance reordering mode field (colorReorderMode) is an unsigned integer. When the reflectance reordering mode field is the first preset value (e.g., 0), it indicates the original point cloud input order; when the reflectance reordering mode field is the second preset value (e.g., 1), it indicates Hilbert reordering; and when the reflectance reordering mode field is the third preset value (e.g., 2), it indicates Morton reordering.

[0180] Maximum cache limit parameter (maxNumofCoeff): This maximum cache limit parameter is an unsigned integer used to calculate the maximum number of transformation parameters that can be cached in attribute transformation coding.

[0181] Maximum Delay Limit Parameter (coeffLengthControl): This maximum delay limit parameter is an unsigned integer used to limit the maximum number of delays for transformation parameters in attribute transformation coding. Specifically, the maximum number of delay points is calculated as maxNumofCoeff * coeffLengthControl.

[0182] Attribute Encoding Order Field (attrEncodeOrder): This field is a binary variable used to control the encoding order of attributes when the point cloud contains multiple attribute types. When the attribute encoding order field is the first set value (e.g., 0), it means that color is encoded first, then reflectance; when the attribute encoding order field is the second set value (e.g., 1), it means that reflectance is encoded first, then color.

[0183] Cross-type attribute prediction field (crossAttrTypePred): This is a binary variable for the cross-type attribute prediction field. When the value of the cross-type attribute prediction field is the first set value (e.g., 1), it means that cross-type attribute prediction is allowed; when the value of the cross-type attribute prediction field is the second set value (e.g., 0), it means that cross-type attribute prediction is not allowed.

[0184] Cross-type attribute prediction weight parameter 1 (crossAttrTypePredParam1): Cross-type attribute prediction weight parameter 1 is a 15-bit unsigned integer used to control the weight parameter 1 for calculating geometric information distance and attribute information distance in cross-type attribute prediction.

[0185] Cross-type attribute prediction weight parameter 2 (crossAttrTypePredParam2): Cross-type attribute prediction weight parameter 2 is a 21-bit unsigned integer used to control the weight parameter 2 for calculating geometric information distance and attribute information distance in cross-type attribute prediction.

[0186] Reflectance Group Prediction Flag Field (refGroupPred): This reflectance group prediction flag field is a binary variable used to control whether the reflectance group prediction mode of the prediction transformation is enabled. When the reflectance group prediction flag field is the first set value (e.g., 1), it means that group prediction is enabled. When the reflectance group prediction flag field is the second set value (e.g., 0), it means that group prediction is disabled.

[0187] Initial Predictive Transform Ratio (initPredTransRatio): This initial predictive transform ratio is a signed integer used in the multi-level transform algorithm for attribute compression (transform=1) to control the size of the initial distance threshold used when constructing the predictive transform tree.

[0188] Transform Residual Layer Flag Field (transResLayer): This transform residual layer flag field is a binary variable used in the multi-layer transform algorithm for attribute compression (transform=1) to control whether attribute residual compensation is used. When transResLayer is the first set value (e.g., 1), it indicates that attribute residual compensation is used; when transResLayer is the second set value (e.g., 0), it indicates that attribute residual compensation is not used.

[0189] Color Golomb Order (ColorGolombNum): This color Golomb order is an unsigned integer used to represent the order K of the K-order exponent Golomb used when decoding the current color prediction residual or transform coefficients, where K = ColorGolombNum.

[0190] Reflectance index Golomb order (RefGolombNum): This reflectance index Golomb order is an unsigned integer used to represent the order K of the K-order exponential Golomb used when decoding the current reflectance prediction residual or transform coefficients, where K = ColorGolombNum.

[0191] The decoding mode flag field of the current coefficient to be decoded (coeffEncodeModeFlag): This decoding mode flag field of the current coefficient to be decoded is a binary variable; when the decoding mode flag field of the current coefficient to be decoded is the first set value (such as 1), it means that the point cloud attribute transformation coefficient is decoded according to 9.3.16.3; when the decoding mode flag field of the current coefficient to be decoded is the second set value (such as 0), it means that the point cloud attribute transformation coefficient is decoded according to 9.3.12.

[0192] (4) Point cloud information

[0193] A point cloud patch may contain only the geometric data of a point cloud frame, in which case the point cloud patch can be called a geometric point cloud patch; a point cloud patch may also contain only one or more sets of attribute data of a point cloud frame, in which case the point cloud patch can be called an attribute point cloud patch; a point cloud patch may contain both the geometric data of a point cloud frame and one or more sets of attribute data of a point cloud frame.

[0194] ① When the point cloud patch information includes a geometric point cloud patch, the syntax of the geometric point cloud patch is shown in Table 6:

[0195] Table 6

[0196]

[0197]

[0198] The meanings of each field in Table 6 above are as follows:

[0199] Slice ID field (slice_id): This slice ID field is an unsigned integer used to represent the slice ID of the point cloud.

[0200] Patch geometry context mode flag field (gsh_context_mode): This field is a binary variable. The point cloud patch selects different context modes in the geometric octree encoding according to the value of the patch geometry context mode flag field.

[0201] The patch geometry isolated point encoding mode flag field (gsh_single_mode_flag) is a binary variable used to indicate whether the patch geometry isolated point encoding mode of the current point cloud is disabled. When the patch geometry isolated point encoding mode flag field is the first value (e.g., '0'), it indicates that the patch geometry isolated point encoding mode of the current point cloud is disabled; when the patch geometry isolated point encoding mode flag field is the second value (e.g., '1'), it indicates that the patch geometry isolated point encoding mode of the current point cloud is enabled. Its default value is 0 when this symbol is not present in the point cloud bitstream.

[0202] Patch geometry planar mode flag field (gsh_planar_mode): This field is a binary variable used to indicate whether the current point cloud patch geometry planar encoding mode is turned off; when the patch geometry planar mode flag field is the first value (e.g., '0'), it means that the current point cloud patch geometry planar encoding mode is turned off; when the patch geometry planar mode flag field is the second value (e.g., '1'), it means that the current point cloud patch geometry planar mode is turned on.

[0203] The high 16-bit portion of the x-coordinate of the bounding box origin (gsh_bounding_box_offset_x_upper): This field is a 16-bit number, representing the high 16 bits of a 32-bit signed integer representing the x-coordinate of the bounding box.

[0204] The lower 16 bits of the x-coordinate of the bounding box origin (gsh_bounding_box_offset_x_lower): This 16-bit field represents the lower 16 bits of a 32-bit signed integer representing the x-coordinate of the bounding box origin. The x-coordinate of the bounding box origin is:

[0205] gsh_bounding_box_offset_x=(gsh_bounding_box_offset_x_upper)<<16+gsh_bounding_box_offset_x_lower.

[0206] The high 16-bit portion of the y-coordinate of the bounding box origin (gsh_bounding_box_offset_y_upper) is a 16-bit field that represents the high 16 bits of a 32-bit signed integer representing the y-coordinate of the bounding box.

[0207] The lower 16 bits of the y-coordinate of the bounding box origin (gsh_bounding_box_offset_y_lower) are 16-bit fields. This field represents the lower 16 bits of a 32-bit signed integer representing the y-coordinate of the bounding box origin. The y-coordinate of the bounding box origin is:

[0208] gsh_bounding_box_offset_y=(gsh_bounding_box_offset_y_upper)<<16+gsh_bounding_box_offset_y_lower.

[0209] The high 16-bit portion of the z-coordinate of the origin of the bounding box (gsh_bounding_box_offset_z_upper): This field is a 16-bit number, representing the high 16 bits of a 32-bit signed integer representing the z-coordinate of the bounding box.

[0210] The lower 16 bits of the z-coordinate of the bounding box origin (gsh_bounding_box_offset_z_lower): This field is a 16-bit number representing the lower 16 bits of a 32-bit signed integer representing the z-coordinate of the bounding box origin. The z-coordinate of the bounding box origin is:

[0211] gsh_bounding_box_offset_z=(gsh_bounding_box_offset_z_upper)<<16+gsh_bounding_box_offset_z_lower.

[0212] The high-order part of the logarithmic size of the bounding box in the X direction (gsh_bounding_box_nodeSizeXLog2_upper): This field is an unsigned integer that represents the number of bits in the logarithmic size of the bounding box in the X direction that are higher than 16 bits.

[0213] The lower 16 bits of the logarithmic size of the piece bounding box in the X direction (gsh_bounding_box_nodeSizeXLog2_lower): This field is an unsigned integer representing the lower 16 bits of the logarithmic size of the piece bounding box in the X direction. The logarithmic size of the piece bounding box in the X direction is:

[0214] gsh_bounding_box_nodeSizeXLog2=(gsh_bounding_box_nodeSizeXLog2_upper)<<16+gsh_bounding_box_nodeSizeXLog2_lower.

[0215] The high-order part of the Y-axis logarithmic size of the bounding box (gsh_bounding_box_nodeSizeYLog2_upper): This field is an unsigned integer that represents the number of bits in the Y-axis logarithmic size of the bounding box that are higher than 16 bits.

[0216] The lower 16 bits of the Y-axis logarithmic size of the bounding box (gsh_bounding_box_nodeSizeYLog2_lower): This field is an unsigned integer representing the lower 16 bits of the Y-axis logarithmic size of the bounding box. The Y-axis logarithmic size of the bounding box is:

[0217] gsh_bounding_box_nodeSizeYLog2=(gsh_bounding_box_nodeSizeYLog2_upper)<<16+gsh_bounding_box_nodeSizeYLog2_lower.

[0218] The high-order part of the Z-axis logarithmic size of the piece bounding box (gsh_bounding_box_nodeSizeZLog2_upper): This field is an unsigned integer that represents the number of bits in the Z-axis logarithmic size of the piece bounding box that are higher than 16 bits.

[0219] The lower 16 bits of the Z-axis logarithmic size of the piece bounding box (gsh_bounding_box_nodeSizeLog2_lower): This field is an unsigned integer representing the lower 16 bits of the Z-axis logarithmic size of the piece bounding box. The Z-axis logarithmic size of the piece bounding box is:

[0220] gsh_bounding_box_nodeSizeZLog2=(gsh_bounding_box_nodeSizeZLog2_upper)<<16+gsh_bounding_box_nodeSizeZLog2_lower.

[0221] The high-order part of the number of points in the slice (num_points_upper): This field is an unsigned integer that represents the number of bits in the slice that are higher than 16 bits.

[0222] The lower 16 bits of the number of points in the slice (num_points_upper): This field is an unsigned integer representing the lower 16 bits of the number of points in the slice. The number of points in the slice is: num_points = ((num_points_upper<<16) + num_points_lower).

[0223] Optionally, a general geometric data flow syntax can also be defined for geometric data flows, as shown in Table 7:

[0224] Table 7 Definition of General Geometric Syntax

[0225]

[0226] ② When the point cloud patch information includes attribute point cloud patches, the syntax of the attribute point cloud patch is shown in Table 8:

[0227] Table 8

[0228]

[0229] The meanings of each field in Table 8 above are as follows:

[0230] Slice ID field: This slice ID field is the label of the point cloud slice.

[0231] Reflectance Attribute Quantization Parameter Offset Field (reflQPoffset): This field is a signed integer used to control the reflectance attribute quantization parameter. The value range of this field is -32 to 32.

[0232] Optionally, a generic attribute data flow syntax can also be defined for attribute data flows, as shown in Table 9:

[0233] Table 9 Attribute Data Flow Definitions

[0234]

[0235] VI. Point Cloud Decoding

[0236] Point cloud decoding refers to the process of decoding the point cloud bitstream obtained from point cloud encoding to reconstruct the point cloud. More specifically, it refers to the process of reconstructing the geometric and attribute data of each point in the point cloud based on the geometric and attribute bitstreams within the point cloud bitstream. After obtaining the point cloud bitstream on the decoding side, for the geometric bitstream, entropy decoding is first performed to obtain the quantized geometric data of each point in the point cloud, and then inverse quantization is performed to reconstruct the geometric data of each point in the point cloud. For the attribute bitstream, entropy decoding is first performed to obtain the quantized prediction residual information or quantized transform coefficients of each point in the point cloud; then, the quantized prediction residual information is inverse quantized to obtain the reconstructed residual information, and the quantized transform coefficients are inverse quantized to obtain the reconstructed transform coefficients. The reconstructed transform coefficients are then inversely transformed to obtain the reconstructed residual information. Based on the reconstructed residual information of each point in the point cloud, the attribute data of each point in the point cloud can be reconstructed. The reconstructed attribute data of each point in the point cloud is then matched sequentially with the reconstructed geometric data to reconstruct the point cloud.

[0237] Based on the above description, this application provides a data processing scheme for point cloud media. The general principle of this scheme is as follows: On the encoding side, the point cloud media can be encoded to obtain a point cloud bitstream. For each point cloud frame in the bitstream: ① Define frame-level parameter information, which may include, but is not limited to: frame sequence number, frame timestamp, frame spatial information, and information of intra-frame data units; ② The parameter information of each point cloud frame can flexibly indicate whether each point cloud frame contains a geometric header parameter set or an attribute header parameter set; ③ According to the encoding process of each point cloud frame, corresponding start codes and end codes can be added to the data units in each point cloud frame to enable the decoding side to quickly locate the data units. The start code can be used to indicate the position where the data unit begins encoding, and the end code can be used to indicate the position where the data unit ends encoding. When decoding the point cloud stream on the decoding side, the data units that need to be decoded in each point cloud frame can be quickly located based on the parameter information of each point cloud frame contained in the point cloud stream, thereby achieving partial decoding. Simultaneously, the corresponding geometric header parameter set or attribute header parameter set can be obtained based on whether the point cloud frame indicated by the parameter information contains such a set, thus achieving correct decoding. Furthermore, the frame sequence number and frame timestamp in the parameter information can also ensure that decoding and presentation are performed at the appropriate time and in the appropriate order, optimizing the resource overhead on the decoding side.

[0238] The point cloud media data processing solution provided in this application embodiment can also be combined with vehicle networking technology. Specifically, the point cloud media data processing solution can obtain the acquisition time of each point cloud frame, and can update the high-precision map in real time according to the acquisition time during autonomous driving; the high-precision map can be used for positioning, or can be used to realize automatic navigation.

[0239] Based on the above description, the following is combined with Figure 2aA data processing system suitable for implementing the point cloud media provided in the embodiments of this application is described. For example... Figure 2a As shown, the point cloud media data processing system 20 may include a content production device 201 and a media processing device 202. The content production device 201 is located on the encoding side of the point cloud media; this content production device 201 can be a terminal device or a server. The media processing device 202 is located on the decoding side of the point cloud media; this media processing device 202 can also be a terminal device or a server. A communication connection can be established between the content production device 201 and the media processing device 202. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, vehicle terminal, smart TV, etc., but is not limited to these. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0240] The content production device 201 and the media processing device 202 involve the transmission process of point cloud media. This transmission process can be based on various transmission protocols (or transmission signaling). The transmission protocols here may include, but are not limited to: DASH (Dynamic Adaptive Streaming over HTTP) protocol, HLS (HTTP Live Streaming) protocol, SMTP (Smart Media Transport Protocol), TCP (Transmission Control Protocol), etc.

[0241] The data processing procedure for point cloud media is described in detail below:

[0242] (1) The process of acquiring point cloud media.

[0243] Content creation device 201 can acquire point cloud media, which can be obtained through scene capture or device generation. Scene capture point cloud media refers to obtaining point cloud media by capturing real-world visual scenes through a capture device associated with content creation device 201. The capture device provides point cloud media acquisition services for content creation device 201 and can include, but is not limited to, any of the following: camera equipment, sensing equipment, and scanning equipment. Camera equipment can include ordinary cameras, stereo cameras, and light field cameras, etc. Sensing equipment can include laser equipment, radar equipment, etc. Scanning equipment can include 3D laser scanning equipment, etc. The capture device associated with content creation device 201 can refer to hardware components installed in content creation device 201, such as a terminal camera or sensor. The capture device associated with content creation device 201 can also refer to hardware devices connected to content creation device 201, such as a camera connected to content creation device 201. Device generation point cloud media refers to content creation device 201 generating point cloud media based on virtual objects (such as virtual 3D objects and virtual 3D scenes obtained through 3D modeling).

[0244] (2) Encoding process of point cloud media.

[0245] Content production equipment 201 can use point cloud encoding to encode the geometric and attribute data in the acquired point cloud media to obtain a point cloud bitstream (including encoded geometric and attribute bitstreams). This point cloud bitstream can include one or more point cloud frames and parameter information for each point cloud frame. The parameter information of a point cloud frame can be used to indicate one or more of the following: frame sequence number, frame timestamp, frame spatial information, and intra-frame data unit information. When the parameter information of a point cloud frame is used to indicate an intra-frame data unit, the parameter information can also include the start and end positions of the encoding of that intra-frame data unit. Furthermore, the parameter information of a point cloud frame can also be used to indicate whether the point cloud frame contains a geometric header parameter set or an attribute header parameter set, so that the media processing equipment 202 can correctly obtain the corresponding geometric header parameter set or attribute header parameter set for decoding when decoding the point cloud frame.

[0246] After obtaining the point cloud stream, the content production device 201 can send the point cloud stream to the media processing device 202.

[0247] (3) Decoding process of point cloud media.

[0248] Media processing device 202 can obtain the point cloud bitstream and corresponding media presentation description information of point cloud media through content production device 201. The point cloud bitstream and media presentation description information are transmitted from content production device 201 to media processing device 202 via transmission signaling (such as DASH, SMT). The decoding process of media processing device 202 is the reverse of the encoding process of content production device 201; media processing device 202 decodes the point cloud bitstream to reconstruct the point cloud media.

[0249] During the decoding process, the media processing device 202 can obtain the parameter information of each point cloud frame in the point cloud bitstream, and then decode the point cloud bitstream according to the parameter information of each point cloud frame included in the point cloud bitstream to present the point cloud media.

[0250] In one embodiment, during the decoding process, the target point cloud frame to be decoded can be determined from the point cloud bitstream, and the correct geometric head parameter set and attribute head parameter set can be obtained based on whether the target point cloud frame contains a geometric head parameter set and an attribute head parameter set as indicated by the parameter information of the target point cloud frame, and the target point cloud frame can be decoded based on the obtained geometric head parameter set and attribute head parameter set.

[0251] Optionally, the parameter information of the target point cloud frame may include the position where the data unit in the frame starts decoding and the position where decoding ends. Based on this parameter information, the data unit that needs to be decoded can be located in the target point cloud frame, and the data unit that needs to be decoded can be decoded and presented, thus achieving partial decoding.

[0252] (4) The rendering process of point cloud media.

[0253] The media processing device 202 renders the decoded point cloud media according to the metadata related to rendering and windowing in the media presentation description information, obtains the point cloud frames of the point cloud media, and presents the point cloud media according to the presentation time of the point cloud frames.

[0254] In one embodiment, this application also provides a flowchart illustrating a data processing method for point cloud media. Please refer to... Figure 2bOn the content production device side: First, the visual scene A of the real world is sampled by the acquisition device to obtain the point cloud source data B of the point cloud media corresponding to the visual scene of the real world. The point cloud source data B is a frame sequence composed of a large number of point cloud frames. Then, the acquired point cloud media is encoded to obtain the point cloud bitstream E (including the encoded geometric bitstream and attribute bitstream). The point cloud media contains one or more point cloud frames and parameter information of each point cloud frame. Next, the point cloud bitstream E is encapsulated to obtain the media file corresponding to the point cloud media. Specifically, the content production device 201 combines one or more encoded bitstreams into a media file F for file playback, or an initialization segment and a sequence of media segments (FS) for streaming transmission, according to a specific media container file format. The media container file format can refer to the ISO basic media file format specified in International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC) 14496-12. In one implementation, the content production device also encapsulates metadata into a media file or a sequence of initialization / media segments, and transmits the sequence of initialization / media segments to the media processing device 202 via transmission signaling (such as a dynamic adaptive streaming media transmission interface).

[0255] At the media processing device: First, the media file sent by the content production device 201 is received. This media file may include: a media file F' for file playback, or an initialization segment and a sequence of media segments Fs' for streaming transmission. Then, the media file is decapsulated to obtain a point cloud stream E'. It should be understood that the file decapsulation process of the media processing device 202 is the reverse of the file encapsulation process of the content production device 201. The media processing device 202 can decapsulate the media file resources according to the file format requirements of the point cloud media to obtain a point cloud stream. Then, based on the parameter information of each point cloud frame included in the point cloud stream, the point cloud stream is decoded to obtain point cloud media D'. In specific implementation, the media processing device determines the media file or media segment sequence required for presenting the point cloud media based on the current viewing position / viewing direction of the object; and decodes the media file or media segment sequence required for presenting the point cloud media to obtain the point cloud media required for presentation. Finally, based on the current viewing (viewing window) orientation of the object, the decoded point cloud media is rendered to obtain point cloud frame A', and then presented on the screen of the head-mounted display or any other display device carried by the media processing device according to the presentation time of the point cloud frame. It should be noted that the current viewing position / viewing orientation of the object is determined by head following and possibly visual following functionality. In addition to rendering the point cloud media based on the current object's viewing position / viewing orientation using a renderer, the audio in the current object's viewing (viewing window) orientation can also be decoded and optimized using an audio decoder. During window-based transmission, the current viewing position and viewing orientation are also passed to the strategy module to determine the track to be received.

[0256] It is understood that the point cloud media data processing technology involved in this application can be implemented using cloud technology; for example, using a cloud server as a content production device. Cloud technology refers to a hosting technology that unifies hardware, software, network, and other resources within a wide area network or local area network to achieve data computation, storage, processing, and sharing. The point cloud media data processing technology provided in this application can be applied to various aspects of point cloud compression-related products and immersive systems.

[0257] In this embodiment, the content production device can acquire point cloud media and encode it to obtain a point cloud bitstream. This bitstream includes one or more point cloud frames and parameter information for each frame. The device then encapsulates the bitstream to obtain a media file. The media processing device can then acquire the media file, decapsulate it to obtain another point cloud bitstream, and decode it based on the parameter information of each point cloud frame. This allows for flexible organization of the parameter information of each point cloud frame within the bitstream, guiding the decoding and presentation of the point cloud media, enabling partial decoding at the decoding end, and thus optimizing resource overhead at the decoding end.

[0258] It should be understood that several descriptive fields, including field extensions at the higher-level syntax layer of the bitstream, can be added to the point cloud bitstream to support the implementation steps of this application. The following description, using the field extensions at the higher-level syntax layer of the bitstream as an example, will illustrate the data processing method for point cloud media provided in the embodiments of this application. Please refer to... Figure 3 , Figure 3 This is a flowchart illustrating a point cloud media data processing method provided in an embodiment of this application. The point cloud media data processing method can be executed by a media processing device 202 in a point cloud media data processing system. The method includes the following steps S301-S302:

[0259] S301. Obtain the point cloud bitstream of the point cloud media. The point cloud bitstream contains one or more point cloud frames and parameter information of each point cloud frame.

[0260] In this point cloud stream, one or more point cloud frames are sequentially arranged to form a point cloud sequence. The parameter information of each point cloud frame is frame-level parameter information, which can indicate at least one of the following: frame sequence number, frame timestamp information, frame spatial information, and intra-frame data unit information. The frame sequence refers to the sequence number of the point cloud frame within the point cloud sequence. The frame timestamp information can include any one or more of the point cloud frame's acquisition time, rendering time, and decoding time. The frame spatial information refers to the spatial information of the point cloud frame, which includes at least one of the following: the coordinates of the origin of the bounding box of the point cloud frame, the width of the bounding box of the point cloud frame, and the depth of the bounding box of the point cloud frame. Intra-frame data units can include point cloud frames, point cloud patches, geometric data of point cloud patches, attribute data of point cloud patches, etc.

[0261] In one embodiment, the parameter information of each point cloud frame can be set in the frame header of each point cloud frame, and the syntax of the frame header of each point cloud frame is shown in Table 10:

[0262] Table 10

[0263]

[0264]

[0265] Next, we will describe in detail how the parameter information of the point cloud frame indicates the corresponding information, referring to the syntax of the frame header shown in Table 10. In this embodiment, we will use the current point cloud frame as an example. The current point cloud frame refers to the point cloud frame being decoded in the point cloud bitstream:

[0266] (1) The parameter information of the point cloud frame is used to indicate the frame number.

[0267] The parameter information of the current point cloud frame may include a frame number field (frameIdx), which indicates the sequence number of the current point cloud frame in the point cloud sequence.

[0268] (2) The parameter information of the point cloud frame is used to indicate the information of the data unit within the frame.

[0269] A point cloud frame can contain one or more data units. In one implementation, a data unit can be a point cloud slice within the point cloud frame. The parameter information of the current point cloud frame includes a point cloud slice count field (numSlice) or a point cloud slice count minus one field (numSliceMinus1). The point cloud slice count field indicates the number of point cloud slices contained in the current point cloud frame; the point cloud slice count minus one field indirectly indicates the number of point cloud slices contained in the current point cloud frame, i.e., the number of point cloud slices contained in the current point cloud frame is numSliceMinus1+1.

[0270] (3) The parameter information of the point cloud frame is used to indicate the frame timestamp information.

[0271] The sampling time, decoding time, or rendering time of the point cloud frame can be determined based on the frame's timestamp information, effectively ensuring that the point cloud frame is decoded within an appropriate timeframe. The parameter information of the current point cloud frame includes at least one of the following fields: timestamp flag field (timestampFlag), timestamp quantity field (numTimestamp), timestamp type field (timestampType), and timestamp field (Timestamp).

[0272] The timestamp flag field is used to indicate whether the current point cloud frame contains a timestamp. When the timestamp flag field is a first preset value (such as 0), it indicates that the current point cloud frame does not contain a timestamp; when the timestamp flag field is a second preset value (such as 1), it indicates that the current point cloud frame contains a timestamp.

[0273] The timestamp count field (numTimestamp) indicates the number of timestamps contained in the current point cloud frame.

[0274] The timestamp type field (timestampType) indicates the type of the current timestamp. If the timestamp type field is the first preset value (e.g., 0), it indicates that the current timestamp is a decoding time; if the timestamp type field is the second preset value (e.g., 1), it indicates that the current timestamp is a presentation time; if the timestamp type field is the third preset value (e.g., 2), it indicates that the current timestamp is a capture time. The current timestamp mentioned above refers to the timestamp currently being used by the point cloud frame.

[0275] The timestamp field is used to indicate one or more of the acquisition time, rendering time, and decoding time of the current point cloud frame. As one implementation, the timestamp field indicates Coordinated Universal Time (UTC); as another implementation, the timestamp field indicates the time according to a preset format, which can be a 32-bit string including at least one of the following fields: Year (TimestampYear), Hour (TimestampHours), Minutes (TimestampMinutes), and Frame (TimestampFrames). The Year, Hour, Minute, and Frame timestamp fields are all unsigned integer fields. For example, taking the timestamp field indicating the acquisition time of the current point cloud frame as an example, the preset format of this timestamp field is shown in Table 11:

[0276] Table 11

[0277]

[0278]

[0279] (4) The parameter information of the point cloud frame is used to indicate the frame space information.

[0280] The parameter information of the current point cloud frame includes a spatial information flag field (spatialInfoFlag), which indicates whether the spatial information of the current point cloud frame is indicated. When the spatial information flag field is the first value (e.g., 1), the spatial information of the current point cloud frame is indicated; when the spatial information flag field is the second value (e.g., 0), the spatial information of the current point cloud frame is not indicated.

[0281] The aforementioned spatial information includes at least one of the following: the coordinates of the origin of the bounding box of the current point cloud frame, the width of the bounding box of the current point cloud frame, and the depth of the bounding box of the current point cloud frame. The coordinates of the origin of the bounding box of the current point cloud frame may include the x-coordinate, y-coordinate, and z-coordinate of the origin of the bounding box of the point cloud frame.

[0282] Among them, the high-order part of the x-coordinate of the origin of the point cloud frame bounding box (frame_bounding_box_offset_x_upper): This field is an unsigned integer, which represents the number of bits above 16 bits in the x-coordinate of the bounding box.

[0283] The lower-order part of the x-coordinate of the bounding box origin (frame_bounding_box_offset_x_lower): This field is an unsigned integer representing the lower 16 bits of the x-coordinate of the bounding box origin. Therefore, the x-coordinate of the point cloud frame bounding box origin is:

[0284] frame_bounding_box_offset_x=(frame_bounding_box_offset_x_upper)<<16+frame_bounding_box_offset_x_lower.

[0285] The high-order part of the y-coordinate of the origin of the point cloud frame bounding box (frame_bounding_box_offset_y_upper): This field is an unsigned integer, which represents the number of bits above 16 bits in the y-coordinate of the bounding box.

[0286] The lower 16 bits of the y-coordinate of the bounding box origin in the point cloud frame (frame_bounding_box_offset_y_lower): This field is an unsigned integer representing the lower 16 bits of the y-coordinate of the bounding box origin. The y-coordinate of the point cloud frame bounding box origin is:

[0287] frame_bounding_box_offset_y=(frame_bounding_box_offset_y_upper)<<16+frame_bounding_box_offset_y_lower.

[0288] The high-order part of the z-coordinate of the origin of the point cloud frame bounding box (frame_bounding_box_offset_z_upper): This field is an unsigned integer, representing the number of bits above 16 bits in the bounding box z-coordinate.

[0289] The lower-order part of the z-coordinate of the bounding box origin in the point cloud frame (frame_bounding_box_offset_z_lower): This field is an unsigned integer representing the lower 16 bits of the z-coordinate of the bounding box origin. The z-coordinate of the bounding box origin in the point cloud frame is:

[0290] frame_bounding_box_offset_z=(frame_bounding_box_offset_z_upper)<<16+frame_bounding_box_offset_z_lower.

[0291] The high-order part of the bounding box width field (frame_bounding_box_size_width_upper): This field is an unsigned integer used to represent the number of bits in the bounding box width that are greater than 16 bits.

[0292] The low-order part of the point cloud frame bounding box width field (frame_bounding_box_size_width_lower): This field is an unsigned integer representing the lower 16 bits of the bounding box width. The point cloud frame bounding box width is:

[0293] frame_bounding_box_size_width=(frame_bounding_box_size_width_upper)<<16+frame_bounding_box_size_width_lower.

[0294] The high-order part of the bounding box height field (frame_bounding_box_size_height_upper): This field is an unsigned integer, representing the number of bits above 16 bits in the bounding box height.

[0295] The low-order portion of the point cloud frame bounding box height (frame_bounding_box_size_height_lower): This field is an unsigned integer representing the lower 16 bits of the bounding box height. The point cloud frame bounding box height is:

[0296] frame_bounding_box_size_height=(frame_bounding_box_size_height_upper)<<16+frame_bounding_box_size_height_lower.

[0297] The high-order part of the bounding box depth field (frame_bounding_box_size_depth_upper): This field is an unsigned integer, representing the number of bits above 16 bits in the bounding box depth.

[0298] The low-order part of the point cloud frame bounding box depth field (frame_bounding_box_size_depth_lower): This field is an unsigned integer representing the lower 16 bits of the bounding box depth. The point cloud frame bounding box depth is:

[0299] frame_bounding_box_size_depth=(frame_bounding_box_size_depth_upper)<<16+frame_bounding_box_size_depth_lower.

[0300] (5) The parameter information of the point cloud frame can also be used to indicate whether the point cloud frame contains a parameter set.

[0301] The parameter set includes any one of the following: geometric head parameter set, attribute head parameter set, and a combination of both. In this case, the parameter information of the current point cloud frame may include any one of the following fields: geometric head flag field, attribute head flag field, and attribute count minus one field. The syntax of the parameter information for the point cloud frame is shown in Table 12.

[0302] Table 12

[0303]

[0304] The meanings of each field in Table 12 above are as follows:

[0305] Geometry Header Flag Field (geo_header_flag): This field indicates whether the current point cloud frame contains a geometry head parameter set. If the geometry head flag field is a first preset value (e.g., 1), it indicates that the current point cloud frame contains a geometry head parameter set; if the geometry head flag field is a second preset value (e.g., 0), it indicates that the current point cloud frame does not contain a geometry head parameter set.

[0306] In one implementation, when the current point cloud frame is the first frame in the point cloud sequence, the geometry head flag field must be set to a first preset value. That is, when the current point cloud frame is the first frame in the point cloud sequence, it must contain a set of geometry head parameters. However, when the current point cloud frame is not the first frame in the point cloud sequence, it may not contain a set of geometry head parameters. In this case, the geometry head parameters contained in the first frame can be applied to the current point cloud frame; that is, when decoding the geometric data in the current point cloud frame, the set of geometry head parameters contained in the first frame of the point cloud sequence can be used. In another implementation, when the current point cloud frame is not the first frame in the point cloud sequence, it may still contain a set of geometry head parameters. If the geometry head flag field is set to the first preset value, it indicates that the set of geometry head parameters in the point cloud sequence has been updated. In this case, during decoding, all point cloud frames in the point cloud sequence whose sequence number follows the current point cloud frame use the updated set of geometry head parameters.

[0307] For example, if the point cloud sequence consists of point cloud frame 1, point cloud frame 2, and point cloud frame 3 in sequence, where point cloud frame 1 is the first frame in the sequence, then for point cloud frame 1: the geometry head flag field must be set to the first preset value, meaning point cloud frame 1 must contain the geometry head parameter set. For point cloud frame 2: point cloud frame 2 is not the first frame in the sequence (i.e., it is the second frame), when the geometry head flag field is the first preset value, it indicates that point cloud frame 2 contains the geometry head parameter set, meaning the geometry head parameter set in the point cloud sequence has been updated. For point cloud frame 3: point cloud frame 3 is the third frame in the sequence, when the geometry head flag field is the second preset value, it indicates that point cloud frame 3 does not contain the geometry head parameter set, and when decoding point cloud frame 3, the updated geometry head parameter set needs to be used.

[0308] Attribute header flag field (attr_header_flag): This attribute header flag field is used to indicate whether the current point cloud frame contains an attribute header parameter set. If the attribute header flag field is a first set value (e.g., 1), it indicates that the current point cloud frame contains an attribute header parameter set; if the attribute header flag field is a second set value (e.g., 0), it indicates that the current point cloud frame does not contain an attribute header parameter set.

[0309] In one implementation, when the current point cloud frame is the first frame in the point cloud sequence, the attribute header flag field must be set to a first preset value. That is, when the current point cloud frame is the first frame in the point cloud sequence, it must contain an attribute header parameter set. However, when the current point cloud frame is not the first frame in the point cloud sequence, it may not contain an attribute header parameter set. In this case, the attribute header parameters contained in the first frame take effect on the current point cloud frame; that is, when decoding the attribute data in the current point cloud frame, the attribute header parameter set contained in the first frame of the point cloud sequence can be used. In another implementation, when the current point cloud frame is not the first frame in the point cloud sequence, it may also contain an attribute header parameter set. That is, if the attribute header flag field is set to the first preset value, it indicates that the attribute header parameter set in the point cloud sequence has been updated, and all point cloud frames in the sequence following the current point cloud frame use the updated attribute header parameter set.

[0310] For example, if the point cloud sequence consists of point cloud frame 1, point cloud frame 2, and point cloud frame 3 in sequence, where point cloud frame 1 is the first frame in the sequence, then for point cloud frame 1: the attribute header flag field must be set to the first preset value, meaning point cloud frame 1 must contain the attribute header parameter set. For point cloud frame 2: point cloud frame 2 is not the first frame in the sequence (i.e., it is the second frame), when the attribute header flag field is at the first preset value, it indicates that point cloud frame 2 contains the attribute header parameter set, meaning the attribute header parameter set in the point cloud sequence has been updated. For point cloud frame 3: point cloud frame 3 is the third frame in the sequence, when the attribute header flag field is at the second preset value, it indicates that point cloud frame 3 does not contain the attribute header parameter set, and when decoding point cloud frame 3, the updated attribute header parameter set needs to be used.

[0311] The `numAttributesMinus1` field indicates the number of attribute data groups corresponding to the attribute header parameter set contained in the current point cloud frame. When the current point cloud frame is the first frame in the point cloud sequence, the value of this field is equal to the maximum number of attribute data groups contained in the point cloud sequence. The maximum number of attribute data groups in the point cloud sequence can be understood as the total number of attribute data groups in the point cloud sequence. For example, if the point cloud sequence contains a maximum of 5 attribute data groups, then when the current point cloud frame is the first frame in the point cloud sequence, the value of this field is 5; or, for example, if the point cloud sequence contains a maximum of 10 attribute data groups, then when the current point cloud frame is the first frame in the point cloud sequence, the value of this field is 10.

[0312] In one embodiment, when a point cloud frame other than the first frame contains a geometric header parameter set or an attribute header parameter set, it indicates that the geometric header parameter set or attribute header parameter set has been updated. The updated geometric header parameter set or attribute header parameter set can replace the corresponding geometric header parameter set and attribute header parameter set in the point cloud sequence. In point cloud media, a set of attribute data corresponds to one attribute header parameter set; a point cloud frame can contain one or more attribute header parameter sets, and an attribute header parameter set can correspond to one or more sets of attribute data. In particular, for attribute header parameter sets, since different sets of attribute data will correspond to different attribute header parameter sets, the parameters in the attribute header parameter set may be partially updated. Therefore, to address this situation, this embodiment of the application can employ at least the following three methods (i.e., method A, method B, and method C) to establish the correspondence between the attribute header parameter set and the attribute data:

[0313] Method A: When a point cloud frame contains only one set of attribute header parameters, meaning there is only one set of attribute header parameters within the point cloud frame, the attribute data groups in the point cloud media are all referenced during decoding. The attribute header parameter set may include an attribute presence flag field. The syntax of this attribute header parameter set is shown in Table 13.

[0314] Table 13

[0315]

[0316] The attribute presence flag field (attributePresentFlag[attrIdx]) is a binary variable. It indicates whether the current attribute header parameter set contains the encoded parameters for the x-th (or attrIdx) group of attribute data. If the attribute presence flag field is the first value (e.g., 1), it indicates that the current attribute header parameter set contains the encoded parameters for the x-th group of attribute data; if the attribute presence flag field is the second value (e.g., 0), it indicates that the current attribute header parameter set does not contain the encoded parameters for the x-th group of attribute data. x (or attrIdx) is an integer greater than or equal to 0 and less than or equal to 15, and its meaning is shown in Table 14 below.

[0317] Table 14 x Attribute Encoding Mapping Table

[0318] 0 Color 1 Reflectance 2..15 reserve

[0319] Method B: When a point cloud frame contains only one set of attribute header parameters (i.e., there is only one set of attribute header parameters within the point cloud frame), the correspondence between attribute encoding parameters and attribute data can be established through the attribute type field and the attribute data identifier. The attribute header parameter set includes an attribute type field and an attribute data identifier field, which represent the correspondence between the attribute header parameter set and the attribute data. In this case, the syntax of the attribute header parameter set is shown in Table 15:

[0320] Table 15

[0321]

[0322] The meanings of each field in Table 15 are as follows:

[0323] The attribute count minus one field (numAttributesMinus1) indicates the number of attribute data groups corresponding to the attribute header parameter set contained in the current point cloud frame. When the current point cloud frame is the first frame in the point cloud sequence, the attribute count minus one field is the maximum number of attribute data groups contained in the point cloud sequence.

[0324] Attribute Encoding Parameter Type Field (attributeType): This field can be used to indicate the type of the attribute header parameter set.

[0325] Attribute data identifier field (attributeID): This field is used to indicate attribute data.

[0326] Method C: When a point cloud frame contains multiple attribute header parameter sets, such as `numAttributesMinus1`, different attribute data in the point cloud frame may refer to different attribute header parameter sets during decoding. For example, if `numAttributesMinus1 = 2`, the point cloud frame contains two attribute header parameter sets: attribute header parameter set 1 and attribute header parameter set 2. Attribute data 1 in the point cloud frame is decoded by referring to attribute header parameter set 1, and attribute data 2 is decoded by referring to attribute header parameter set 2. In this case, the correspondence between attribute encoding parameters and attribute data can be established through the attribute type field and the attribute data identifier field. The attribute header parameter set includes an attribute type field and an attribute data identifier field, which represent the correspondence between the attribute header parameter set and the attribute data. The syntax of an attribute header parameter set is shown in Table 16.

[0327] Table 16

[0328]

[0329] (6) The parameter information of the point cloud frame can also be used to indicate the start code and end code of the data unit within the frame, so as to achieve rapid positioning.

[0330] A point cloud frame can include one or more data units, which may include point cloud frames, point cloud patches, geometric data of point cloud patches, attribute data of point cloud patches, and so on. For example, Figure 4 As shown, Figure 4 A point cloud frame, provided as an example in this application, includes data units. Figure 4 In this context, a point cloud frame can include data units 41, 42, 43, 421, 422, 423, 431, 432, and 433. Data unit 41 is the point cloud frame itself; specifically, data unit 41 includes the point cloud frame header, a geometric header parameter set, an attribute header parameter set, and a point cloud patch. Data unit 42 refers to point cloud patch 1. Data unit 43 refers to point cloud patch 2. Data unit 421 refers to the geometric data of point cloud patch 1; data unit 422 refers to attribute data 1 (attr1) of point cloud patch 1; data unit 423 refers to attribute data 2 (attr2) of point cloud patch 1. Data unit 431 refers to the geometric data of point cloud patch 2; data unit 432 refers to attribute data 1 (attr1) of point cloud patch 2; data unit 433 refers to attribute data 2 (attr2) of point cloud patch 2.

[0331] The following section explains how the parameter information of a point cloud frame indicates the start and end codes of data units within the frame:

[0332] ① If the data unit includes point cloud frames, the parameter information of the current point cloud frame may include the start code and end code of the current point cloud frame; the current point cloud frame refers to the point cloud frame being decoded in the point cloud bitstream; wherein, the start code of the current point cloud frame is a preset specific bit string, or the start code of the current point cloud frame is the end code of the previous point cloud frame; the start code of the current point cloud frame is used to indicate the position where the current point cloud frame starts decoding, and the end code of the current point cloud frame is used to indicate the position where the current point cloud frame ends decoding.

[0333] ② If the data unit includes point cloud slices, the parameter information of the current point cloud frame includes the start code and end code of the current point cloud slice in the current point cloud frame. The current point cloud slice refers to the point cloud slice being decoded in the point cloud bitstream. The start code of the current point cloud slice is used to indicate the position where the current point cloud slice begins decoding, and the end code of the current point cloud slice is used to indicate the position where the current point cloud slice ends decoding.

[0334] Here, the start code of the current point cloud patch is a preset specific bit string, or the start code of the current point cloud patch is the end code of the previous point cloud patch. For example, in Figure 4In the diagram, the current point cloud patch is point cloud patch 2. The start code of point cloud patch 2 can be the end code of the previous point cloud patch (i.e., ...). Figure 4 (End code of midpoint cloud 1).

[0335] ③ The point cloud patch may include the geometric data of the point cloud frame. If the data unit includes the geometric data of the point cloud patch, then the parameter information of the current point cloud frame includes the start code and end code of the geometric data of the current point cloud patch. The geometric data of the current point cloud patch refers to the geometric data of the current point cloud patch that is being decoded. The start code of the geometric data of the current point cloud patch is used to indicate the position where the decoding of the geometric data of the current point cloud patch begins, and the end code of the geometric data of the current point cloud patch is used to indicate the position where the decoding of the geometric data of the current point cloud patch ends.

[0336] As one optional implementation, the start code of the geometric data of the current point cloud patch can be a preset specific bit string. As another optional implementation, the start code of the geometric data of the current point cloud patch can be the end code of the previous point cloud patch. For example, as... Figure 4 As shown, the geometric data of the current point cloud patch is the geometric data of point cloud patch 2. The start code of the geometric data of the current point cloud patch can be the end code of the previous point cloud patch (i.e., point cloud patch 1) of point cloud patch 2. Figure 4 (End code of midpoint cloud 1).

[0337] ④ The point cloud patch may also include one or more sets of attribute data for the point cloud frame. If the data unit includes attribute data of the point cloud patch, the parameter information of the current point cloud frame includes the start code and end code of the current attribute data of the current point cloud patch, where the current attribute data of the current point cloud patch refers to the attribute data being decoded in the current point cloud patch. The start code of the current attribute data of the current point cloud patch is used to indicate the position where the decoding of the current attribute data of the current point cloud patch begins, and the end code of the current attribute data of the current point cloud patch is used to indicate the position where the decoding of the current attribute data of the current point cloud patch ends.

[0338] As an optional approach, the start code of the current attribute data of the current point cloud patch can be a preset specific bit string. Alternatively, the start code of the current attribute data of the current point cloud patch can be the end code of the previous attribute data in the current point cloud patch. For example, as... Figure 4 As shown, if the current point cloud patch is point cloud patch 1, and the current attribute data of the current point cloud patch is attribute data 2 in point cloud patch 1, then the previous attribute data can be attribute data 1 in point cloud patch 1; correspondingly, the start code of the current attribute data of the current point cloud patch is the end code of attribute data 1 in point cloud patch 1.

[0339] As another alternative, the start code of the current attribute data of the current point cloud patch can be the end code of the geometric data of the current point cloud patch. For example, as... Figure 4 As shown, if the current point cloud patch is point cloud patch 1, and the current attribute data of the current point cloud patch is attribute data 1 in point cloud patch 1, then the start code of the current attribute data of the current point cloud patch is the end code of the geometric data in point cloud patch 1.

[0340] It should be understood that in a bitstream conforming to this bitstream, the aforementioned specific bit string should not appear in any of the cases described in ①-④ above, except for the start code. The aforementioned start code is a specific bit string composed of a start code prefix and a start code value, and all start codes should be byte-aligned. The start code prefix can be the bit string '0000 0000 0000 0000 00000001', and the start code value is an 8-bit integer used to represent the type of start code. For example, the start code for a point cloud patch could be 0000 0000 0000 0000 0000 000100; another example is that the start code for the geometric data of a point cloud patch could be 0000 00000000 0000 0000 000180. The start code values ​​are shown in Table 17:

[0341] Table 17 Starting Code Values

[0342] Slice start code (slice_start_code) 00～7F Geometric data start code of point cloud 80～87 Starting code of attribute data for point cloud patches 88～8E Slice end code (slice_end_code) 8F reserve 90~AF Point cloud sequence start code (sequence_start_code) B0 Point cloud sequence end code (sequence_end_code) B1 Point cloud frame start code (frame_start_code) B2 Point cloud frame end code (frame_end_code) B3 Geometric data end code of point cloud B4 End code of attribute data for point cloud patch B5 reserve B6~FF

[0343] Optionally, the start code of the current attribute data of the current point cloud patch can be used to indicate the type of the current attribute data of the current point cloud patch. For example, the start code 88 of the current attribute data of the current point cloud patch is used to indicate that the type of the current attribute data of the current point cloud patch is color; the start code 89 of the current attribute data of the current point cloud patch is used to indicate that the type of the current attribute data of the current point cloud patch is reflectance; the start code 8A of the current attribute data of the current point cloud patch is used to indicate that the type of the current attribute data of the current point cloud patch is color with cross-attribute prediction enabled; and the start code 8B of the current attribute data of the current point cloud patch is used to indicate that the type of the current attribute data of the current point cloud patch is reflectance with cross-attribute prediction enabled.

[0344] S302. Based on the parameter information of each point cloud frame, decode the point cloud bitstream to present the point cloud media.

[0345] A point cloud frame includes one or more data units. Decoding the point cloud stream to present point cloud media based on the parameter information of each point cloud frame may include: determining the target point cloud frame to be decoded from the point cloud stream; locating the data units to be decoded in the target point cloud frame according to the parameter information of the target point cloud frame; and decoding and presenting the data units to be decoded in the target point cloud frame. Specifically, the parameter information of the target point cloud frame includes the start code and end code of the data units within the frame. Based on the start code and end code of the data units within the frame, the data units to be decoded can be quickly located, thus enabling partial decoding.

[0346] In one embodiment, the parameter information of the target point cloud frame is used to indicate timestamp information and frame sequence number. The target point cloud frame to be decoded can be determined from the point cloud bitstream based on the frame sequence number, and the target point cloud frame can be decoded based on the timestamp information indicated by the parameter information of the target point cloud frame, thereby realizing the decoding and presentation of point cloud media in sequence at an appropriate time.

[0347] In one embodiment, the parameter information of the target point cloud frame is used to indicate whether the target point cloud frame contains a parameter set. The corresponding parameter set is obtained based on the parameter information of the target point cloud frame, and then the target point cloud frame is decoded based on the parameter set. By flexibly indicating the parameter set, the correct decoding of the point cloud frame can be effectively guaranteed. At the same time, it can also reduce the unnecessary burden on the decoding side caused by each point cloud frame carrying a geometric header parameter set and an attribute header parameter set when the parameter set is relatively fixed.

[0348] In this embodiment of the application, a point cloud bitstream of point cloud media is obtained. The point cloud bitstream contains one or more point cloud frames and parameter information of each point cloud frame. Based on the parameter information of each point cloud frame, the point cloud bitstream is decoded to present the point cloud media. This method can flexibly organize the parameter information in the point cloud bitstream to guide the decoding and presentation of the point cloud media, thereby optimizing the resource overhead of the decoding end.

[0349] Please see Figure 5 , Figure 5 This is a flowchart illustrating another point cloud media data processing method provided in this application embodiment. This point cloud media data processing method can be executed by the content creation device 201 in the aforementioned point cloud media data processing system. The point cloud media data processing method described in this embodiment may include the following steps S501-S503:

[0350] S501, Obtain point cloud media.

[0351] The specific implementation of step S501 can be found in the corresponding description above, and will not be repeated here.

[0352] S502. Encode the point cloud media to obtain a point cloud bitstream. The point cloud bitstream contains one or more point cloud frames and parameter information for each point cloud frame.

[0353] When encoding point cloud media, the parameter information of each point cloud frame can be set in the frame header of each point cloud frame. The parameter information of the point cloud frame is used to indicate at least one of the following: frame sequence number, frame timestamp information, frame spatial information, and intra-frame data unit information. The following uses the current point cloud frame as an example to illustrate the information indicated by the parameter information. The current point cloud frame refers to the point cloud frame that is being encoded.

[0354] (1) The parameter information of the point cloud frame is used to indicate the frame number.

[0355] One or more point cloud frames in a point cloud bitstream form a point cloud sequence in sequence. The parameter information of the current point cloud frame includes a frame sequence number field, which indicates the sequence number of the current point cloud frame in the point cloud sequence.

[0356] (2) The parameter information of the point cloud frame is used to indicate the frame timestamp information.

[0357] The parameter information of the current point cloud frame includes at least one of the following fields: timestamp flag field, timestamp quantity field, timestamp type field, and timestamp field.

[0358] The timestamp flag field is used to indicate whether the current point cloud frame contains a timestamp. When the current point cloud frame does not contain a timestamp, the timestamp flag field is set to the first preset value (e.g., 0); when the current point cloud frame contains a timestamp, the timestamp flag field is set to the second preset value (e.g., 1).

[0359] The timestamp count field is used to indicate the number of timestamps contained in the current point cloud frame.

[0360] The timestamp type field is used to indicate the type of the current timestamp. If the current timestamp type is encoding time, the timestamp type field is set to the first preset value (e.g., 0); if the current timestamp type is presentation time, the timestamp type field is set to the second preset value (e.g., 1); if the current timestamp type is acquisition time, the timestamp type field is set to the third preset value (e.g., 2).

[0361] The timestamp field indicates one or more of the acquisition time, rendering time, and encoding time of the current point cloud frame. As one implementation, the timestamp field indicates Universal Time Coordinated (UTC). As another implementation, the timestamp field indicates the time according to a preset format, which includes at least one of the following fields: year timestamp, hour timestamp, second timestamp, and frame timestamp. The current timestamp refers to the timestamp of the current point cloud frame in use.

[0362] (3) The parameter information of the point cloud frame is used to indicate the frame space information.

[0363] The parameter information of the current point cloud frame includes a spatial information flag field; the spatial information flag field indicates whether the spatial information of the current point cloud frame is indicated. When the spatial information flag field indicates the spatial information of the current point cloud frame, the spatial information flag field is set to a first value (e.g., 1); when the spatial information flag field does not indicate the spatial information of the current point cloud frame, the spatial information flag field is set to a second value (e.g., 0). The spatial information includes at least one of the following: the coordinates of the origin of the bounding box of the current point cloud frame, the width of the bounding box of the current point cloud frame, and the depth of the bounding box of the current point cloud frame.

[0364] (4) The parameter information of the point cloud frame is used to indicate the information of the data unit within the frame.

[0365] A point cloud frame contains one or more data units, each of which includes a point cloud patch. The parameter information of the current point cloud frame includes a point cloud patch count field or a point cloud patch count minus one field. The point cloud patch count field indicates the number of point cloud patches contained in the current point cloud frame. The point cloud patch count minus one field indirectly indicates the number of point cloud patches contained in the current point cloud frame.

[0366] (5) The parameter information of the point cloud frame is used to indicate the start code and end code of the data unit within the frame. The data unit may include the point cloud frame itself, the point cloud patch, the geometric data of the point cloud patch, the attribute data of the point cloud patch, etc. The geometric data of the point cloud patch refers to the geometric data contained in the point cloud patch, and the attribute data of the point cloud patch can be understood as the attribute data of the point cloud patch.

[0367] ① If the data unit includes a point cloud frame, the parameter information of the current point cloud frame may include the start code and end code of the current point cloud frame; the current point cloud frame refers to the point cloud frame being encoded in the point cloud bitstream; wherein, the start code of the current point cloud frame is a preset specific bit string, or the start code of the current point cloud frame is the end code of the previous point cloud frame; wherein, the start code of the current point cloud frame is used to indicate the position where the current point cloud frame begins encoding, and the end code of the current point cloud frame is used to indicate the position where the current point cloud frame ends encoding.

[0368] ② If the data unit includes point cloud patches, the parameter information of the current point cloud frame includes the start code and end code of the current point cloud patch in the current point cloud frame. The current point cloud patch refers to the point cloud patch being encoded in the point cloud bitstream. The start code of the current point cloud patch is a preset specific bit string, or the start code of the current point cloud patch is the end code of the previous point cloud patch. The start code of the current point cloud patch indicates the position where encoding of the current point cloud patch begins, and the end code of the current point cloud patch indicates the position where encoding of the current point cloud patch ends.

[0369] ③ If the data unit includes the geometric data of a point cloud patch, then the parameter information of the current point cloud frame includes the start code and end code of the geometric data of the current point cloud patch. The geometric data of the current point cloud patch refers to the geometric data of the current point cloud patch that is being encoded. The start code of the geometric data of the current point cloud patch is used to indicate the position where the encoding of the geometric data of the current point cloud patch begins, and the end code of the geometric data of the current point cloud patch is used to indicate the position where the encoding of the geometric data of the current point cloud patch ends.

[0370] As one optional implementation, the start code of the geometric data of the current point cloud patch can be a preset specific bit string. As another optional implementation, the start code of the geometric data of the current point cloud patch can be the end code of the previous point cloud patch.

[0371] ④ A point cloud patch may contain one or more sets of attribute data. If a data unit includes the attribute data of a point cloud patch, then the parameter information of the current point cloud frame includes the start code and end code of the attribute data of the current point cloud patch. The attribute data of the current point cloud patch refers to the attribute data being encoded in the point cloud patch before the current point cloud patch. The start code of the current attribute data of the current point cloud patch is used to indicate the position where the encoding of the current attribute data of the current point cloud patch begins, and the end code of the current attribute data of the current point cloud patch is used to indicate the position where the encoding of the current attribute data of the current point cloud patch ends.

[0372] As one optional implementation, the start code of the current attribute data of the current point cloud patch is a preset specific bit string. As another optional implementation, the start code of the current attribute data of the current point cloud patch is the end code of the previous attribute data in the current point cloud patch. As yet another optional implementation, the start code of the current attribute data of the current point cloud patch is the end code of the geometric data of the current point cloud patch.

[0373] In one embodiment, the start code of the current attribute data of the current point cloud patch is also used to indicate the type of the current attribute data of the current point cloud patch.

[0374] (6) The parameter information of the point cloud frame is also used to indicate whether the point cloud frame contains a parameter set.

[0375] The parameter set can include any of the following: geometry header parameter set, attribute header parameter set, and a combination of both. The parameter information of the current point cloud frame can include any of the following fields: geometry header flag field, attribute header flag field, and attribute count minus one field.

[0376] The geometry head flag field is used to indicate whether the current point cloud frame contains a geometry head parameter set. If the current point cloud frame contains a geometry head parameter set, the geometry head flag field is set to a first preset value (e.g., 1); if the current point cloud frame does not contain a geometry head parameter set, the geometry head flag field is set to a second preset value (e.g., 0).

[0377] In one embodiment, when the current point cloud frame is the first frame in the point cloud sequence, the geometry head identifier field is set to a first preset value. In another embodiment, when the current point cloud frame is not the first frame in the point cloud sequence, and the set of geometry head parameters in the point cloud sequence has been updated, the geometry head identifier field is set to the first preset value. In yet another embodiment, when the current point cloud frame is not the first frame in the point cloud sequence, and the set of geometry head parameters in the point cloud sequence has not been updated, the geometry head identifier field is set to a second preset value.

[0378] The attribute header field is used to indicate whether the current point cloud frame contains an attribute header parameter set. If the current point cloud frame contains an attribute header parameter set, the attribute header flag field is set to the first set value (e.g., 1); if the current point cloud frame does not contain an attribute header parameter set, the attribute header flag field is set to the second set value (e.g., 0).

[0379] In one embodiment, when the current point cloud frame is the first frame in the point cloud sequence, the attribute header flag field is set to a first preset value. In another embodiment, when the current point cloud frame is not the first frame in the point cloud sequence, and the attribute header parameter set in the point cloud sequence has been updated, the attribute header flag field is set to the first preset value. In yet another embodiment, when the current point cloud frame is not the first frame in the point cloud sequence, and the attribute header parameter set in the point cloud sequence has not been updated, the attribute header flag field is set to a second preset value.

[0380] The "Attribute Quantity -1" field indicates the number of attribute data groups corresponding to the attribute header parameter set contained in the current point cloud frame. When the current point cloud frame is the first frame in the point cloud sequence, the value of the "Attribute Quantity -1" field is set to the maximum number of attribute data groups contained in the point cloud sequence.

[0381] In point cloud media, a set of attribute data corresponds to a set of attribute header parameters; a point cloud frame can contain one or more sets of attribute header parameters, and a set of attribute header parameters can correspond to one or more sets of attribute data. Regarding the attribute header parameter sets, since different sets of attribute data correspond to different sets of attribute header parameters, and the parameters in the attribute header parameter sets may be partially updated, this application embodiment can employ at least the following three methods (i.e., method A, method B, and method C) to establish the correspondence between the attribute header parameter sets and the attribute data:

[0382] Method A: When a point cloud frame contains only one set of attribute header parameters (i.e., there is only one set of attribute header parameters within the point cloud frame), the attribute data groups in the point cloud media are encoded with reference to this set of attribute header parameters. In point cloud media, one set of attribute data corresponds to one set of attribute header parameters, and a point cloud frame contains one or more sets of attribute header parameters. When a point cloud frame contains only one set of attribute header parameters (i.e., there is only one set of attribute header parameters within the point cloud frame), this set of attribute header parameters includes an attribute presence flag field. This attribute presence flag field indicates whether the current attribute header parameter set contains the encoding parameters for the x-th set of attribute data, where x is an integer greater than or equal to 0 and less than or equal to 15. If the current attribute header parameter set contains the encoding parameters for the x-th set of attribute data, the attribute presence flag field is set to the first value; if the current attribute header parameter set does not contain the encoding parameters for the x-th set of attribute data, the attribute presence flag field is set to the second value.

[0383] Method B: When a point cloud frame contains only one set of attribute header parameters, that is, the number of attribute header parameter sets in the point cloud frame is only one, the attribute header parameter set includes an attribute type field and an attribute data identifier field, which represent the correspondence between the attribute header parameter set and the attribute data. The attribute quantity minus one field is used to indicate the number of attribute data groups corresponding to the attribute header parameter set contained in the current point cloud frame. Among them, when the current point cloud frame is the first frame in the point cloud sequence, the value of the attribute quantity minus one field is set to the maximum number of attribute data groups contained in the point cloud sequence.

[0384] Method C: When a point cloud frame contains multiple attribute header parameter sets, the point cloud frame contains one attribute header parameter set (numAttributesMinus1). The correspondence between attribute encoding parameters and attribute data can be established through the attribute type field and the attribute data identifier field. Any attribute header parameter set includes an attribute type field and an attribute data identifier field, which represent the correspondence between the attribute header parameter set and the attribute data.

[0385] S503, transmit point cloud code stream.

[0386] In this embodiment, point cloud media is acquired and encoded to obtain a point cloud bitstream. The point cloud bitstream contains one or more point cloud frames and parameter information of each point cloud frame. Transmitting the point cloud bitstream can flexibly organize the parameter information in the point cloud bitstream, guide the decoding and presentation of the point cloud media, and optimize the resource overhead of the decoding end.

[0387] The following is a detailed explanation of the point cloud media data processing method provided in this application through a complete example:

[0388] 1. The content production equipment can acquire point cloud media and encode it to obtain a point cloud bitstream. This point cloud bitstream includes one or more point cloud frames. For each point cloud frame in the point cloud bitstream:

[0389] 1) Define frame-level parameter information, including frame number, frame timestamp, frame spatial information, and information about intra-frame data units; 2) Indicate in the frame-level parameter information whether the current frame contains a geometry header or attribute header parameter set; 3) Add corresponding start and end codes to the data units within the point cloud frame. For example, assume the parameter information of each point cloud frame in the point cloud bitstream is as follows:

[0390] Frame1: frameIdx=1; numSlice=2; timestampFlag=1; numTimestamp=1; timestampType=2; Timestamp=2022-09-05T16:24:05.023Z; spatialInfoFlag=0; geo_header_flag=1; attr_header_flag=1; numAttributesMinus1=1.

[0391] Wherein, frameIdx=1 indicates that point cloud frame 1 has the sequence number 1 in the point cloud sequence (i.e., it is the first frame in the point cloud sequence), numSlice=2 indicates that point cloud frame 1 contains 2 point cloud slices, timestampFlag=1 indicates that point cloud frame 1 contains a timestamp, numTimestamp=1 indicates that point cloud frame 1 contains 1 timestamp, timestampType=2 indicates that the timestamp contained in point cloud frame 1 is the sampling time, which is 2022-09-05T16:24:05.023Z; spatialInfoFlag=0 indicates that the spatial information of point cloud frame 1 is not indicated; geo_header_flag=1 indicates that point cloud frame 1 contains a set of geometric header parameters; attr_header_flag=1 indicates that point cloud frame 1 contains a set of attribute header parameters, and numAttributesMinus1=1 indicates that the maximum number of attribute data groups contained in the point cloud sequence is 1.

[0392] Frame2~Frame99: frameIdx values ​​range from 2 to 99, numSlice = 2; timestampFlag = 1; numTimestamp = 1; timestampType = 2; Timestamp is the corresponding UTC time; spatialInfoFlag = 0; geo_header_flag = 0; attr_header_flag = 0.

[0393] Among them, point cloud frames 2 to 99 (i.e., Frame2 to Frame99) are numbered 2 to 99 in the point cloud sequence. For any point cloud frame among point cloud frames 2 to 99: numSlice = 2 indicates that the point cloud frame contains 2 point cloud slices, timestampFlag = 1 indicates that the point cloud frame contains a timestamp, numTimestamp = 1 indicates that the point cloud frame contains 1 timestamp, spatialInfoFlag = 0 indicates that the spatial information of the point cloud frame is not indicated; geo_header_flag = 0 indicates that the point cloud frame does not contain a geometric header parameter set; attr_header_flag = 0 indicates that the point cloud frame does not contain an attribute header parameter set. At this time, when decoding any point cloud frame among Frame2 to Frame99, the geometric header parameter set or the attribute header parameter set in point cloud frame 1 (Frame1) can be obtained.

[0394] Frame100: frameIdx = 100; numSlice = 2; timestampFlag = 1; numTimestamp = 1; timestampType = 2; Timestamp is the corresponding UTC time; spatialInfoFlag = 0; geo_header_flag = 1; attr_header_flag = 1; numAttributesMinus1 = 0.

[0395] In this sequence, `frameIdx = 100` indicates that point cloud frame 100 has the sequence number 100 in the point cloud sequence; `numSlice = 2` indicates that point cloud frame 100 contains 2 point cloud slices; `timestampFlag = 1` indicates that point cloud frame 100 contains a timestamp; `numTimestamp = 1` indicates that point cloud frame 100 contains 1 timestamp; `timestampType = 2` indicates that the timestamp in point cloud frame 100 is the sampling time; `spatialInfoFlag = 0` indicates that spatial information of point cloud frame 100 is not indicated; `geo_header_flag = 1` indicates that point cloud frame 100 contains a geometric header parameter set; `attr_header_flag = 1` indicates that point cloud frame 100 contains an attribute header parameter set; `numAttributesMinus1 = 0` indicates the number of attribute data groups corresponding to the attribute header parameter set contained in point cloud frame 100, i.e., the number of attribute data groups corresponding to the attribute header parameter set contained in point cloud frame 100 is 1. At this time, the attribute header parameter set and geometric header parameter set in the point cloud sequence are updated.

[0396] Frame101~Frame200: frameIdx values ​​correspond to 101~200, numSlice=2; timestampFlag=1; numTimestamp=1; timestampType=2; Timestamp is the corresponding UTC time; spatialInfoFlag=0; geo_header_flag=0; attr_header_flag=0.

[0397] In this sequence, point cloud frames 101 to 200 (i.e., Frame 101 to Frame 200) are numbered 101 to 200 respectively. For any point cloud frame among Frame 101 to Frame 200: numSlice = 2 indicates that the point cloud frame contains 2 point cloud slices; timestampFlag = 1 indicates that the point cloud frame contains a timestamp; numTimestamp = 1 indicates that the point cloud frame contains 1 timestamp; spatialInfoFlag = 0 indicates that the spatial information of the point cloud frame is not indicated; geo_header_flag = 0 indicates that the point cloud frame does not contain a geometric header parameter set; attr_header_flag = 0 indicates that the point cloud frame does not contain an attribute header parameter set. At this time, when decoding any point cloud frame among Frame 101 to Frame 200, the geometric header parameter set or the attribute header parameter set in point cloud frame 100 (Frame 100) can be obtained (i.e., the updated geometric header parameter set and attribute header parameter set are used).

[0398] 2. Content production equipment can transmit point cloud bitstreams to media processing equipment.

[0399] 3. When the media processing device decodes the point cloud bitstream: ① It can correctly decode the geometric and attribute data of point cloud frames 1-99 based on the geometric header parameter set and attribute header parameter set indicated in frame1. Then, the geometric header parameter set and attribute header parameter set contained in frame1 remain in effect until frame100 updates the geometric header parameter set and the attribute parameter set of a set of attribute data. Then, it can correctly decode point cloud frames 100-200 based on the updated geometric header parameter set and the updated attribute parameter set of a set of attribute data in frame100.

[0400] ② The timestamp information indicated by the parameter information of each point cloud frame can be obtained from the point cloud code stream. Then, the acquisition time of each point cloud frame can be obtained according to the timestamp information carried in each point cloud frame. Each point cloud frame can be displayed at an appropriate time according to the sampling time of each point cloud frame. The point cloud map can be updated in real time in scenarios such as autonomous driving through the sampling time of each point cloud frame.

[0401] ③ The parameter information of each point cloud frame can indicate the start and end codes of the data units within the frame. The media processing device can quickly locate the required data units based on the start and end codes of the data units within the frame indicated by the parameter information of each point cloud frame, thereby achieving partial decoding. For example, in each point cloud frame, there are two point cloud patches. The parameter information of the point cloud frame can indicate the start and end codes of each point cloud patch. When only the spatial area corresponding to a point cloud patch needs to be consumed, the corresponding point cloud patch can be selected for partial decoding based on the start and end codes of the point cloud patch indicated by the parameter information of each point cloud frame, such as selecting the corresponding point cloud patch for partial decoding based on the start codes 00 to 7F. For another example, when the data unit includes the attribute data of a point cloud patch, the start code of the attribute data of that point cloud patch is 88. The start code 88 of the attribute data of that point cloud patch is used to indicate that the type of the attribute data of that point cloud patch is color. When only color attribute data needs to be presented, the color attribute data can be selected for partial decoding based on the start code 88 of the attribute data of the point cloud patch.

[0402] In this embodiment, the content production device can acquire point cloud media and encode the point cloud media to obtain a point cloud bitstream; and encapsulate the point cloud bitstream to obtain a media file of the point cloud media; then the media processing device can acquire the media file and decode the media file to obtain a point cloud bitstream. It can flexibly organize the parameter information of each point cloud frame in the point cloud bitstream to guide the transmission, decoding and presentation of the point cloud media, realize partial decoding at the decoding end, and thus optimize the resource overhead of the decoding end.

[0403] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a point cloud media data processing device provided in an embodiment of this application. The point cloud media data processing device can be installed in the computer equipment provided in the embodiment of this application. The computer equipment can be the media processing equipment mentioned in the above method embodiment. Figure 6 The data processing device for point cloud media shown can be a computer program (including program code) running on a computer device, and the data processing device for point cloud media can be used to execute... Figure 3 Some or all of the steps in the method embodiments shown. Please refer to [link / reference]. Figure 6 The data processing device for the point cloud media may include the following units:

[0404] The acquisition unit 601 is used to acquire the point cloud bitstream of the point cloud media. The point cloud bitstream includes one or more point cloud frames and parameter information of each point cloud frame. The parameter information of the point cloud frame is used to indicate at least one of the following: frame sequence number, frame timestamp information, frame spatial information, and information of intra-frame data units.

[0405] The processing unit 602 is used to decode the point cloud bitstream according to the parameter information of each point cloud frame to present the point cloud media.

[0406] In one embodiment, the parameter information of the point cloud frame is used to indicate the frame sequence number; one or more point cloud frames in the point cloud bitstream constitute a point cloud sequence in sequence.

[0407] The parameter information of the current point cloud frame includes a frame sequence number field, which indicates the sequence number of the current point cloud frame in the point cloud sequence.

[0408] The current point cloud frame refers to the point cloud frame being decoded in the point cloud bitstream.

[0409] In one embodiment, the parameter information of the point cloud frame is used to indicate the frame timestamp information; the parameter information of the current point cloud frame includes at least one of the following fields: timestamp flag field, timestamp quantity field, timestamp type field, and timestamp field;

[0410] The timestamp flag field is used to indicate whether the current point cloud frame contains a timestamp. When the timestamp flag field is a first preset value, it indicates that the current point cloud frame does not contain a timestamp; when the timestamp flag field is a second preset value, it indicates that the current point cloud frame contains a timestamp.

[0411] The timestamp count field is used to indicate the number of timestamps contained in the current point cloud frame;

[0412] The timestamp type field is used to indicate the type of the current timestamp. If the timestamp type field is the first preset value, it indicates that the current timestamp is the decoding time; if the timestamp type field is the second preset value, it indicates that the current timestamp is the presentation time; if the timestamp type field is the third preset value, it indicates that the current timestamp is the acquisition time.

[0413] The timestamp field is used to indicate any one or more of the acquisition time, presentation time, and decoding time of the current point cloud frame; the timestamp field is used to indicate Coordinated Universal Time, or the timestamp field is indicated according to a preset format, the preset format including at least one of the following fields: year timestamp field, hour timestamp field, second timestamp field, and frame timestamp field;

[0414] Here, the current point cloud frame refers to the point cloud frame being decoded in the point cloud bitstream, and the current timestamp refers to the timestamp of the current point cloud frame being used.

[0415] In one embodiment, the parameter information of the point cloud frame is used to indicate the frame spatial information; the parameter information of the current point cloud frame includes a spatial information flag field.

[0416] The spatial information flag field indicates whether to indicate the spatial information of the current point cloud frame. When the spatial information flag field is the first value, it indicates the spatial information of the current point cloud frame; when the spatial information flag field is the second value, it does not indicate the spatial information of the current point cloud frame.

[0417] Spatial information includes at least one of the following: coordinates of the origin of the bounding box of the current point cloud frame, width of the bounding box of the current point cloud frame, and depth of the bounding box of the current point cloud frame.

[0418] In one embodiment, the parameter information of a point cloud frame is used to indicate the information of the data unit within the frame; a point cloud frame contains one or more data units, and the data unit includes point cloud patches; the parameter information of the current point cloud frame includes a point cloud patch count field or a point cloud patch count minus one field.

[0419] The point cloud patch quantity field is used to indicate the number of point cloud patches contained in the current point cloud frame;

[0420] The "Number of Point Cloud Pieces - One" field is used to indirectly indicate the number of point cloud pieces contained in the current point cloud frame.

[0421] The current point cloud frame refers to the point cloud frame being decoded in the point cloud bitstream.

[0422] In one embodiment, the parameter information of the point cloud frame is also used to indicate whether the point cloud frame contains a parameter set, which includes any of the following: a geometry head parameter set, an attribute head parameter set, a geometry head parameter set, and an attribute head parameter set.

[0423] The parameter information of the current point cloud frame includes any one of the following fields: geometry header flag field, attribute header flag field, and attribute count minus one field;

[0424] The geometry head flag field is used to indicate whether the current point cloud frame contains a geometry head parameter set; if the geometry head flag field is a first preset value, it indicates that the current point cloud frame contains a geometry head parameter set; if the geometry head flag field is a second preset value, it indicates that the current point cloud frame does not contain a geometry head parameter set.

[0425] The attribute header flag field is used to indicate whether the current point cloud frame contains an attribute header parameter set. If the attribute header flag field is a first set value, it indicates that the current point cloud frame contains an attribute header parameter set; if the attribute header flag field is a second set value, it indicates that the current point cloud frame does not contain an attribute header parameter set.

[0426] The "Attribute Count Minus One" field indicates the number of attribute data groups corresponding to the attribute header parameter set contained in the current point cloud frame.

[0427] In one embodiment, one or more point cloud frames in the point cloud bitstream sequentially constitute a point cloud sequence;

[0428] When the current point cloud frame is the first frame in the point cloud sequence, the geometry header flag field is set to the first preset value, and the attribute header flag field is set to the first set value; the attribute quantity minus one field is the maximum number of attribute data groups contained in the point cloud sequence.

[0429] When the current point cloud frame is not the first frame in the point cloud sequence, if the geometry head flag field is set to the first preset value, it indicates that the set of geometry head parameters in the point cloud sequence has been updated.

[0430] If the current point cloud frame is not the first frame in the point cloud sequence, and the attribute header flag field is set to the first set value, it indicates that the set of attribute header parameters in the point cloud sequence has been updated.

[0431] In one embodiment, in point cloud media, a set of attribute data corresponds to a set of attribute header parameters;

[0432] The attribute header parameter set includes an attribute presence flag field, which indicates whether the current attribute header parameter set contains the encoded parameters of the x-th attribute data, where x is an integer greater than or equal to 0 and less than or equal to 15;

[0433] If the attribute exists and the flag field is the first value, it indicates that the current attribute header parameter set contains the encoded parameters of the xth group of attribute data;

[0434] If the attribute exists and the flag field has the second value, it indicates that the current attribute header parameter set does not contain the encoded parameters of the xth attribute data.

[0435] In one embodiment, the attribute header parameter set includes an attribute type field and an attribute data identifier field, which represent the correspondence between the attribute header parameter set and the attribute data.

[0436] In one embodiment, the parameter information of the point cloud frame is used to indicate the information of the data unit within the frame;

[0437] If the data unit includes point cloud frames, the parameter information of the current point cloud frame includes the start code and end code of the current point cloud frame; the current point cloud frame refers to the point cloud frame being decoded in the point cloud bitstream.

[0438] If the data unit includes point cloud slices, the parameter information of the current point cloud frame includes the start code and end code of the current point cloud slice in the current point cloud frame. The current point cloud slice refers to the point cloud slice being decoded in the point cloud bitstream.

[0439] If the data unit includes the geometric data of the point cloud patch, then the parameter information of the current point cloud frame includes the start code and end code of the geometric data of the current point cloud patch;

[0440] If the data unit includes attribute data of a point cloud patch, then the parameter information of the current point cloud frame includes the start code and end code of the current attribute data of the current point cloud patch. The current attribute data of the current point cloud patch refers to the attribute data being decoded in the current point cloud patch.

[0441] In one embodiment, the start code of the current point cloud frame is a preset specific bit string, or the start code of the current point cloud frame is the end code of the previous point cloud frame; the start code of the current point cloud frame is used to indicate the position where the current point cloud frame starts decoding, and the end code of the current point cloud frame is used to indicate the position where the current point cloud frame ends decoding.

[0442] The start code of the current point cloud patch is a preset specific bit string, or the start code of the current point cloud patch is the end code of the previous point cloud patch; the start code of the current point cloud patch is used to indicate the position where the current point cloud patch starts decoding, and the end code of the current point cloud patch is used to indicate the position where the current point cloud patch ends decoding.

[0443] The start code of the geometric data of the current point cloud patch is a preset specific bit string, or the start code of the geometric data of the current point cloud patch is the end code of the previous point cloud patch; the start code of the geometric data of the current point cloud patch is used to indicate the position where the geometric data of the current point cloud patch begins to be decoded, and the end code of the geometric data of the current point cloud patch is used to indicate the position where the geometric data of the current point cloud patch ends to be decoded.

[0444] The start code of the current attribute data of the current point cloud patch is a preset specific bit string, or the start code of the current attribute data of the current point cloud patch is the end code of the geometric data of the current point cloud patch, or the start code of the current attribute data of the current point cloud patch is the end code of the previous attribute data of the current attribute data in the current point cloud patch; the start code of the current attribute data of the current point cloud patch is used to indicate the position where the current attribute data of the current point cloud patch begins to be decoded, and the end code of the current attribute data of the current point cloud patch is used to indicate the position where the current attribute data of the current point cloud patch ends to be decoded.

[0445] In one embodiment, the start code of the current attribute data of the current point cloud patch is also used to indicate the type of the current attribute data of the current point cloud patch.

[0446] In one embodiment, the parameter information of each point cloud frame is set in the frame header of each point cloud frame; a point cloud frame includes one or more data units, and the processing unit 602, when decoding the point cloud bitstream according to the parameter information of each point cloud frame to present point cloud media, can specifically be used for:

[0447] Determine the target point cloud frame that needs to be decoded from the point cloud bitstream;

[0448] Based on the parameter information of the target point cloud frame, locate the data unit that needs to be decoded in the target point cloud frame;

[0449] Decode and present the data units that need to be decoded in the target point cloud frame.

[0450] In this embodiment of the application, a point cloud bitstream of point cloud media is obtained. The point cloud bitstream contains one or more point cloud frames and parameter information of each point cloud frame. Based on the parameter information of each point cloud frame, the point cloud bitstream is decoded to present the point cloud media. This method can flexibly organize the parameter information in the point cloud bitstream to guide the decoding and presentation of the point cloud media, thereby optimizing the resource overhead of the decoding end.

[0451] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of a point cloud media data processing device provided in an embodiment of this application. The point cloud media data processing device can be installed in the computer equipment provided in the embodiment of this application. The computer equipment can be the content production equipment mentioned in the above method embodiment. Figure 7 The data processing device for point cloud media shown can be a computer program (including program code) running on a computer device, and the data processing device for point cloud media can be used to execute... Figure 5 Some or all of the steps in the method embodiments shown. Please refer to [link / reference]. Figure 7 The data processing device for the point cloud media may include the following units:

[0452] Acquisition unit 701 is used to acquire point cloud media;

[0453] The processing unit 702 is used to encode the point cloud media to obtain a point cloud bitstream. The point cloud bitstream contains one or more point cloud frames and parameter information of each point cloud frame. The parameter information of the point cloud frame is used to indicate at least one of the following: frame sequence number, frame timestamp information, frame spatial information, and information of intra-frame data units.

[0454] The processing unit 702 is also used to transmit point cloud code streams.

[0455] In one embodiment, the parameter information of each point cloud frame is set in the frame header of each point cloud frame; the parameter information of the point cloud frame is also used to indicate whether the point cloud frame contains a parameter set, which includes any one of the following: geometric header parameter set, attribute header parameter set, geometric header parameter set, and attribute header parameter set.

[0456] In one embodiment, the parameter information of the point cloud frame is used to indicate the information of the data unit within the frame;

[0457] If the data unit includes point cloud frames, the parameter information of the current point cloud frame includes the start code and end code of the current point cloud frame; the current point cloud frame refers to the point cloud frame being encoded in the point cloud bitstream.

[0458] If the data unit includes point cloud slices, the parameter information of the current point cloud frame includes the start code and end code of the current point cloud slice in the current point cloud frame. The current point cloud slice refers to the point cloud slice being encoded in the point cloud bitstream.

[0459] If the data unit includes the geometric data of the point cloud patch, then the parameter information of the current point cloud frame includes the start code and end code of the geometric data of the current point cloud patch;

[0460] If the data unit includes the attribute data of the point cloud patch, then the parameter information of the current point cloud frame includes the start code and end code of the current attribute data of the current point cloud patch. The current attribute data of the current point cloud patch refers to the attribute data being encoded in the current point cloud patch.

[0461] In this embodiment, point cloud media is acquired and encoded to obtain a point cloud bitstream. The point cloud bitstream contains one or more point cloud frames and parameter information of each point cloud frame. Transmitting the point cloud bitstream can flexibly organize the parameter information in the point cloud bitstream, guide the decoding and presentation of the point cloud media, and optimize the resource overhead of the decoding end.

[0462] Furthermore, this application also provides a schematic diagram of the structure of a computer device, which can be found in [reference needed]. Figure 8The computer device may be the aforementioned media processing device or content production device; the computer device may include: a processor 801, an input device 802, an output device 803, and a memory 804. The processor 801, input device 802, output device 803, and memory 804 are connected via a bus. The memory 804 is used to store computer programs, which include program instructions, and the processor 801 is used to execute the program instructions stored in the memory 804.

[0463] When the computer device is the aforementioned media processing device, in this embodiment of the application, the processor 801 performs the following operations by running the executable program code in the memory 804:

[0464] The point cloud stream of the point cloud media is obtained. The point cloud stream contains one or more point cloud frames and parameter information of each point cloud frame. The parameter information of the point cloud frame is used to indicate at least one of the following: frame sequence number, frame timestamp information, frame spatial information, and information of intra-frame data units.

[0465] Based on the parameter information of each point cloud frame, the point cloud bitstream is decoded to present the point cloud media.

[0466] In one embodiment, the parameter information of the point cloud frame is used to indicate the frame sequence number; one or more point cloud frames in the point cloud bitstream constitute a point cloud sequence in sequence.

[0467] The parameter information of the current point cloud frame includes a frame sequence number field, which indicates the sequence number of the current point cloud frame in the point cloud sequence.

[0468] The current point cloud frame refers to the point cloud frame being decoded in the point cloud bitstream.

[0469] In one embodiment, the parameter information of the point cloud frame is used to indicate the frame timestamp information; the parameter information of the current point cloud frame includes at least one of the following fields: timestamp flag field, timestamp quantity field, timestamp type field, and timestamp field;

[0470] The timestamp flag field is used to indicate whether the current point cloud frame contains a timestamp. When the timestamp flag field is a first preset value, it indicates that the current point cloud frame does not contain a timestamp; when the timestamp flag field is a second preset value, it indicates that the current point cloud frame contains a timestamp.

[0471] The timestamp count field is used to indicate the number of timestamps contained in the current point cloud frame;

[0472] The timestamp type field is used to indicate the type of the current timestamp. If the timestamp type field is the first preset value, it indicates that the current timestamp is the decoding time; if the timestamp type field is the second preset value, it indicates that the current timestamp is the presentation time; if the timestamp type field is the third preset value, it indicates that the current timestamp is the acquisition time.

[0473] The timestamp field is used to indicate any one or more of the acquisition time, presentation time, and decoding time of the current point cloud frame; the timestamp field is used to indicate Coordinated Universal Time, or the timestamp field is indicated according to a preset format, the preset format including at least one of the following fields: year timestamp field, hour timestamp field, second timestamp field, and frame timestamp field;

[0474] Here, the current point cloud frame refers to the point cloud frame being decoded in the point cloud bitstream, and the current timestamp refers to the timestamp of the current point cloud frame being used.

[0475] In one embodiment, the parameter information of the point cloud frame is used to indicate the frame spatial information; the parameter information of the current point cloud frame includes a spatial information flag field.

[0476] The spatial information flag field indicates whether to indicate the spatial information of the current point cloud frame. When the spatial information flag field is the first value, it indicates the spatial information of the current point cloud frame; when the spatial information flag field is the second value, it does not indicate the spatial information of the current point cloud frame.

[0477] Spatial information includes at least one of the following: coordinates of the origin of the bounding box of the current point cloud frame, width of the bounding box of the current point cloud frame, and depth of the bounding box of the current point cloud frame.

[0478] In one embodiment, the parameter information of a point cloud frame is used to indicate the information of the data unit within the frame; a point cloud frame contains one or more data units, and the data unit includes point cloud patches; the parameter information of the current point cloud frame includes a point cloud patch count field or a point cloud patch count minus one field.

[0479] The point cloud patch quantity field is used to indicate the number of point cloud patches contained in the current point cloud frame;

[0480] The "Number of Point Cloud Pieces - One" field is used to indirectly indicate the number of point cloud pieces contained in the current point cloud frame.

[0481] The current point cloud frame refers to the point cloud frame being decoded in the point cloud bitstream.

[0482] In one embodiment, the parameter information of the point cloud frame is also used to indicate whether the point cloud frame contains a parameter set, which includes any of the following: a geometry head parameter set, an attribute head parameter set, a geometry head parameter set, and an attribute head parameter set.

[0483] The parameter information of the current point cloud frame includes any one of the following fields: geometry header flag field, attribute header flag field, and attribute count minus one field;

[0484] The geometry head flag field is used to indicate whether the current point cloud frame contains a geometry head parameter set; if the geometry head flag field is a first preset value, it indicates that the current point cloud frame contains a geometry head parameter set; if the geometry head flag field is a second preset value, it indicates that the current point cloud frame does not contain a geometry head parameter set.

[0485] The attribute header flag field is used to indicate whether the current point cloud frame contains an attribute header parameter set. If the attribute header flag field is a first set value, it indicates that the current point cloud frame contains an attribute header parameter set; if the attribute header flag field is a second set value, it indicates that the current point cloud frame does not contain an attribute header parameter set.

[0486] The "Attribute Count Minus One" field indicates the number of attribute data groups corresponding to the attribute header parameter set contained in the current point cloud frame.

[0487] In one embodiment, one or more point cloud frames in the point cloud bitstream sequentially constitute a point cloud sequence;

[0488] When the current point cloud frame is the first frame in the point cloud sequence, the geometry header flag field is set to the first preset value, and the attribute header flag field is set to the first set value; the attribute quantity minus one field is the maximum number of attribute data groups contained in the point cloud sequence.

[0489] When the current point cloud frame is not the first frame in the point cloud sequence, if the geometry head flag field is set to the first preset value, it indicates that the set of geometry head parameters in the point cloud sequence has been updated.

[0490] If the current point cloud frame is not the first frame in the point cloud sequence, and the attribute header flag field is set to the first set value, it indicates that the set of attribute header parameters in the point cloud sequence has been updated.

[0491] In one embodiment, in point cloud media, a set of attribute data corresponds to a set of attribute header parameters;

[0492] The attribute header parameter set includes an attribute presence flag field, which indicates whether the current attribute header parameter set contains the encoded parameters of the x-th attribute data, where x is an integer greater than or equal to 0 and less than or equal to 15;

[0493] If the attribute exists and the flag field is the first value, it indicates that the current attribute header parameter set contains the encoded parameters of the xth group of attribute data;

[0494] If the attribute exists and the flag field has the second value, it indicates that the current attribute header parameter set does not contain the encoded parameters of the xth attribute data.

[0495] In one embodiment, the attribute header parameter set includes an attribute type field and an attribute data identifier field, which represent the correspondence between the attribute header parameter set and the attribute data.

[0496] In one embodiment, the parameter information of the point cloud frame is used to indicate the information of the data unit within the frame;

[0497] If the data unit includes point cloud frames, the parameter information of the current point cloud frame includes the start code and end code of the current point cloud frame; the current point cloud frame refers to the point cloud frame being decoded in the point cloud bitstream.

[0498] If the data unit includes point cloud slices, the parameter information of the current point cloud frame includes the start code and end code of the current point cloud slice in the current point cloud frame. The current point cloud slice refers to the point cloud slice being decoded in the point cloud bitstream.

[0499] If the data unit includes the geometric data of the point cloud patch, then the parameter information of the current point cloud frame includes the start code and end code of the geometric data of the current point cloud patch;

[0500] If the data unit includes attribute data of a point cloud patch, then the parameter information of the current point cloud frame includes the start code and end code of the current attribute data of the current point cloud patch. The current attribute data of the current point cloud patch refers to the attribute data being decoded in the current point cloud patch.

[0501] In one embodiment, the start code of the current point cloud frame is a preset specific bit string, or the start code of the current point cloud frame is the end code of the previous point cloud frame; the start code of the current point cloud frame is used to indicate the position where the current point cloud frame starts decoding, and the end code of the current point cloud frame is used to indicate the position where the current point cloud frame ends decoding.

[0502] The start code of the current point cloud patch is a preset specific bit string, or the start code of the current point cloud patch is the end code of the previous point cloud patch; the start code of the current point cloud patch is used to indicate the position where the current point cloud patch starts decoding, and the end code of the current point cloud patch is used to indicate the position where the current point cloud patch ends decoding.

[0503] The start code of the geometric data of the current point cloud patch is a preset specific bit string, or the start code of the geometric data of the current point cloud patch is the end code of the previous point cloud patch; the start code of the geometric data of the current point cloud patch is used to indicate the position where the geometric data of the current point cloud patch begins to be decoded, and the end code of the geometric data of the current point cloud patch is used to indicate the position where the geometric data of the current point cloud patch ends to be decoded.

[0504] The start code of the current attribute data of the current point cloud patch is a preset specific bit string, or the start code of the current attribute data of the current point cloud patch is the end code of the geometric data of the current point cloud patch, or the start code of the current attribute data of the current point cloud patch is the end code of the previous attribute data of the current attribute data in the current point cloud patch; the start code of the current attribute data of the current point cloud patch is used to indicate the position where the current attribute data of the current point cloud patch begins to be decoded, and the end code of the current attribute data of the current point cloud patch is used to indicate the position where the current attribute data of the current point cloud patch ends to be decoded.

[0505] In one embodiment, the start code of the current attribute data of the current point cloud patch is also used to indicate the type of the current attribute data of the current point cloud patch.

[0506] In one embodiment, the parameter information of each point cloud frame is set in the frame header of each point cloud frame; a point cloud frame includes one or more data units, and when the processor 801 decodes the point cloud bitstream according to the parameter information of each point cloud frame to present the point cloud media, it can be specifically used for:

[0507] Determine the target point cloud frame that needs to be decoded from the point cloud bitstream;

[0508] Based on the parameter information of the target point cloud frame, locate the data unit that needs to be decoded in the target point cloud frame;

[0509] Decode and present the data units that need to be decoded in the target point cloud frame.

[0510] In this embodiment of the application, a point cloud bitstream of point cloud media is obtained. The point cloud bitstream contains one or more point cloud frames and parameter information of each point cloud frame. Based on the parameter information of each point cloud frame, the point cloud bitstream is decoded to present the point cloud media. This method can flexibly organize the parameter information in the point cloud bitstream to guide the decoding and presentation of the point cloud media, thereby optimizing the resource overhead of the decoding end.

[0511] Optionally, when the computer device is used to create the above-mentioned content, in this embodiment of the application, the processor 801 performs the following operations by running the executable program code in the memory 804:

[0512] Acquire point cloud media;

[0513] The point cloud media is encoded to obtain a point cloud bitstream. The point cloud bitstream contains one or more point cloud frames and parameter information of each point cloud frame. The parameter information of the point cloud frame is used to indicate at least one of the following: frame sequence number, frame timestamp information, frame spatial information, and information of intra-frame data units.

[0514] Transmit point cloud bitstream.

[0515] In one embodiment, the parameter information of each point cloud frame is set in the frame header of each point cloud frame;

[0516] The parameter information of the point cloud frame is also used to indicate whether the point cloud frame contains a parameter set, which includes any of the following: geometric head parameter set, attribute head parameter set, geometric head parameter set, and attribute head parameter set.

[0517] In one embodiment, the parameter information of the point cloud frame is used to indicate the information of the data unit within the frame;

[0518] If the data unit includes point cloud frames, the parameter information of the current point cloud frame includes the start code and end code of the current point cloud frame; the current point cloud frame refers to the point cloud frame being encoded in the point cloud bitstream.

[0519] If the data unit includes point cloud slices, the parameter information of the current point cloud frame includes the start code and end code of the current point cloud slice in the current point cloud frame. The current point cloud slice refers to the point cloud slice being encoded in the point cloud bitstream.

[0520] If the data unit includes the geometric data of the point cloud patch, then the parameter information of the current point cloud frame includes the start code and end code of the geometric data of the current point cloud patch;

[0521] If the data unit includes the attribute data of the point cloud patch, then the parameter information of the current point cloud frame includes the start code and end code of the current attribute data of the current point cloud patch. The current attribute data of the current point cloud patch refers to the attribute data being encoded in the current point cloud patch.

[0522] In this embodiment, point cloud media is acquired and encoded to obtain a point cloud bitstream. The point cloud bitstream contains one or more point cloud frames and parameter information of each point cloud frame. Transmitting the point cloud bitstream can flexibly organize the parameter information in the point cloud bitstream, guide the decoding and presentation of the point cloud media, and optimize the resource overhead of the decoding end.

[0523] Furthermore, it should be noted that this application also provides a computer-readable storage medium storing a computer program, which includes program instructions. When a processor executes these program instructions, it can execute the aforementioned... Figure 3 and Figure 5 The methods described in the corresponding embodiments are therefore not repeated here. For technical details not disclosed in the computer-readable storage medium embodiments related to this application, please refer to the description of the method embodiments of this application. As an example, program instructions may be deployed on a computer device, executed on multiple computer devices located in one location, or executed on multiple computer devices distributed in multiple locations and interconnected through a communication network.

[0524] According to one aspect of this application, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, enabling the computer device to perform the aforementioned... Figure 3 and Figure 5 The methods described in the corresponding embodiments will not be repeated here.

[0525] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0526] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Those skilled in the art will understand that implementing all or part of the processes of the above embodiments and making equivalent changes in accordance with the claims of this application are still within the scope of the invention.

Claims

1. A data processing method for point cloud media, characterized in that, include: The point cloud stream of the point cloud media is obtained, wherein the point cloud stream contains one or more point cloud frames and parameter information of each point cloud frame; wherein the parameter information of each point cloud frame is set in the frame header of each point cloud frame, and the parameter information of the point cloud frame is used to indicate at least one of the following information: frame sequence number, frame timestamp information, frame spatial information, and information of intra-frame data units. Based on the parameter information of each point cloud frame, the point cloud bitstream is decoded to present the point cloud media; The parameter information of the point cloud frame is also used to indicate whether the point cloud frame contains a parameter set, which includes any of the following: a geometry head parameter set, an attribute head parameter set, a geometry head parameter set, and an attribute head parameter set; The parameter information of the current point cloud frame includes any one of the following fields: geometric header flag field, attribute header flag field, and attribute count minus one field; the current point cloud frame refers to the point cloud frame being decoded in the point cloud bitstream, the geometric header flag field is used to indicate whether the current point cloud frame contains the geometric header parameter set, the attribute header flag field is used to indicate whether the current point cloud frame contains the attribute header parameter set, and the attribute count minus one field is used to indicate the number of attribute data groups corresponding to the attribute header parameter set contained in the current point cloud frame.

2. The method as described in claim 1, characterized in that, The parameter information of the point cloud frame is used to indicate the frame sequence number; one or more point cloud frames in the point cloud bitstream are arranged in sequence to form a point cloud sequence. The parameter information of the current point cloud frame includes a frame sequence number field, which is used to indicate the sequence number of the current point cloud frame in the point cloud sequence; The current point cloud frame refers to the point cloud frame being decoded in the point cloud bitstream.

3. The method as described in claim 1, characterized in that, The parameter information of the point cloud frame is used to indicate the frame timestamp information; the parameter information of the current point cloud frame includes at least one of the following fields: timestamp flag field, timestamp quantity field, timestamp type field, and timestamp field; The timestamp flag field is used to indicate whether the current point cloud frame contains a timestamp. When the timestamp flag field is a first preset value, it indicates that the current point cloud frame does not contain a timestamp; when the timestamp flag field is a second preset value, it indicates that the current point cloud frame contains a timestamp. The timestamp count field is used to indicate the number of timestamps contained in the current point cloud frame; The timestamp type field is used to indicate the type of the current timestamp. If the timestamp type field is a first preset value, it indicates that the type of the current timestamp is decoding time. If the timestamp type field is a second preset value, it indicates that the type of the current timestamp is presentation time; If the timestamp type field is a third preset value, it indicates that the current timestamp type is collection time; The timestamp field is used to indicate any one or more of the acquisition time, presentation time, and decoding time of the current point cloud frame; the timestamp field is used to indicate Coordinated Universal Time, or the timestamp field is indicated according to a preset format, the preset format including at least one of the following fields: year timestamp field, hour timestamp field, second timestamp field, and frame timestamp field; Wherein, the current point cloud frame refers to the point cloud frame being decoded in the point cloud bitstream, and the current timestamp refers to the timestamp of the current point cloud frame being in use.

4. The method as described in claim 1, characterized in that, The parameter information of the point cloud frame is used to indicate the frame spatial information; the parameter information of the current point cloud frame includes a spatial information flag field; The spatial information flag field indicates whether to indicate the spatial information of the current point cloud frame. When the spatial information flag field is the first value, it indicates the spatial information of the current point cloud frame. When the spatial information flag field is the second value, it does not indicate the spatial information of the current point cloud frame; The spatial information includes at least one of the following: the coordinate information of the origin of the bounding box of the current point cloud frame, the width information of the bounding box of the current point cloud frame, and the depth information of the bounding box of the current point cloud frame.

5. The method as described in claim 1, characterized in that, The parameter information of a point cloud frame is used to indicate the information of the data units within the frame; a point cloud frame contains one or more data units, and the data units include point cloud patches; the parameter information of the current point cloud frame includes a point cloud patch count field or a point cloud patch count minus one field. The point cloud patch quantity field is used to indicate the number of point cloud patches contained in the current point cloud frame; The field "point cloud piece count minus one" is used to indirectly indicate the number of point cloud pieces contained in the current point cloud frame. The current point cloud frame refers to the point cloud frame being decoded in the point cloud bitstream.

6. The method as described in claim 1, characterized in that, If the geometry head flag field is a first preset value, it indicates that the current point cloud frame contains a geometry head parameter set; if the geometry head flag field is a second preset value, it indicates that the current point cloud frame does not contain a geometry head parameter set. If the attribute header flag field is a first set value, it indicates that the current point cloud frame contains an attribute header parameter set; if the attribute header flag field is a second set value, it indicates that the current point cloud frame does not contain an attribute header parameter set.

7. The method as described in claim 6, characterized in that, One or more point cloud frames in the point cloud bitstream are sequentially arranged to form a point cloud sequence; When the current point cloud frame is the first frame in the point cloud sequence, the geometry header flag field is set to a first preset value, and the attribute header flag field is set to a first set value; the attribute quantity minus one field is set to the maximum number of attribute data groups contained in the point cloud sequence. When the current point cloud frame is not the first frame in the point cloud sequence, if the geometry head flag field is set to a first preset value, it indicates that the set of geometry head parameters in the point cloud sequence has been updated. When the current point cloud frame is not the first frame in the point cloud sequence, if the attribute header flag field is set to a first preset value, it indicates that the set of attribute header parameters in the point cloud sequence has been updated.

8. The method as described in claim 6 or 7, characterized in that, In the point cloud media, a set of attribute data corresponds to a set of attribute header parameters; The attribute header parameter set includes an attribute presence flag field, which is used to indicate whether the current attribute header parameter set contains the encoding parameter of the xth group of attribute data, where x is an integer greater than or equal to 0 and less than or equal to 15; If the attribute has a flag field with a first value, it indicates that the current attribute header parameter set contains the encoding parameters of the xth group of attribute data; If the attribute has a flag field with a second value, it indicates that the current attribute header parameter set does not contain the encoding parameter of the xth group of attribute data.

9. The method as described in claim 6 or 7, characterized in that, The attribute header parameter set includes an attribute type field and an attribute data identifier field, which represent the correspondence between the attribute header parameter set and the attribute data.

10. The method as described in claim 1, characterized in that, The parameter information of the point cloud frame is used to indicate the information of the data units within the frame; If the data unit includes a point cloud frame, then the parameter information of the current point cloud frame includes the start code and end code of the current point cloud frame; the current point cloud frame refers to the point cloud frame being decoded in the point cloud bitstream. If the data unit includes a point cloud piece, then the parameter information of the current point cloud frame includes the start code and end code of the current point cloud piece in the current point cloud frame, and the current point cloud piece refers to the point cloud piece being decoded in the point cloud code stream. If the data unit includes geometric data of a point cloud patch, then the parameter information of the current point cloud frame includes the start code and end code of the geometric data of the current point cloud patch. If the data unit includes attribute data of a point cloud patch, then the parameter information of the current point cloud frame includes the start code and end code of the current attribute data of the current point cloud patch, wherein the current attribute data of the current point cloud patch refers to the attribute data being decoded in the current point cloud patch.

11. The method as described in claim 10, characterized in that, The start code of the current point cloud frame is a preset specific bit string, or the start code of the current point cloud frame is the end code of the previous point cloud frame; the start code of the current point cloud frame is used to indicate the position where the current point cloud frame starts decoding, and the end code of the current point cloud frame is used to indicate the position where the current point cloud frame ends decoding. The start code of the current point cloud patch is a preset specific bit string, or the start code of the current point cloud patch is the end code of the previous point cloud patch; the start code of the current point cloud patch is used to indicate the position where the current point cloud patch starts decoding, and the end code of the current point cloud patch is used to indicate the position where the current point cloud patch ends decoding. The start code of the geometric data of the current point cloud patch is a preset specific bit string, or the start code of the geometric data of the current point cloud patch is the end code of the previous point cloud patch; the start code of the geometric data of the current point cloud patch is used to indicate the position where the geometric data of the current point cloud patch begins to be decoded, and the end code of the geometric data of the current point cloud patch is used to indicate the position where the geometric data of the current point cloud patch ends to be decoded. The start code of the current attribute data of the current point cloud patch is a preset specific bit string, or the start code of the current attribute data of the current point cloud patch is the end code of the geometric data of the current point cloud patch, or the start code of the current attribute data of the current point cloud patch is the end code of the previous attribute data of the current attribute data in the current point cloud patch. The start code of the current attribute data of the current point cloud patch is used to indicate the position where the current attribute data of the current point cloud patch begins to be decoded, and the end code of the current attribute data of the current point cloud patch is used to indicate the position where the current attribute data of the current point cloud patch ends to be decoded.

12. The method as described in claim 10 or 11, characterized in that, The start code of the current attribute data of the current point cloud patch is also used to indicate the type of the current attribute data of the current point cloud patch.

13. The method as described in claim 1, characterized in that, A point cloud frame includes one or more data units. The step of decoding the point cloud bitstream to present the point cloud media based on the parameter information of each point cloud frame includes: Determine the target point cloud frame to be decoded from the point cloud bitstream; According to the parameter information of the target point cloud frame, locate the data unit that needs to be decoded in the target point cloud frame; The data units that need to be decoded in the target point cloud frame are decoded and presented.

14. A data processing method for point cloud media, characterized in that, include: Acquire point cloud media; The point cloud media is encoded to obtain a point cloud bitstream. The point cloud bitstream contains one or more point cloud frames and parameter information for each point cloud frame. The parameter information for each point cloud frame is set in the frame header of each point cloud frame. The parameter information of the point cloud frame is used to indicate at least one of the following: frame sequence number, frame timestamp information, frame spatial information, and information of intra-frame data units. The parameter information of the point cloud frame is also used to indicate whether the point cloud frame contains a parameter set, which includes any of the following: a geometry head parameter set, an attribute head parameter set, a geometry head parameter set, and an attribute head parameter set; The parameter information of the current point cloud frame includes any one of the following fields: geometric header flag field, attribute header flag field, and attribute count minus one field; the current point cloud frame refers to the point cloud frame being encoded in the point cloud bitstream, the geometric header flag field is used to indicate whether the current point cloud frame contains the geometric header parameter set, the attribute header flag field is used to indicate whether the current point cloud frame contains the attribute header parameter set, and the attribute count minus one field is used to indicate the number of attribute data groups corresponding to the attribute header parameter set contained in the current point cloud frame; Transmit the point cloud code stream.

15. The method as described in claim 14, characterized in that, The parameter information of the point cloud frame is used to indicate the information of the data units within the frame; If the data unit includes a point cloud frame, then the parameter information of the current point cloud frame includes the start code and end code of the current point cloud frame; the current point cloud frame refers to the point cloud frame being encoded in the point cloud bitstream. If the data unit includes a point cloud piece, then the parameter information of the current point cloud frame includes the start code and end code of the current point cloud piece in the current point cloud frame, and the current point cloud piece refers to the point cloud piece being encoded in the point cloud code stream. If the data unit includes geometric data of a point cloud patch, then the parameter information of the current point cloud frame includes the start code and end code of the geometric data of the current point cloud patch. If the data unit includes attribute data of a point cloud patch, then the parameter information of the current point cloud frame includes the start code and end code of the current attribute data of the current point cloud patch, wherein the current attribute data of the current point cloud patch refers to the attribute data being encoded in the current point cloud patch.

16. A data processing device for point cloud media, characterized in that, include: The acquisition unit is used to acquire the point cloud bitstream of the point cloud media, wherein the point cloud bitstream includes one or more point cloud frames and parameter information of each point cloud frame; wherein the parameter information of each point cloud frame is set in the frame header of each point cloud frame, and the parameter information of the point cloud frame is used to indicate at least one of the following information: frame sequence number, frame timestamp information, frame spatial information, and information of intra-frame data units. The processing unit is used to decode the point cloud bitstream according to the parameter information of each point cloud frame to present the point cloud media. The parameter information of the point cloud frame is also used to indicate whether the point cloud frame contains a parameter set, which includes any of the following: a geometry head parameter set, an attribute head parameter set, a geometry head parameter set, and an attribute head parameter set; The parameter information of the current point cloud frame includes any one of the following fields: geometric header flag field, attribute header flag field, and attribute count minus one field; the current point cloud frame refers to the point cloud frame being decoded in the point cloud bitstream, the geometric header flag field is used to indicate whether the current point cloud frame contains the geometric header parameter set, the attribute header flag field is used to indicate whether the current point cloud frame contains the attribute header parameter set, and the attribute count minus one field is used to indicate the number of attribute data groups corresponding to the attribute header parameter set contained in the current point cloud frame.

17. A data processing device for point cloud media, characterized in that, include: The acquisition unit is used to acquire point cloud media; The processing unit is used to encode the point cloud media to obtain a point cloud bitstream. The point cloud bitstream includes one or more point cloud frames and parameter information of each point cloud frame. The parameter information of each point cloud frame is set in the frame header of each point cloud frame. The parameter information of the point cloud frame is used to indicate at least one of the following information: frame sequence number, frame timestamp information, frame spatial information, and information of intra-frame data units. The parameter information of the point cloud frame is also used to indicate whether the point cloud frame contains a parameter set, which includes any of the following: a geometry head parameter set, an attribute head parameter set, a geometry head parameter set, and an attribute head parameter set; The parameter information of the current point cloud frame includes any one of the following fields: geometric header flag field, attribute header flag field, and attribute count minus one field; the current point cloud frame refers to the point cloud frame being encoded in the point cloud bitstream, the geometric header flag field is used to indicate whether the current point cloud frame contains the geometric header parameter set, the attribute header flag field is used to indicate whether the current point cloud frame contains the attribute header parameter set, and the attribute count minus one field is used to indicate the number of attribute data groups corresponding to the attribute header parameter set contained in the current point cloud frame; The processing unit is also used to transmit the point cloud code stream.

18. A computer device, characterized in that, include: A processor is used to execute computer programs; A computer-readable storage medium storing a computer program, which, when executed by the processor, performs the data processing method for point cloud media as described in any one of claims 1-15.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, performs the data processing method for point cloud media as described in any one of claims 1-15.

20. A computer program product, comprising a computer program, characterized in that, The computer program is stored in a computer-readable storage medium, and when read and executed by the processor of a computer device, causes the computer device to perform the data processing method for point cloud media as described in any one of claims 1-15.

Images