Point cloud file processing methods and apparatuses, medium, electronic device, and program product

By defining unit type fields in the data box of point cloud code stream, the problem of poor data access flexibility caused by data encapsulation of multiple attribute types in point cloud files is solved, and flexible decoding and access to specified data types is achieved, improving data Access efficiency.

WO2024245182A1PCT designated stage expired Publication Date: 2024-12-05TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2024/095467
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-31
Filing Date
2024-05-27
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

Data of multiple different attribute types in point cloud files are encapsulated in the same data structure, resulting in poor data access flexibility. This data structure needs to be completely decoded to obtain data of all attribute types.

Method used

The unit type field is defined in the data box of the point cloud code stream, and the data type of the data unit is indicated according to the field, so that a portion of the point cloud file with a specified data type is selectively decoded, and data access flexibility is optimized.

Benefits of technology

Through the use of unit type fields, data units of specified data types in point cloud files can be flexibly decoded and accessed, improving the flexibility and efficiency of data access.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024095467_05122024_PF_FP_ABST
    Figure CN2024095467_05122024_PF_FP_ABST
Patent Text Reader

Abstract

The present application belongs to the technical field of point cloud media, and particularly relates to point cloud file processing methods, point cloud file processing apparatuses, a computer-readable medium, an electronic device, and a computer program product. A method comprises: acquiring a point cloud file carrying a point cloud code stream, a data box of the point cloud code stream comprising a unit type field, and the unit type field being used for indicating the data type of a data unit in the point cloud code stream; and, according to the unit type field, unpackaging and decoding the point cloud file, so as to obtain point cloud media content corresponding to the data unit. The embodiments of the present application can optimize the data access flexibility of point cloud files.
Need to check novelty before this filing date? Find Prior Art

Description

Point cloud file processing method, device, medium, electronic equipment and program product Technical Field

[0001] The present application belongs to the field of point cloud media technology, and specifically relates to a point cloud file processing method, a point cloud file processing device, a computer-readable medium, an electronic device, and a computer program product.

[0002] Background of the Invention

[0003] A point cloud is a collection of randomly distributed, discrete points in space that represent the spatial structure and surface properties of a three-dimensional object or scene. After acquiring large amounts of point cloud data through point cloud acquisition equipment, this data can be encoded and packaged for transmission to the user, where it can be decoded and presented.

[0004] In the data code stream of the point cloud, if data of multiple different attribute types are encapsulated in the same data structure, the data structure needs to be fully decoded to obtain data of all attribute types, resulting in poor data access flexibility.

[0005] Summary of the Invention

[0006] The present application provides a point cloud file processing method, a point cloud file processing device, a computer-readable medium, an electronic device, and a computer program product, the purpose of which is to optimize the data access flexibility of point cloud files.

[0007] According to one aspect of an embodiment of the present application, a method for processing a point cloud file is provided, the method comprising:

[0008] Acquire a point cloud file carrying a point cloud code stream, wherein a data box of the point cloud code stream includes a unit type field, and the unit type field is used to indicate a data type of a data unit in the point cloud code stream;

[0009] The point cloud file is decapsulated and decoded according to the unit type field to obtain point cloud media content corresponding to the data unit.

[0010] According to one aspect of an embodiment of the present application, a device for processing a point cloud file is provided, the device comprising:

[0011] A first acquisition module is configured to acquire a point cloud file carrying a point cloud code stream, wherein a data box of the point cloud code stream includes a unit type field, and the unit type field is used to indicate a data type of a data unit in the point cloud code stream;

[0012] The decoding module is configured to decapsulate and decode the point cloud file according to the unit type field to obtain point cloud media content corresponding to the data unit.

[0013] According to one aspect of an embodiment of the present application, a method for processing a point cloud file is provided, the method comprising:

[0014] Get point cloud media content;

[0015] The point cloud media content is encoded and packaged to obtain a point cloud file carrying a point cloud code stream; the data box of the point cloud code stream includes a unit type field, and the unit type field is used to indicate the data type of the data unit in the point cloud code stream.

[0016] According to one aspect of an embodiment of the present application, a device for processing a point cloud file is provided, the device comprising:

[0017] A second acquisition module is configured to acquire point cloud media content;

[0018] The encoding module is configured to encode and encapsulate the point cloud media content to obtain a point cloud file carrying a point cloud code stream; the data box of the point cloud code stream includes a unit type field, and the unit type field is used to indicate the data type of the data unit in the point cloud code stream.

[0019] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for processing a point cloud file in the above technical solution is implemented.

[0020] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the executable instructions to implement a method for processing point cloud files as in the above technical solution.

[0021] According to one aspect of an embodiment of the present application, a computer program product is provided, including a computer program, which, when executed by a processor, implements the method for processing a point cloud file in the above technical solution.

[0022] In the technical solution provided in the embodiment of the present application, by defining a unit type field in the data box of the point cloud code stream, the data type of the data unit in the point cloud code stream can be indicated based on the unit type field, and then, according to the data type indicated by the unit type field, a part of the data units with the specified data type in the point cloud file can be selectively decoded, thereby optimizing the data access flexibility of the point cloud file.

[0023] BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0025] FIG1 shows a schematic diagram of a system framework of point cloud media encoding and decoding in an application scenario according to an embodiment of the present application.

[0026] FIG2 shows a flowchart of the steps of a method for processing a point cloud file in one embodiment of the present application.

[0027] FIG3 shows a point cloud file obtained by performing multi-track packaging based on component types in one embodiment of the present application.

[0028] FIG4 shows a point cloud file obtained based on single-track packaging in an example of the present application.

[0029] FIG5 shows a point cloud file that is packaged in multiple tracks based on point cloud slices in one embodiment of the present application.

[0030] FIG6 shows a point cloud file obtained by simultaneously encapsulating the geometric data and attribute data of a point cloud slice through a single track in one embodiment of the present application.

[0031] FIG7 shows a flowchart of a method for processing a point cloud file in one embodiment of the present application.

[0032] FIG8 schematically shows a structural block diagram of a point cloud file processing device at a decoding end provided in an embodiment of the present application.

[0033] FIG9 schematically shows a structural block diagram of a point cloud file processing device at the encoding end provided in an embodiment of the present application.

[0034] FIG10 schematically shows a block diagram of a computer system structure of an electronic device suitable for implementing an embodiment of the present application.

[0035] Modes for Carrying Out the Invention

[0036] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art.

[0037] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner.In the following description, many specific details are provided so as to provide a full understanding of the embodiments of the present application. However, it will be appreciated by those skilled in the art that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps etc. can be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0038] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0039] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0040] In the specific implementation of this application, user-related data such as the transmission content, decoding content and consumption content of point cloud media is involved. When the various embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0041] The relevant terms or abbreviations involved in the embodiments of this application are explained as follows.

[0042] AVS: Audio Video Coding Standard, China's national video coding standard AVS.

[0043] MPEG: Moving Picture Experts Group, an organization established by ISO (International Standardization Organization) and IEC (International Electrotechnical Commission) to develop international standards for moving picture and speech compression.

[0044] ISOBMFF: ISO Based Media File Format, a media file format based on ISO standards. ISOBMFF is a media file encapsulation standard, and the most typical ISOBMFF file is the MP4 file.

[0045] Media Segment: A playable segment that conforms to a specific media format. Playback may require the cooperation of zero or more preceding segments and the initialization segment.

[0046] PCC: Point Cloud Compression, point cloud compression.

[0047] V-PCC: Video-based Point Cloud Compression, point cloud compression based on traditional video coding methods.

[0048] G-PCC: Geometry-based Point Cloud Compression, point cloud compression based on geometric model.

[0049] AVS-PCC: Point cloud compression based on China's national video coding standard AVS.

[0050] Track: A track is a collection of media data in the media file encapsulation process, consisting of multiple time-sequential samples. A media file can be composed of one or more tracks. For example, a media file can contain a video media track, an audio media track, and a subtitle media track. Metadata information can also be included in the file as a media type in the form of a metadata media track.

[0051] Sample: A sample is a unit of media file encapsulation. A track consists of many samples, each of which corresponds to a specific timestamp. For example, a video media track can consist of many samples, with a sample typically representing a video frame. In this embodiment of the present application, a sample in a point cloud media track can represent a point cloud frame.

[0052] Slice: A point cloud slice / point cloud strip, representing a collection of syntax elements (such as geometry slices and attribute slices) of part or all of the encoded point cloud frame data.

[0053] Sequence Header: Point cloud sequence header parameter set, the parameter set required for point cloud sequence decoding. (AVS definition)

[0054] Geometry Header: Point cloud frame geometry header parameter set, the parameter set required for decoding point cloud geometry data.

[0055] Attribute Header: Point cloud frame attribute header parameter set, the parameter set required for decoding point cloud attribute data.

[0056] Sample Number: The number of a specific sample. The first sample in a track has a number of 1.

[0057] Sample Entry: A sample entry that indicates metadata related to all samples in a track. For example, a sample entry for a video track typically contains metadata related to decoder initialization.

[0058] Sample Group: Sample group, used to group some samples in the track according to specific rules.

[0059] A point cloud is a collection of randomly distributed, discrete points in space that represent the spatial structure and surface properties of a three-dimensional object or scene. Each point in a point cloud has at least 3D position information and, depending on the application scenario, may also have color, material, or other attribute information. Typically, each point in a point cloud has the same number of additional attributes.

[0060] Point cloud media can be categorized by encoding method into V-PCC (point cloud media compressed using traditional video coding methods), G-PCC (point cloud media compressed based on geometric features), and AVS-PCC (point cloud compression based on China's national video coding standard). In point cloud media file packaging, 3D position information is typically referred to as the geometry component of the point cloud media file, and attribute information is referred to as the attribute component of the point cloud media file. A point cloud media file has only one geometry component, but can contain one or more attribute components.

[0061] Point clouds can flexibly and conveniently express the spatial structure and surface properties of three-dimensional objects or scenes, and therefore have a wide range of applications, including virtual reality (VR) games, computer-aided design (CAD), geographic information systems (GIS), autonomous navigation systems (ANS), digital cultural heritage, free viewpoint broadcasting, three-dimensional immersive telepresence, and three-dimensional reconstruction of biological tissues and organs.

[0062] Point clouds are primarily acquired through computer generation, 3D laser scanning, and 3D photogrammetry. Computers can generate point clouds of virtual three-dimensional objects and scenes. 3D scanning can obtain point clouds of static, real-world three-dimensional objects or scenes, generating millions of point clouds per second. 3D imaging can obtain point clouds of dynamic, real-world three-dimensional objects or scenes, generating tens of millions of point clouds per second. Furthermore, in the medical field, MRI, CT, and electromagnetic positioning information can be used to obtain point clouds of biological tissues and organs. These technologies reduce the cost and time required to acquire point cloud data while improving data accuracy. Changes in point cloud data acquisition methods have made it possible to acquire large amounts of point cloud data. With the continuous accumulation of large-scale point cloud data, efficient storage, transmission, publication, sharing, and standardization of point cloud data have become key to point cloud applications.

[0063] After encoding the point cloud media, the encoded data stream needs to be encapsulated and transmitted to the user. Correspondingly, on the point cloud media player side, the point cloud file needs to be decapsulated first, then decoded, and finally the decoded data stream is presented.

[0064] FIG1 shows a schematic diagram of a system framework of point cloud media encoding and decoding in an application scenario according to an embodiment of the present application.

[0065] The real-world visual scene A can be captured by collecting point cloud data through the collection device 110. The collection device 110 can be, for example, a group of cameras or a camera device with multiple lenses and sensors. The collection result is point cloud source data B, which is a frame sequence consisting of a large number of point cloud frames. The encoder 120 can encode one or more point cloud frames to obtain an encoded G-PCC bit stream, which can specifically include an encoded geometry bit stream and an attribute bit stream E. The file encapsulator 130 can encapsulate one or more encoded bit streams according to a specific media container file format to obtain a media file F for file playback or a series of initialization segments and a media segment Fs for streaming. In some embodiments of the present application, the media container file format can be, for example, the ISO basic media file format specified in ISO / IEC 14496-11 [ISOBMFF]. The file encapsulator 130 can also encapsulate metadata in the media file F or the media segment Fs.

[0066] The media file F output by the file encapsulator 130 is the same as the media file F' input by the file decapsulator 140. The file decapsulator can extract the encoded bit stream E' and parse the metadata by processing the media file F' or processing the received media fragments F's. The decoder 150 can decode the G-PCC bit stream into a decoded signal D' and generate point cloud data based on the decoded signal D'. When applicable, the point cloud data can be rendered and displayed on the screen of a head-mounted display or any other display device by the renderer 160 based on the current viewing position, viewing direction or viewport determined by various types of sensors (such as head, position or eye movement sensors). In addition to being used by the player to access the appropriate part of the decoded point cloud data, the current viewing position or viewing direction can also be used for decoding optimization. In the viewport-related content distributor 170, the current viewing position and viewing direction are also passed to the policy module, which can be used to determine the track to be received.

[0067] In point cloud media transmission technology, streaming transmission technology is often used to handle the transmission of media resources between the server and the client. Common media streaming transmission technologies include DASH (Dynamic Adaptive Streaming over HTTP), HLS (HTTP Live Streaming), SMT (Smart Media Transport), and other technologies.

[0068] Take DASH as an example. DASH is an adaptive bitrate streaming technology that enables high-quality streaming media to be delivered over the Internet through traditional HTTP web servers. DASH breaks content into a series of small HTTP-based file segments, each of which contains a short length of playable content, while the total length of the content may be up to several hours (such as a movie or live sports event). The content will be made into alternative segments with multiple bitrates to provide multiple bitrate versions for selection. When the media content is played by a DASH client, the client will automatically choose which alternative to download and play based on the current network conditions. The client will choose to play the highest bitrate segment that can be downloaded in time, thereby avoiding playback freezes or rebuffering events. As a result, DASH clients can seamlessly adapt to changing network conditions and provide a high-quality playback experience with less freezes and rebuffering.

[0069] DASH uses the existing HTTP web server infrastructure. It allows devices such as Internet TVs, TV set-top boxes, desktop computers, smartphones, tablets, etc. to consume multimedia content (such as video, TV, radio, etc.) delivered over the Internet and can cope with variable Internet reception conditions.

[0070] The following describes in detail the technical solutions provided by this application, including the point cloud file processing method, point cloud file processing device, computer-readable medium, electronic device, and computer program product, in conjunction with specific implementation methods. The various technical solutions of the embodiments of this application can be applied to the server, player, or intermediate nodes of a point cloud media system.

[0071] Figure 2 shows a flowchart of the steps of a method for processing a point cloud file in one embodiment of the present application. This embodiment of the present application uses a method for decoding a point cloud file executed by a client device equipped with a point cloud decoding device as an example. As shown in Figure 2, the method for decoding a point cloud file includes the following steps S210 to S220.

[0072] S210: Acquire a point cloud file carrying a point cloud code stream. The data box of the point cloud code stream includes a unit type field. The unit type field is used to indicate the data type of the data unit in the point cloud code stream.

[0073] S220: Decapsulating and decoding the point cloud file according to the unit type field to obtain point cloud media content corresponding to the data unit.

[0074] The point cloud file may be a media file or a media segment obtained after encoding and encapsulation as shown in FIG1 , and the media file or the media segment carries point cloud media content encapsulated in the form of a point cloud track.

[0075] In one embodiment of the present application, a data source may encapsulate the point cloud media content into a point cloud file containing a single point cloud track, or into a point cloud file containing multiple point cloud tracks, based on the geometric parameter information, attribute parameter information, and parameter information of the point cloud slice contained in the point cloud media content. A point cloud track is a volumetric visual track used to carry a point cloud code stream (e.g., a coded geometry bit stream or a coded attribute bit stream), or may be a volumetric visual track that carries both a coded geometry bit stream and a coded attribute bit stream.

[0076] The data box of the point cloud track may be, for example, a data box based on the ISO Base Media File Format (ISOBMFF). Detailed information about ISOBMFF can be found in the standard ISO / IEC 14496-12.

[0077] A data unit is the basic encapsulation unit obtained by dividing the content of a point cloud sample in a point cloud stream. Different data unit division methods can be used in different encapsulation modes. For example, in a component-based multi-track encapsulation mode, a point cloud sample in a point cloud stream can include multiple data units such as parameter information, geometric data, and attribute data. For another example, in a slice-based multi-track encapsulation mode, a point cloud sample in a point cloud stream can include multiple data units such as a geometry header, an attribute header, a geometry slice header, an attribute slice header, geometry data, and attribute data.

[0078] The following schematically introduces the data structure related to point cloud encoding and decoding in the related technology of this application.

[0079] Table 1 shows the data structure of the compressed point cloud code stream sequence in the related art of this application.

[0080] Table 1

[0081] Table 2 shows the data structure of the attribute header data unit in the related art of this application.

[0082] Table 2

[0083] Table 3 shows the data structure of the attribute slice data unit in the related art of this application.

[0084] Table 3

[0085] Table 4 shows the data structure of the general attribute data code stream in the related art of this application.

[0086] Table 4

[0087] The attribute information flag attributeID[attrIdx][attr] is an unsigned integer. It specifies that the attr-th data of the attribute identified by the attribute index attrIdx uses the attribute information identified by the attribute index attrIdx. This is a number between 0 and attribute_info_num_set_minus1[attrIdx]. When this syntax element does not exist in the codestream, its default value is zero.

[0088] In the related art of this application, as shown in the table above, the data structure of the point cloud stream contains all attribute data in a single point cloud slice. That is, when multiple sets of attribute data exist, such as color and reflectivity, color and reflectivity are considered to belong to the same point cloud slice. However, when encapsulating point cloud files into multiple tracks, different attribute data can be encapsulated into different attribute component tracks, which is inconsistent with the data structure of the related point cloud stream.

[0089] In response to the problems existing in the related art, the embodiments of the present application define a unit type field in the data box of the point cloud code stream, which can indicate the data type of the data unit in the point cloud code stream based on the unit type field. Then, according to the data type indicated by the unit type field, a part of the data units with the specified data type in the point cloud file can be selectively decoded, thereby optimizing the data access flexibility of the point cloud file.

[0090] In one embodiment of the present application, the point cloud samples in the point cloud code stream are defined as follows.

[0091] A point cloud sample may include one or more data units.

[0092] avs_pcc_unit_length indicates the number of bytes of the corresponding data unit.

[0093] avs_pcc_unit indicates the corresponding data unit.

[0094] The value definition of the unit type field avs_pcc_unit_type is shown in Table 5.

[0095] Table 5

[0096] In one embodiment of the present application, the data type of the data unit includes a point cloud attribute slice with an unspecified attribute type, or a point cloud attribute slice with a specified attribute type.

[0097] The point cloud attribute slice of unspecified attribute type may be, for example, the point cloud attribute slice attribute_slice of general attribute data as shown in Table 5, and the point cloud attribute slice of specified attribute type may be, for example, the point cloud color data attribute_slice_color or the point cloud reflectivity data attribute_slice_ref as shown in Table 5.

[0098] The data structures of the point cloud color data attribute_slice_color and the point cloud reflectivity data attribute_slice_ref are shown in Tables 6 and 7 below, respectively.

[0099] Table 6

[0100] Table 7

[0101] The embodiment of the present application indicates through the unit type field that the data type of a data unit is a point cloud attribute slice of an unspecified attribute type, or a point cloud attribute slice of a specified attribute type. When the file is decoded and presented, the attribute data of all attribute types of the point cloud can be accessed separately, or the attribute data of a specified attribute type can be accessed separately.

[0102] In one embodiment of the present application, data units in a point cloud codestream can be encapsulated using the TLV codestream format. Specifically, each data unit (avs_pcc_unit) is defined as a TLV unit. The TLV codestream format, also known as the Type-Length-Value bytestream format, refers to a structure consisting of the data type (Type), the data length (Length), and the data value (Value). For more information about the TLV codestream format, refer to the ISO / IEC 23090-9 standard.

[0103] Table 8 shows the syntax structure of a data unit encapsulated based on the TLV code stream format in one embodiment of the present application.

[0104] Table 8

[0105] Among them, tlv_type corresponds to the unit type field avs_pcc_unit_type in the above point cloud sample definition, tlv_num_payload_bytes corresponds to the number of bytes of the data unit avs_pcc_unit_length in the above point cloud sample definition, and the tlv_payload_byte set is the set of data units avs_pcc_unit.

[0106] For the semantic description of different values ​​of the unit type field in the data unit, please refer to the meaning in Table 5 above.

[0107] In one embodiment of the present application, the point cloud file includes one or more attribute component tracks obtained by encapsulating multiple sets of attribute data.

[0108] When multiple groups of attribute data are encapsulated into an attribute component track, the unit type field indicates that the data type of the data unit in the attribute component track is a point cloud attribute header or a point cloud attribute slice with an unspecified attribute type; for example, an embodiment of the present application can encapsulate N groups of attribute data into an attribute component track. At this time, the data unit in the attribute component track may include two data types, namely: a point cloud attribute header attribute_header, or a point cloud attribute slice attribute_slice with an unspecified attribute type.

[0109] When multiple groups of attribute data are encapsulated into multiple attribute component tracks, the unit type field indicates that the data type of the data unit in the attribute component track is a point cloud attribute header or a point cloud attribute slice of a specified attribute type. For example, an embodiment of the present application can encapsulate N groups of attribute data into multiple attribute component tracks, where each attribute component track can contain M groups of attribute data, where M is an integer greater than or equal to 1 and less than N; that is, an attribute component track contains a portion of attribute data of a specified attribute type. Taking the two attribute types of color and reflectivity as an example, the data unit in one attribute component track can include two data types, namely: point cloud attribute header attribute_header, or point cloud color data attribute_slice_color; the data unit in another attribute component track can include two data types, namely: point cloud attribute header attribute_header, or point cloud reflectivity data attribute_slice_ref.

[0110] In the embodiments of the present application, when adopting different point cloud track encapsulation modes, the unit type field is used to indicate whether the data type of a data unit is a point cloud attribute header or a point cloud attribute slice, and whether the point cloud attribute slice is a specified attribute type, which can further optimize the flexibility of data access selection.

[0111] In one embodiment of the present application, when the data type of the data unit is a point cloud attribute slice of a specified attribute type, the data structure of the point cloud attribute slice includes a slice data type field, and the slice data type field is used to indicate the specified attribute type.

[0112] In one embodiment of the present application, when the data type of the data unit is a point cloud attribute slice of a specified attribute type, the data unit includes at least one of a point cloud attribute slice header and a point cloud attribute data code stream;

[0113] The data structure of the point cloud attribute slice header includes a slice data type field and a slice identifier field. The slice identifier field is used to indicate the identifier of the point cloud attribute slice;

[0114] The data structure of the point cloud attribute data code stream includes a slice data type field and a parameter set flag field. The parameter set flag field is used to indicate the parameter set used when decoding the point cloud attribute data code stream.

[0115] Table 9 shows the data structure of the point cloud attribute slice header attribute_slice_header in one embodiment of the present application.

[0116] Table 9

[0117] Table 10 shows the data structure of the point cloud attribute slice data bitstream general_attribute_data_bitstream in one embodiment of the present application.

[0118] Table 10

[0119] Wherein, slice_id indicates the identifier of the point cloud attribute slice.

[0120] reflQPoffset is used to control the reflectivity attribute quantization parameter, and the value range can be, for example, -32 to 32.

[0121] slice_data_type indicates the type of data contained in the current point cloud attribute slice. For example, a value of 0 indicates that the current point cloud attribute slice contains color data attribute_data_color; a value of 1 indicates that the current point cloud attribute slice contains reflectance data attribute_data_reflectance; and a value of 2 indicates that the current point cloud attribute slice contains mixed color and reflectance data attribute_data_color_ref.

[0122] para_set_index_color indicates the parameter set index used when decoding color data of a point cloud attribute slice containing mixed color and reflectance data. The value of this field can be, for example, 0 to attribute_info_num_set_minus1[0].

[0123] para_set_index_ref indicates the parameter set index used when decoding reflectance data of a point cloud attribute slice containing mixed color and reflectance data. The value of this field can be, for example, 0 to attribute_info_num_set_minus1[1].

[0124] para_set_index indicates the parameter set index used when decoding the current point cloud attribute slice. If the current point cloud attribute slice is color data, the value of this field can be 0 to attribute_info_num_set_minus1[0]; if the current point cloud attribute slice is reflectivity data, the value of this field can be 0 to attribute_info_num_set_minus1[1].

[0125] In one embodiment of the present application, the point cloud file also includes a geometry component track obtained by encapsulating geometric data; the unit type field indicates that the data type of the data unit in the geometry component track is at least one of the point cloud sequence header sequence_header, point cloud geometry header geometry_header, point cloud frame header frame_header or point cloud geometry slice geometry_slice.

[0126] FIG3 shows a point cloud file obtained by performing multi-track packaging based on component types in one embodiment of the present application.

[0127] As shown in Figure 3, the point cloud file includes three point cloud tracks obtained based on component type encapsulation, wherein AVS-PCC Track 1 is a geometry component track, AVS-PCC Track 2 is an attribute component track 1 corresponding to one attribute type (e.g., color attribute), and AVS-PCC Track 3 is an attribute component track 2 corresponding to another attribute type (e.g., reflectivity attribute). In some other optional implementations, AVS-PCC Track 2 and AVS-PCC Track 3 may also be attribute component tracks corresponding to multiple attribute types, for example, AVS-PCC Track 2 includes attribute data of two different attribute types, and AVS-PCC Track 3 includes attribute data of another two different types.

[0128] In one embodiment of the present application, multiple sets of attribute data and geometric data can be encapsulated in the same point cloud track, that is, point cloud file encapsulation can be performed based on a single track mode.

[0129] FIG4 shows a point cloud file obtained based on single-track packaging in an example of the present application.

[0130] As shown in Figure 4, the point cloud file only includes one point cloud track, namely the AVS-PCC track. The point cloud samples in the point cloud track may include data units such as parameter information, geometric data, and attribute data.

[0131] In one embodiment of the present application, the point cloud file includes a point cloud slice track obtained by encapsulating attribute data of multiple point cloud slices.

[0132] When all the attribute data of multiple point cloud slices are encapsulated into a point cloud slice track, the unit type field indicates that the data type of the data unit in the point cloud slice track is a point cloud attribute slice with an unspecified attribute type; for example, the data corresponding to J point cloud slices is encapsulated into a point cloud slice track (assuming there are K point cloud slices in the point cloud frame, J < K), and each point cloud slice track contains all N groups of corresponding attribute component data (i.e., all attribute data) of the corresponding point cloud slice.

[0133] When partial attribute data of multiple point cloud slices are encapsulated into a point cloud slice track, the unit type field indicates that the data type of the data unit in the point cloud slice track is a point cloud attribute slice with a specified attribute type; for example, the data corresponding to J point cloud slices is encapsulated into a point cloud slice track (assuming there are K point cloud slices in the point cloud frame, J < K), and each point cloud slice track only contains M groups of corresponding attribute component data (1 <= M < N, i.e., partial attribute data) of the corresponding point cloud slice

[0134] In an embodiment of the present application, the point cloud file further includes a point cloud slice track obtained by encapsulating the geometric data of multiple point cloud slices; the unit type field indicates that the data type of the data unit in the point cloud slice track is a point cloud geometry slice.

[0135] In an embodiment of the present application, the point cloud file further includes a point cloud slice base track obtained by encapsulating the parameter information of multiple point cloud slices; the unit type field indicates that the data type of the data unit in the point cloud slice base track is at least one of a point cloud sequence header, a point cloud geometry header, a point cloud attribute header, or a point cloud frame header.

[0136] FIG. 5 shows a point cloud file for multi-track encapsulation based on point cloud slices in an embodiment of the present application.

[0137] As shown in FIG. 5, in the mode of multi-track encapsulation based on point cloud slices, the point cloud file includes a point cloud slice base track slice base track, a geometric component track associated with the point cloud slice base track slice base track, and an attribute component track associated with the geometric component track.

[0138] For example, the point cloud file corresponds to three point cloud slices, namely slice1, slice2, and slice3.

[0139] Among them, the samples of the point cloud slice base track slice base track include a geometry header and an attribute header.

[0140] Slice track1 is a geometric component track corresponding to point cloud slices slice1 and slice2, and the samples in this track include the geometry headers and geometric data of point cloud slices slice1 and slice2.

[0141] The slice track2 is an attribute component track corresponding to the point cloud slices slice1 and slice2. The samples in this track include the attribute headers and attribute data of the point cloud slices slice1 and slice2.

[0142] The slice track3 is a geometric component track corresponding to the point cloud slice slice3. The samples in this track include the geometric headers and geometric data of the point cloud slice slice3.

[0143] The slice track4 is an attribute component track corresponding to the point cloud slice slice4. The samples in this track include the attribute headers and attribute data of the point cloud slice slice4.

[0144] In an embodiment of the present application, the attribute data of multiple point cloud slices and the geometric data of multiple point cloud slices can be encapsulated in the same point cloud slice track. For example, the data corresponding to J point cloud slices is encapsulated into one point cloud slice track (assuming there are K point cloud slices in the point cloud frame, J < K), and the geometric data and attribute data corresponding to each point cloud slice are included in the point cloud slice track.

[0145] FIG. 6 shows a point cloud file obtained by simultaneously encapsulating the geometric data and attribute data of point cloud slices in an embodiment of the present application.

[0146] As shown in FIG. 6, in the mode of simultaneously encapsulating the geometric data and attribute data of point cloud slices, the point cloud file includes a point cloud slice base track slice base track, and one or more point cloud slice tracks associated with the point cloud slice base track slice base track. <00003***8>For example, the point cloud file corresponds to three point cloud slices, namely slice1, slice2 and slice3.

[0148] Among them, the samples of the point cloud slice base track slice base track include geometric headers and attribute headers.

[0149] The slice track1 is a point cloud slice track corresponding to the point cloud slices slice1 and slice2. The samples in this track include the geometric headers, geometric data, attribute headers and attribute data of the point cloud slices slice1 and slice2.

[0150] The slice track2 is a point cloud slice track corresponding to the point cloud slice slice3. The samples in this track include the geometric headers, geometric data, attribute headers and attribute data of the point cloud slice slice3.

[0151] In one embodiment of the present application, a point cloud file includes one or more point cloud tracks obtained by encapsulating a point cloud code stream, the point cloud track includes multiple point cloud samples, and the point cloud sample includes one or more sub-samples; the data box of the sub-sample includes a type flag field, and the type flag field is used to indicate the data type of the sub-sample.

[0152] A subsample is a data encapsulation unit in a point cloud sample. Different types of subsamples can be obtained in a point cloud sample based on different division methods. For example, subsamples with different data capacities can be obtained in a point cloud sample based on different dimensions such as data units, spatial blocks, or point cloud subframes.

[0153] In one embodiment of the present application, if the value of the type flag field is a preset first value (for example, a value of 0), the data type of the subsample is the data type indicated by the unit type field; if the value of the type flag field is a second value different from the first value (for example, other optional values ​​different from 0), the data type of the subsample is not the data type indicated by the unit type field.

[0154] In one embodiment of the present application, if the value of the type field is a first numerical value, and the data type indicated by the unit type field is a type related to a point cloud piece, the data box of the subsample also includes a piece identification field, which is used to indicate an identifier of the point cloud piece corresponding to the subsample.

[0155] Based on the sample definition in the embodiment of the present application, a data structure of a sub-sample may be defined accordingly.

[0156] Each point cloud sample can be divided into one or more point cloud subsamples. SubSampleInformationBox is used in the encapsulation of point cloud data. The subsample is defined according to the value of the class flag field (flags) of the subsample information data. The type flag field specifies the data type of the subsample information in this data box.

[0157] When the type flag field is set to 0, it indicates that the subsample is based on the point cloud data type. That is, a subsample contains only one data type defined by avs_pcc_unit_type. The type flag field can also have other reserved flag values.

[0158] The codec_specific_parameters field in SubsampleInformationBox is defined as follows:

[0159] Among them, avs_pcc_unit_type indicates the data type of the point cloud contained in the subsample, and the value meaning and value range are the same as the data definition in the sample.

[0160] In one embodiment of the present application, when a data unit in a point cloud codestream includes a point cloud attribute slice of a specified attribute type, the data box of the point cloud codestream further includes: setting an attribute flag field default_attr_index to indicate whether the parameter set referenced when decoding the point cloud attribute slice is the target parameter set in the point cloud attribute header. The target parameter set can be a pre-specified parameter set in the point cloud header.

[0161] In one embodiment of the present application, the point cloud attribute header of the point cloud codestream includes one or more parameter sets corresponding to a specified attribute type; the target parameter set is the only parameter set corresponding to the specified attribute type in the point cloud attribute header, or the parameter set corresponding to the specified attribute type in a specified position order in the point cloud attribute header. For example, if the current point cloud attribute slice is point cloud color data, the target parameter set can be the only parameter set corresponding to the color attribute in the point cloud attribute header, or it can be the first parameter set corresponding to the color attribute in the point cloud attribute header.

[0162] In one embodiment of the present application, when the data unit in the point cloud code stream includes a point cloud attribute slice of a specified attribute type, the data box of the point cloud code stream also includes: a parameter set flag field attr_para_set_index, which is used to indicate the position order of the parameter set referenced when decoding the point cloud attribute slice in the point cloud attribute header.

[0163] In one embodiment of the present application, the position order is the order of arrangement of sets in multiple parameter sets corresponding to a specified attribute type.

[0164] In one embodiment of the present application, when the data unit in the point cloud code stream includes a point cloud attribute slice of a specified attribute type, the data box of the point cloud code stream also includes: a dynamic attribute flag field dynamic_attr_index, which is used to indicate whether the position order of the parameter set referenced when decoding the point cloud attribute slice in the point cloud attribute header is a fixed value.

[0165] In one embodiment of the present application, when a data unit in a point cloud codestream includes a point cloud attribute slice of a specified attribute type, the data box of the point cloud codestream further includes at least one of the following fields:

[0166] The attribute number field attr_num is used to indicate the number of types of point cloud attribute slices contained in the point cloud codestream; for example, if the point cloud codestream includes two attribute types, color and reflectivity, the attribute number field attr_num should be 2.

[0167] The attribute type field attr_type is used to indicate the attribute type of the point cloud attribute slice contained in the point cloud code stream; for example, when the value of this field is 0, it indicates the color attribute, and when the value is 1, it indicates the reflectivity attribute.

[0168] The attribute identification field attr_id is used to indicate the identifier of the point cloud attribute slice contained in the point cloud code stream.

[0169] In one embodiment of the present application, the attribute parameter set information data box in the point cloud code stream can be defined as follows.

[0170] attr_num indicates the number of attribute components contained in the track.

[0171] attr_type indicates the type of attribute component contained in the track. A value of 0 indicates a color attribute; a value of 1 indicates a reflectance attribute.

[0172] attr_id indicates the identifier of the attribute component data.

[0173] When the value of default_attr_index is 1, it indicates that the parameter set referenced when decoding the current attribute component data is the first parameter set of this type of attribute in the attribute header; when the value is 0, it indicates that the parameter set referenced when decoding the current attribute component data is not the first parameter set of this type of attribute in the attribute header.

[0174] In another implementation method, the meaning of this field can also be defined as follows:

[0175] When the value of default_attr_index is 1, it indicates that the parameter set referenced when decoding the current attribute component data is the only parameter set of this type of attribute in the attribute header; when the value is 0, it indicates that the parameter set referenced when decoding the current attribute component data is not the only parameter set of this type of attribute in the attribute header.

[0176] attr_para_set_index indicates the order of the parameter sets referenced when decoding the current attribute component data in the attribute header. (In another implementation method, it can be further defined that the order is the order under the corresponding attribute type)

[0177] When dynamic_attr_index is 0, it indicates that the order of the parameter sets referenced when decoding the current attribute component data in the attribute header (under this type of attribute) remains unchanged. When it is 1, it indicates that the order of the parameter sets referenced when decoding the current attribute component data in the attribute header (under this type of attribute) will change.

[0178] In one embodiment of the present application, the point cloud codestream includes a parameter header sample group consisting of multiple parameter header samples, and the parameter header sample includes a point cloud geometry header or a point cloud attribute header; when the data unit in the point cloud codestream includes a point cloud attribute slice of a specified attribute type, the data box of the parameter header sample group includes: an attribute identification field for indicating an identifier of the point cloud attribute slice; a parameter set flag field for indicating the position order of the parameter set referenced when decoding the point cloud attribute slice in the point cloud attribute header.

[0179] In one embodiment of the present application, the position order is the order of arrangement of sets in multiple parameter sets corresponding to a specified attribute type.

[0180] In one embodiment of the present application, the data box of the parameter header sample group further includes:

[0181] The parameter set information flag field is used to indicate whether the parameter set referenced when decoding the point cloud attribute slice is the target parameter set in the point cloud attribute header.

[0182] In one embodiment of the present application, if the parameter set information flag field of any point cloud sample in the point cloud code stream takes a preset value, the parameter set information flag field of the synchronization sample in the point cloud code stream takes a preset value; the parameter set information flag field taking a preset value indicates that the parameter set referenced when decoding the point cloud sample is the target parameter set in the point cloud attribute header, and the synchronization sample is a sample that can be directly decoded when decoding the point cloud file.

[0183] In one embodiment of the present application, the data box of the parameter header sample group may be defined as follows.

[0184] attr_id indicates the identifier of the attribute component data.

[0185] attr_para_set_index indicates the order of the parameter sets in the attribute header (under the attribute of this type) that are referenced when decoding the corresponding attribute component data (i.e., the attribute component data corresponding to the corresponding attr_id). (In another implementation method, it can be further defined that the order is the order under the corresponding attribute type)

[0186] The attr_num field takes the value of the corresponding field in the attribute component track.

[0187] At the same time, if the attr_para_set_info_flag value of a sample in the point cloud codestream is 1, the attr_para_set_info_flag value of any synchronized sample of the track must be 1.

[0188] In one embodiment of the present application, when the data unit in the point cloud code stream includes a point cloud attribute slice of a specified attribute type, the data structure of the point cloud attribute slice includes: a parameter set flag field, which is used to indicate the parameter set corresponding to the specified attribute type that is referenced when decoding the point cloud attribute slice.

[0189] Taking color attributes and reflectivity attributes as examples, the data structures of point cloud color data attribute_slice_color and point cloud reflectivity data attribute_slice_ref in the embodiments of the present application are shown in Tables 11 and 12 below, respectively.

[0190] Table 11

[0191] Table 12

[0192] The above embodiments introduce the processing method implemented by the present application at the decoding end of the point cloud file.

[0193] FIG7 is a flowchart illustrating the steps of a method for processing a point cloud file in one embodiment of the present application. This embodiment of the present application uses a method for encoding a point cloud file executed by a client device equipped with a point cloud encoding device as an example. As shown in FIG7 , the method for encoding a point cloud file includes the following steps S710 to S720.

[0194] S710: Acquire point cloud media content;

[0195] S720: Encode and encapsulate the point cloud media content to obtain a point cloud file carrying a point cloud code stream; the data box of the point cloud code stream includes a unit type field, which is used to indicate the data type of the data unit in the point cloud code stream.

[0196] The embodiment of the present application is a method for processing point cloud files implemented at the encoding end. The specific implementation of this method corresponds to the various embodiments at the decoding end described above and will not be repeated here.

[0197] The following is an example of a specific application scenario to illustrate the network transmission process of point cloud data by the server at the encoding end and the client at the decoding end in an embodiment of the present application.

[0198] The point cloud file encapsulation method proposed in the present invention has the following implementation steps:

[0199] 1. The server obtains the point cloud bit stream, which contains geometric data and N sets of attribute data (N>0);

[0200] 2. The server encapsulates the point cloud bitstream into one or more file tracks in one of the following ways:

[0201] 1) Encapsulation is performed based on component types. The geometric data is encapsulated into a point cloud geometry component track, and N groups of attribute data are encapsulated into an attribute component track. The avs point cloud units included in the geometric component track sample can only have avs_pcc_unit_type values ranging from 0 to 1 and 3 to 4. The avs point cloud units included in the attribute component track sample can only have avs_pcc_unit_type values of 2 or 5.

[0202] 2) Encapsulation is performed based on component types. The geometric data is encapsulated into a point cloud geometry component track, and N groups of attribute data are encapsulated into M attribute component tracks (1 <= M < N). The avs point cloud units included in the geometric component track sample can only have avs_pcc_unit_type values ranging from 0 to 1 and 3 to 4. The avs point cloud units included in the attribute component track sample can only have avs_pcc_unit_type values of 2 or 6 to 7.

[0203] 3) Encapsulation is performed based on point cloud patches. The data corresponding to J point cloud patches is encapsulated into a point cloud patch track (assuming there are K point cloud patches in the point cloud frame, J < K), and each point cloud patch track only contains the geometric component data corresponding to the respective point cloud patch or N groups of attribute component data. The avs point cloud units included in the point cloud patch base track sample can only have avs_pcc_unit_type values ranging from 0 to 3. If the point cloud patch track corresponds to geometric components, the avs point cloud units included in the sample can only have avs_pcc_unit_type values of 4. If the point cloud patch track corresponds to attribute components, the avs point cloud units included in the sample can only have avs_pcc_unit_type values of 5.

[0204] 4) Encapsulation is performed based on point cloud patches. The data corresponding to J point cloud patches is encapsulated into a point cloud patch track (assuming there are K point cloud patches in the point cloud frame, J < K), and each point cloud patch track only contains the geometric component data corresponding to the respective point cloud patch or M groups of attribute component data (1 <= M < N). The avs point cloud units included in the point cloud patch base track sample can only have avs_pcc_unit_type values ranging from 0 to 3. If the point cloud patch track corresponds to geometric components, the avs point cloud units included in the sample can only have avs_pcc_unit_type values of 4. If the point cloud patch track corresponds to attribute components, the avs point cloud units included in the sample can only have avs_pcc_unit_type values of 6 or 7.

[0205] 5) Encapsulation is performed based on point cloud slices. The data corresponding to J point cloud slices (assuming there are K point cloud slices in a point cloud frame, and J < K) is encapsulated into a point cloud slice track, and the geometric and attribute data corresponding to each point cloud slice is included in the point cloud slice track. For the avs point cloud units included in the basic point cloud slice track sample, the value of avs_pcc_unit_type can only be 0 to 3. For the avs point cloud units included in the point cloud slice track sample, the value of avs_pcc_unit_type can only be 4, 5 or 4, 6 to 7.

[0206] 6) Single-track encapsulation, encapsulating the point cloud bitstream into a file track. For the avs point cloud units included in the point cloud track sample, the value of avs_pcc_unit_type can only be 0 to 4, 5 or 0 to 4, 6 to 7.

[0207] 3. While the server adopts the two different encapsulation methods in 2, corresponding metadata information is added to indicate the necessary information for decoding the file track, including:

[0208] 1) Define sample and subsample information.

[0209] 2) Define the parameter set information data box.

[0210] 3) Modify the metadata information indicated by the parameter header sample group.

[0211] 4. According to the transmission method between the server and the client,

[0212] 1) The server directly transmits the point cloud file F to the client;

[0213] 2) The server slices the point cloud file F to obtain the Fs set. Each Fs slice contains the corresponding slices of X file tracks. According to the user's requirements, the file track data required by the user in the corresponding slices is transmitted to the user.

[0214] 5. According to the transmission method between the server and the client,

[0215] 1) The client receives the complete file F1. When the client unpacks, decodes and presents the file, according to the performance of the user device and the requirements of the presentation effect, combined with the corresponding metadata information in the file, all or part of the tracks in F1 are selected for decoding and presentation.

[0216] 2) The client receives the sliced file Fs. The client unpacks and decodes the received file slices, and finally presents the media content.

[0217] During the decoding process, the client can selectively extract the corresponding point cloud slices and parameter headers for decoding based on the information in the sample and subsample. The parameter information required for decoding attribute data can be found based on the order of the parameter sets corresponding to the attribute data in the attribute header.

[0218] Based on the introduction of the above application scenarios, it can be seen that the solution proposed in the embodiment of the present application can flexibly encapsulate the point cloud bitstream into one or more file tracks and support partial access to samples within the track.

[0219] It should be noted that although the steps of the method of the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all steps must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0220] The following describes an embodiment of a device of the present application, which can be used to execute the method for processing point cloud files in the above-mentioned embodiment of the present application. FIG8 schematically shows a block diagram of the structure of a point cloud file processing device at the decoding end provided in an embodiment of the present application. As shown in FIG8 , the point cloud file decoding device 800 may include:

[0221] A first acquisition module 810 is configured to acquire a point cloud file carrying a point cloud code stream, wherein a data box of the point cloud code stream includes a unit type field, and the unit type field is used to indicate a data type of a data unit in the point cloud code stream;

[0222] The decoding module 820 is configured to decapsulate and decode the point cloud file according to the unit type field to obtain point cloud media content corresponding to the data unit.

[0223] FIG9 schematically shows a block diagram of a point cloud file processing device provided by an embodiment of the present application. As shown in FIG9 , the point cloud file encoding device 900 may include:

[0224] A second acquisition module 910 is configured to acquire point cloud media content;

[0225] The encoding module 920 is configured to encode and encapsulate the point cloud media content to obtain a point cloud file carrying a point cloud code stream; the data box of the point cloud code stream includes a unit type field, and the unit type field is used to indicate the data type of the data unit in the point cloud code stream.

[0226] The specific details of the point cloud file processing device provided in each embodiment of the present application have been described in detail in the corresponding method embodiments and will not be repeated here.

[0227] FIG10 schematically shows a block diagram of a computer system structure of an electronic device for implementing an embodiment of the present application.

[0228] It should be noted that the computer system 1000 of the electronic device shown in FIG10 is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0229] As shown in Figure 10, the computer system 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1002 or the program loaded from the storage part 1008 into the random access memory (RAM) 1003. Various programs and data required for system operation are also stored in the random access memory 1003. The CPU 1001, the read-only memory 1002, and the random access memory 1003 are connected to each other via a bus 1004. An input / output interface 1005 (i.e., an I / O interface) is also connected to the bus 1004.

[0230] The following components are connected to the input / output interface 1005: an input section 1006 including a keyboard, a mouse, and the like; an output section 1007 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 1008 including a hard disk; and a communication section 1009 including a network interface card such as a local area network card or a modem. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the input / output interface 1005 as needed. Removable media 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 1010 as needed, so that computer programs read therefrom can be installed into the storage section 1008 as needed.

[0231] In particular, according to an embodiment of the present application, the processes described in the various method flow charts can be implemented as computer software programs. For example, an embodiment of the present application includes a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for executing the methods shown in the flow charts. In such an embodiment, the computer program can be downloaded and installed from a network via the communication portion 1009 and / or installed from a removable medium 1011. When the computer program is executed by the central processing unit 1001, the various functions defined in the system of the present application are performed.

[0232] It should be noted that the computer-readable medium shown in the embodiments of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0233] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0234] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiment of the application, the features and functions of two or more modules or units described above can be concretized in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0235] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present application.

[0236] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of this application and include common knowledge or customary techniques in the art that are not disclosed herein.

[0237] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for processing a point cloud file, characterized in that: include: Acquire a point cloud file encapsulating a point cloud code stream, wherein a data box of the point cloud code stream includes a unit type field, and the unit type field is used to indicate a data type of a data unit in the point cloud code stream; The point cloud file is decapsulated and selectively decoded according to the unit type field to obtain point cloud media content corresponding to the data unit with the specified data type.

2. The method for processing a point cloud file according to claim 1, characterized in that: The data type of the data unit includes a point cloud attribute slice of an unspecified attribute type or a point cloud attribute slice of a specified attribute type.

3. The method for processing point cloud files according to claim 2, characterized in that: The point cloud file includes one or more attribute component tracks obtained by encapsulating multiple sets of attribute data; When the multiple groups of attribute data are encapsulated into one attribute component track, the unit type field indicates that the data type of the data unit in the attribute component track is a point cloud attribute header or a point cloud attribute slice with unspecified attribute type; When the multiple groups of attribute data are encapsulated into a plurality of the attribute component tracks, the unit type field indicates that the data type of the data unit in the attribute component track is a point cloud attribute header or a point cloud attribute slice of a specified attribute type.

4. The method for processing a point cloud file according to claim 3, characterized in that: The point cloud file also includes a geometry component track obtained by encapsulating geometry data; the unit type field indicates that the data type of the data unit in the geometry component track is at least one of a point cloud sequence header, a point cloud geometry header, a point cloud frame header or a point cloud geometry piece.

5. The method for processing point cloud files according to claim 4, characterized in that: The multiple sets of attribute data and the geometric data are encapsulated in the same point cloud track.

6. The method for processing point cloud files according to claim 2, characterized in that: When the data type of the data unit is a point cloud attribute slice of the specified attribute type, the data structure of the point cloud attribute slice includes a slice data type field, and the slice data type field is used to indicate the specified attribute type.

7. The method for processing a point cloud file according to claim 6, characterized in that: When the data type of the data unit is a point cloud attribute slice of the specified attribute type, the data unit includes at least one of a point cloud attribute slice header and a point cloud attribute data code stream; The data structure of the point cloud attribute slice header includes the slice data type field and a slice identifier field, and the slice identifier field is used to indicate the identifier of the point cloud attribute slice; The data structure of the point cloud attribute data code stream includes the slice data type field and a parameter set flag field, and the parameter set flag field is used to indicate the parameter set used when decoding the point cloud attribute data code stream.

8. The method for processing point cloud files according to claim 2, characterized in that: The point cloud file includes a point cloud track obtained by encapsulating attribute data of a plurality of point cloud slices; When all attribute data of the plurality of point cloud slices are encapsulated into one point cloud slice track, the unit type field indicates that a data type of a data unit in the point cloud slice track is a point cloud attribute slice of unspecified attribute type; When partial attribute data of the plurality of point cloud slices are encapsulated into one point cloud slice track, the unit type field indicates that the data type of the data unit in the point cloud slice track is a point cloud attribute slice of a specified attribute type.

9. The method for processing a point cloud file according to claim 8, characterized in that: The point cloud file also includes a point cloud slice track obtained by encapsulating the geometric data of the multiple point cloud slices; the unit type field indicates that the data type of the data unit in the point cloud slice track is a point cloud geometric slice.

10. The method for processing point cloud files according to claim 9, characterized in that: The attribute data of the plurality of point cloud slices and the geometric data of the plurality of point cloud slices are encapsulated in the same point cloud slice track.

11. The method for processing a point cloud file according to claim 8, characterized in that: The point cloud file also includes a point cloud slice basic track obtained by encapsulating parameter information of the multiple point cloud slices; the unit type field indicates that the data type of the data unit in the point cloud slice basic track is at least one of a point cloud sequence header, a point cloud geometry header, a point cloud attribute header or a point cloud frame header.

12. The method for processing a point cloud file according to any one of claims 1 to 11, characterized in that: The point cloud file includes one or more point cloud tracks obtained by encapsulating the point cloud code stream, the point cloud track includes multiple point cloud samples, and the point cloud sample includes one or more sub-samples; the data box of the sub-sample includes a type flag field, and the type flag field is used to indicate the data type of the sub-sample.

13. The method for processing a point cloud file according to claim 12, characterized in that: If the value of the type flag field is a preset first value, the data type of the subsample is the data type indicated by the unit type field; If the value of the type flag field is a second value different from the first value, the data type of the subsample is not the data type indicated by the unit type field.

14. The method for processing a point cloud file according to claim 13, characterized in that: If the value of the type field is the first numerical value, and the data type indicated by the unit type field is a type related to a point cloud piece, then the data box of the subsample also includes a piece identification field, and the piece identification field is used to indicate an identifier of the point cloud piece corresponding to the subsample.

15. The method for processing a point cloud file according to any one of claims 1 to 11, characterized in that: When the data unit in the point cloud code stream includes a point cloud attribute slice of a specified attribute type, the data box of the point cloud code stream further includes: The attribute flag field is set to indicate whether the parameter set referenced when decoding the point cloud attribute slice is the target parameter set in the point cloud attribute header.

16. The method for processing a point cloud file according to claim 15, characterized in that: The point cloud attribute header of the point cloud code stream includes one or more parameter sets corresponding to the specified attribute type; the target parameter set is the only parameter set corresponding to the specified attribute type in the point cloud attribute header, or the parameter set in a specified position order corresponding to the specified attribute type in the point cloud attribute header.

17. The method for processing a point cloud file according to claim 16, characterized in that: When the data unit in the point cloud code stream includes a point cloud attribute slice of a specified attribute type, the data box of the point cloud code stream further includes: The parameter set flag field is used to indicate the position order of the parameter set referenced when decoding the point cloud attribute slice in the point cloud attribute header.

18. The method for processing a point cloud file according to claim 17, characterized in that: The position order is the order in which the sets are arranged in a plurality of parameter sets corresponding to the specified attribute type.

19. The method for processing a point cloud file according to claim 15, characterized in that: When the data unit in the point cloud code stream includes a point cloud attribute slice of a specified attribute type, the data box of the point cloud code stream further includes: The dynamic attribute flag field is used to indicate whether the position order of the parameter set referenced when decoding the point cloud attribute slice in the point cloud attribute header is a fixed value.

20. The method for processing a point cloud file according to claim 15, characterized in that: When the data unit in the point cloud codestream includes a point cloud attribute slice of a specified attribute type, the data box of the point cloud codestream further includes at least one of the following fields: The attribute quantity field is used to indicate the number of types of the point cloud attribute slices contained in the point cloud code stream; The attribute type field is used to indicate the attribute type of the point cloud attribute slice contained in the point cloud code stream; The attribute identification field is used to indicate the identifier of the point cloud attribute slice contained in the point cloud code stream.

21. The method for processing a point cloud file according to any one of claims 1 to 11, characterized in that: The point cloud codestream includes a parameter header sample group consisting of a plurality of parameter header samples, wherein the parameter header sample includes a point cloud geometry header or a point cloud attribute header; when a data unit in the point cloud codestream includes a point cloud attribute slice of a specified attribute type, a data box of the parameter header sample group includes: An attribute identification field, used to indicate an identifier of the point cloud attribute slice; The parameter set flag field is used to indicate the position order of the parameter set referenced when decoding the point cloud attribute slice in the point cloud attribute header.

22. The method for processing a point cloud file according to claim 21, characterized in that: The position order is the order in which the sets are arranged in a plurality of parameter sets corresponding to the specified attribute type.

23. The method for processing a point cloud file according to claim 21, characterized in that: The data box of the parameter header sample group also includes: The parameter set information flag field is used to indicate whether the parameter set referenced when decoding the point cloud attribute slice is the target parameter set in the point cloud attribute header.

24. The method for processing a point cloud file according to claim 23, characterized in that: If the parameter set information flag field value of any point cloud sample in the point cloud code stream is a preset value, the parameter set information flag field value of the synchronization sample in the point cloud code stream is the preset value; The parameter set information flag field whose value is the preset value indicates that the parameter set referenced when decoding the point cloud sample is the target parameter set in the point cloud attribute header, and the synchronization sample is a sample that can be directly decoded when decoding the point cloud file.

25. The method for processing a point cloud file according to any one of claims 1 to 11, characterized in that: When the data unit in the point cloud code stream includes a point cloud attribute slice of a specified attribute type, the data structure of the point cloud attribute slice includes: The parameter set flag field is used to indicate the parameter set corresponding to the specified attribute type that is referenced when decoding the point cloud attribute slice.

26. A method for processing a point cloud file, characterized in that: include: Get point cloud media content; Encoding and packaging the point cloud media content to obtain a point cloud file carrying a point cloud code stream; The data box of the point cloud codestream includes a unit type field, and the unit type field is used to indicate the data type of the data unit in the point cloud codestream.

27. A point cloud file processing device, characterized in that: include: A first acquisition module is configured to acquire a point cloud file carrying a point cloud code stream, wherein a data box of the point cloud code stream includes a unit type field, and the unit type field is used to indicate a data type of a data unit in the point cloud code stream; The decoding module is configured to decapsulate and selectively decode the point cloud file according to the unit type field to obtain point cloud media content corresponding to the data unit with the specified data type.

28. A point cloud file processing device, characterized in that: include: A second acquisition module is configured to acquire point cloud media content; The encoding module is configured to encode and encapsulate the point cloud media content to obtain a point cloud file carrying a point cloud code stream; the data box of the point cloud code stream includes a unit type field, and the unit type field is used to indicate the data type of the data unit in the point cloud code stream.

29. A computer readable medium, characterized in that The computer-readable medium stores a computer program, and when the computer program is executed by a processor, the method for processing a point cloud file according to any one of claims 1 to 26 is implemented.

30. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the executable instructions to implement the point cloud file processing method as described in any one of claims 1 to 26.

31. A program product, characterized in that include: A computer program, which, when executed by a processor, implements the method for processing a point cloud file as described in any one of claims 1 to 26.

Citation Information

Patent Citations

  • Grouping-based point cloud code stream packaging method and system

    CN111435991A

  • Coding and decoding method of point cloud media and related products

    CN114697668A

  • Three-dimensional point cloud transmission method, receiving method, transmission device, receiving device and storage medium

    CN115348243A

  • Data processing method of point cloud media and related equipment

    CN115396646A

  • Method and apparatus for storage and signaling of sub-sample entry descriptions

    US20210105492A1