Point cloud file processing method and device, medium and electronic equipment

By defining unit type fields in the data box of the point cloud code stream, the problem of poor data access flexibility caused by data encapsulation of multiple attribute types in point cloud files is solved, and flexible decoding of point cloud files is realized, and data access efficiency is improved. .

CN119071571APending Publication Date: 2024-12-03TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310634707.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-31
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Data of multiple different attribute types in point cloud files are encapsulated in the same data structure, resulting in poor data access flexibility and the inability to flexibly decode data units of specific data types.

Method used

The unit type field is defined in the data box of the point cloud code stream, and the data type of the data unit is indicated according to the field, so as to realize selective decoding of some data types of the point cloud file.

Benefits of technology

Optimizes the data access flexibility of point cloud files, allowing selective decoding of data units of specified data types according to the unit type field, improving the flexibility and efficiency of data access.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119071571A_ABST
    Figure CN119071571A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of point cloud media, and particularly relates to a point cloud file processing method, a point cloud file processing device, a computer readable medium, electronic equipment and a computer program product. The method comprises the steps that a point cloud file carrying a point cloud code stream is acquired, a data box of the point cloud code stream comprises a unit type field, and the unit type field is used for indicating the data type of a data unit in the point cloud code stream; and decapsulating and decoding the point cloud file according to the unit type field to obtain the point cloud media content corresponding to the data unit. According to the embodiment of the invention, the data access flexibility of the point cloud file can be optimized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the field of point cloud media technology, and specifically relates to a point cloud file processing method, a point cloud file processing device, a computer-readable medium, an electronic device, and a computer program product. Background Art

[0002] Point cloud is a set of irregularly distributed discrete points in space that express the spatial structure and surface properties of a three-dimensional object or scene. After acquiring large-scale point cloud data through point cloud acquisition equipment, the point cloud data can be encoded and packaged for transmission to users, and decoded and presented on the user side.

[0003] In the data code stream of the point cloud, if data of multiple different attribute types are encapsulated in the same data structure, the data structure needs to be fully decoded to obtain data of all attribute types, resulting in poor data access flexibility. Summary of the invention

[0004] The present application provides a point cloud file processing method, a point cloud file processing device, a computer-readable medium, an electronic device, and a computer program product, the purpose of which is to optimize the data access flexibility of the point cloud file.

[0005] According to one aspect of an embodiment of the present application, a method for processing a point cloud file is provided, the method comprising:

[0006] Acquire a point cloud file carrying a point cloud code stream, wherein a data box of the point cloud code stream includes a unit type field, and the unit type field is used to indicate a data type of a data unit in the point cloud code stream;

[0007] The point cloud file is decapsulated and decoded according to the unit type field to obtain point cloud media content corresponding to the data unit.

[0008] According to one aspect of an embodiment of the present application, a point cloud file processing device is provided, the device comprising:

[0009] A first acquisition module is configured to acquire a point cloud file carrying a point cloud code stream, wherein a data box of the point cloud code stream includes a unit type field, and the unit type field is used to indicate a data type of a data unit in the point cloud code stream;

[0010] The decoding module is configured to decapsulate and decode the point cloud file according to the unit type field to obtain point cloud media content corresponding to the data unit.

[0011] According to one aspect of an embodiment of the present application, a method for processing a point cloud file is provided, the method comprising:

[0012] Get point cloud media content;

[0013] The point cloud media content is encoded and packaged to obtain a point cloud file carrying a point cloud code stream; the data box of the point cloud code stream includes a unit type field, and the unit type field is used to indicate the data type of the data unit in the point cloud code stream.

[0014] According to one aspect of an embodiment of the present application, a point cloud file processing device is provided, the device comprising:

[0015] A second acquisition module is configured to acquire point cloud media content;

[0016] The encoding module is configured to encode and encapsulate the point cloud media content to obtain a point cloud file carrying a point cloud code stream; the data box of the point cloud code stream includes a unit type field, and the unit type field is used to indicate the data type of the data unit in the point cloud code stream.

[0017] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for processing a point cloud file in the above technical solution is implemented.

[0018] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the executable instructions to implement a method for processing a point cloud file as in the above technical solution.

[0019] According to one aspect of an embodiment of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method for processing a point cloud file in the above technical solution is implemented.

[0020] In the technical solution provided in the embodiment of the present application, by defining a unit type field in the data box of the point cloud code stream, the data type of the data unit in the point cloud code stream can be indicated based on the unit type field, and then according to the data type indicated by the unit type field, a part of the data units with the specified data type in the point cloud file can be selectively decoded, thereby optimizing the data access flexibility of the point cloud file. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A schematic diagram of the system framework of point cloud media encoding and decoding in an application scenario of an embodiment of the present application is shown.

[0023] Figure 2 A flowchart showing the steps of a method for processing a point cloud file in one embodiment of the present application is shown.

[0024] Figure 3 A point cloud file obtained by performing multi-track packaging based on component type in one embodiment of the present application is shown.

[0025] Figure 4 A point cloud file obtained based on single track packaging in an example of the present application is shown.

[0026] Figure 5 A point cloud file that is packaged in multiple tracks based on point cloud slices in one embodiment of the present application is shown.

[0027] Figure 6 A point cloud file obtained by simultaneously encapsulating the geometric data and attribute data of a point cloud slice through a single track in one embodiment of the present application is shown.

[0028] Figure 7 A flowchart showing the steps of a method for processing a point cloud file in one embodiment of the present application is shown.

[0029] Figure 8 The structural block diagram of the point cloud file processing device at the decoding end provided in an embodiment of the present application is schematically shown.

[0030] Fig. 9 The structural block diagram of the point cloud file processing device of the encoding end provided in an embodiment of the present application is schematically shown.

[0031] Fig.10 The structure block diagram of a computer system suitable for implementing an electronic device of an embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0032] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0033] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0034] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0035] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0036] In the specific implementation of this application, user-related data such as the transmission content, decoding content and consumption content of point cloud media are involved. When the various embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0037] The relevant terms or abbreviations involved in the embodiments of the present application are explained as follows.

[0038] AVS: Audio Video Coding Standard, China's national video coding standard AVS.

[0039] MPEG: Moving Picture Experts Group, an organization established by ISO (International Standardization Organization) and IEC (International Electrotechnical Commission) to develop international standards for moving picture and speech compression.

[0040] ISOBMFF: ISO Based Media File Format, a media file format based on ISO standards. ISOBMFF is a media file packaging standard, and the most typical ISOBMFF file is the MP4 file.

[0041] Media Segment: A media segment that conforms to a certain media format and can be played. When playing, it may need to be coordinated with zero or more previous segments and the initialization segment.

[0042] PCC: Point Cloud Compression, point cloud compression.

[0043] V-PCC: Video-based Point Cloud Compression, point cloud compression based on traditional video coding method.

[0044] G-PCC: Geometry-based Point Cloud Compression, point cloud compression based on geometric model.

[0045] AVS-PCC: Point cloud compression based on China's national video coding standard AVS.

[0046] Track: A track is a collection of media data in the process of media file encapsulation, consisting of multiple time-sequenced samples. A media file can consist of one or more tracks. For example, a media file can contain a video media track, an audio media track, and a subtitle media track. In particular, metadata information can also be included in the file as a media type in the form of a metadata media track.

[0047] Sample: Sample is the encapsulation unit in the process of media file encapsulation. A track consists of many samples, and each sample corresponds to specific timestamp information. For example, a video media track can consist of many samples, and a sample is usually a video frame. In the embodiment of the present application, a sample in a point cloud media track can be a point cloud frame.

[0048] Slice: A point cloud slice / point cloud strip, which represents a collection of syntax elements (such as geometry slices and attribute slices) of part or all of the encoded point cloud frame data.

[0049] Sequence Header: Point cloud sequence header parameter set, the parameter set required for point cloud sequence decoding.

[0050] (AVS definition)

[0051] Geometry Header: Point cloud frame geometry header parameter set, the parameter set required for decoding point cloud geometry data.

[0052] Attribute Header: Point cloud frame attribute header parameter set, the parameter set required for decoding point cloud attribute data.

[0053] Sample Number: The number of a specific sample. The first sample in a track has number 1.

[0054] Sample Entry: Sample entry, used to indicate metadata information related to all samples in the track. For example, the sample entry of a video track usually contains metadata information related to decoder initialization.

[0055] Sample Group: Sample group, used to group some samples in the track according to specific rules.

[0056] A point cloud is a set of irregularly distributed discrete points in space that express the spatial structure and surface properties of a three-dimensional object or scene. Each point in a point cloud has at least three-dimensional position information, and may also have color, material or other attribute information depending on the application scenario. Usually, each point in a point cloud has the same number of additional attributes.

[0057] Point cloud media can be divided into point cloud media V-PCC based on traditional video coding, point cloud media G-PCC based on geometric features, and point cloud compression based on China's national video coding standard AVS-PCC. In the file packaging of point cloud media, the three-dimensional position information is usually called the geometry component of the point cloud media file, and the attribute information is called the attribute component of the point cloud media file. A point cloud media file has only one geometry component, but can have one or more attribute components.

[0058] Point clouds can flexibly and conveniently express the spatial structure and surface properties of three-dimensional objects or scenes, and therefore have a wide range of applications, including virtual reality (VR) games, computer-aided design (CAD), geographic information system (GIS), autonomous navigation system (ANS), digital cultural heritage, free viewpoint broadcasting, three-dimensional immersive remote presentation, and three-dimensional reconstruction of biological tissues and organs.

[0059] Point clouds can be obtained mainly through the following methods: computer generation, 3D laser scanning, 3D photogrammetry, etc. Computers can generate point clouds of virtual three-dimensional objects and scenes. 3D scanning can obtain point clouds of static real-world three-dimensional objects or scenes, and can obtain millions of point clouds per second. 3D photography can obtain point clouds of dynamic real-world three-dimensional objects or scenes, and can obtain tens of millions of point clouds per second. In addition, in the medical field, point clouds of biological tissues and organs can be obtained from MRI, CT, and electromagnetic positioning information. These technologies reduce the cost and time cycle of point cloud data acquisition and improve the accuracy of data. The change in the way point cloud data is acquired makes it possible to obtain a large amount of point cloud data. With the continuous accumulation of large-scale point cloud data, the efficient storage, transmission, publication, sharing and standardization of point cloud data have become the key to point cloud applications.

[0060] After encoding the point cloud media, the encoded data stream needs to be encapsulated and transmitted to the user. Correspondingly, on the point cloud media player side, the point cloud file needs to be decapsulated first, then decoded, and finally the decoded data stream is presented.

[0061] Figure 1 A schematic diagram of the system framework of point cloud media encoding and decoding in an application scenario of an embodiment of the present application is shown.

[0062] The point cloud data acquisition through the acquisition device 110 can capture the real-world visual scene A, and the acquisition device 110 can be, for example, a group of cameras or a camera device with multiple lenses and sensors. The acquisition result is the point cloud source data B, and the point cloud source data B is a frame sequence composed of a large number of point cloud frames. The encoder 120 can encode one or more point cloud frames to obtain an encoded G-PCC bit stream, which can specifically include an encoded geometry bit stream and an attribute bit stream E. The file encapsulator 130 can encapsulate one or more encoded bit streams according to a specific media container file format to obtain a media file F for file playback or a series of initialization segments and a media segment Fs for streaming. In some embodiments of the present application, the media container file format can be, for example, the ISO basic media file format specified in ISO / IEC 14496-11 [ISOBMFF]. The file encapsulator 130 can also encapsulate metadata in the media file F or the media segment Fs.

[0063] The media file F output by the file encapsulator 130 is the same as the media file F' input by the file decapsulator 140. The file decapsulator can extract the encoded bitstream E' and parse the metadata by processing the media file F' or processing the received media fragments F's. The decoder 150 can decode the G-PCC bitstream into a decoded signal D' and generate point cloud data based on the decoded signal D'. When applicable, the point cloud data can be rendered and displayed on the screen of a head-mounted display or any other display device by the renderer 160 based on the current viewing position, viewing direction or viewport determined by various types of sensors (such as head, position or eye movement sensors). In addition to being used by the player to access the appropriate part of the decoded point cloud data, the current viewing position or viewing direction can also be used for decoding optimization. In the viewport-related content distributor 170, the current viewing position and viewing direction are also passed to the policy module, which can be used to determine the track to be received.

[0064] In point cloud media transmission technology, streaming transmission technology is usually used to handle the transmission of media resources between the server and the client. Common media streaming transmission technologies include DASH (Dynamic Adaptive Streaming over HTTP), HLS (HTTP Live Streaming), SMT (Smart Media Transport) and other technologies.

[0065] Take DASH as an example. DASH is an adaptive bitrate streaming technology that enables high-quality streaming media to be delivered over the Internet through traditional HTTP network servers. DASH breaks the content into a series of small HTTP-based file fragments, each of which contains a short length of playable content, while the total length of the content may be up to several hours (such as a movie or live sports event). The content will be made into alternative fragments with multiple bit rates to provide multiple bitrate versions for selection. When the media content is played by the DASH client, the client will automatically choose which alternative to download and play based on the current network conditions. The client will choose the highest bitrate segment that can be downloaded in time for playback, thereby avoiding playback freezes or rebuffering events. Because of this, the DASH client can seamlessly adapt to changing network conditions and provide a high-quality playback experience with less freezes and rebuffering rates.

[0066] DASH uses the existing HTTP web server infrastructure. It allows devices such as Internet TVs, TV set-top boxes, desktop computers, smartphones, tablets, etc. to consume multimedia content (such as video, TV, radio, etc.) delivered over the Internet and cope with variable Internet reception conditions.

[0067] The following is a detailed description of the point cloud file processing method, point cloud file processing device, computer readable medium, electronic device, computer program product and other technical solutions provided by the present application in combination with specific implementation methods. The various technical solutions of the embodiments of the present application can be applied to the server side, player side or intermediate node of the point cloud media system.

[0068] Figure 2 The flowchart of the step of the method for processing a point cloud file in one embodiment of the present application is shown. The present application embodiment takes the method for decoding a point cloud file executed by a client device equipped with a point cloud decoding device as an example. Figure 2 As shown, the decoding method of the point cloud file includes the following steps S210 to S220.

[0069] S210: Acquire a point cloud file carrying a point cloud code stream, where a data box of the point cloud code stream includes a unit type field, and the unit type field is used to indicate a data type of a data unit in the point cloud code stream.

[0070] S220: Decapsulating and decoding the point cloud file according to the unit type field to obtain point cloud media content corresponding to the data unit.

[0071] Point cloud files can be Figure 1 The media file or media segment obtained after encoding and encapsulation shown in the figure carries point cloud media content encapsulated in the form of point cloud tracks.

[0072] In one embodiment of the present application, the data source may encapsulate the point cloud media content into a point cloud file containing a single point cloud track, or may encapsulate the point cloud file into a point cloud file containing multiple point cloud tracks, based on the geometric parameter information, attribute parameter information, and parameter information of the point cloud slice contained in the point cloud media content. The point cloud track is a volumetric visual track for carrying a point cloud code stream (e.g., a coded geometry bit stream or a coded attribute bit stream), or may be a volumetric visual track that carries both a coded geometry bit stream and a coded attribute bit stream.

[0073] The data box of the point cloud track may be, for example, a data box based on the ISO Base Media File Format (ISOBMFF). For detailed information of ISOBMFF, reference may be made to the standard ISO / IEC 14496-12.

[0074] The data unit is the basic encapsulation unit obtained by dividing the content of the point cloud sample in the point cloud code stream. Different data unit division methods can be used in different encapsulation modes. For example, in the component-based multi-track encapsulation mode, a point cloud sample in the point cloud code stream can include multiple data units such as parameter information, geometric data, and attribute data. For another example, in the slice-based multi-track encapsulation mode, a point cloud sample in the point cloud code stream can include multiple data units such as geometry header, attribute header, geometry slice header, attribute slice header, geometry data, and attribute data.

[0075] The following schematically introduces the data structure related to point cloud encoding and decoding in the related technology of this application.

[0076] Table 1 shows the data structure of the compressed point cloud code stream sequence in the related art of this application.

[0077] Table 1

[0078]

[0079] Table 2 shows the data structure of the attribute header data unit in the related art of this application.

[0080] Table 2

[0081]

[0082]

[0083]

[0084] Table 3 shows the data structure of the attribute slice data unit in the related art of this application.

[0085] Table 3

[0086]

[0087] Table 4 shows the data structure of the general attribute data code stream in the related art of this application.

[0088] Table 4

[0089]

[0090]

[0091] Among them, the attribute information flag attributeID[attrIdx][attr] is an unsigned integer. It specifies that the attr-th data of the attribute determined by the attribute index attrIdx uses the attribute information of the attributeID[attrIdx][attr]-th attribute determined by the attribute index attrIdx, which is a number between 0 and attribute_info_num_set_minus1[attrIdx]. When the syntax element does not exist in the bitstream, its default value is zero.

[0092] In the related technology of this application, as shown in the table above, the data structure of the point cloud code stream contains all the attribute data in one point cloud slice, that is, when there are multiple sets of attribute data, such as color and reflectivity, color and reflectivity will be considered to belong to the same point cloud slice. When the point cloud file is packaged in multiple tracks, different attribute data can be packaged into different attribute component tracks, which is inconsistent with the data structure of the related point cloud code stream.

[0093] In response to the problems existing in the related technology, the embodiments of the present application define a unit type field in the data box of the point cloud code stream, and can indicate the data type of the data unit in the point cloud code stream based on the unit type field. Then, according to the data type indicated by the unit type field, a part of the data units with the specified data type in the point cloud file can be selectively decoded, thereby optimizing the data access flexibility of the point cloud file.

[0094] In one embodiment of the present application, the point cloud samples in the point cloud code stream are defined as follows.

[0095]

[0096] Among them, a point cloud sample may include one or more data units.

[0097] avs_pcc_unit_length indicates the number of bytes of the corresponding data unit.

[0098] avs_pcc_unit indicates the corresponding data unit.

[0099] The value definition of the unit type field avs_pcc_unit_type is shown in Table 5.

[0100] Table 5

[0101] avs_pcc_unit_type meaning 0 Point cloud sequence header sequence_header() 1 Point cloud geometry header geometry_header() 2 Point cloud attribute header attribute_header() 3 Point cloud frame header frame_header() 4 Point cloud geometry slice geometry_slice() 5 Point cloud attribute slice attribute_slice() 6 Point cloud color data attribute_slice_color() 7 Point cloud reflectivity data attribute_slice_ref()

[0102] In one embodiment of the present application, the data type of the data unit includes a point cloud attribute slice with an unspecified attribute type, or a point cloud attribute slice with a specified attribute type.

[0103] The point cloud attribute slice of unspecified attribute type may be, for example, the point cloud attribute slice attribute_slice of general attribute data as shown in Table 5, and the point cloud attribute slice of specified attribute type may be, for example, the point cloud color data attribute_slice_color or the point cloud reflectivity data attribute_slice_ref as shown in Table 5.

[0104] The data structures of the point cloud color data attribute_slice_color and the point cloud reflectivity data attribute_slice_ref are shown in Tables 6 and 7 below, respectively.

[0105] Table 6

[0106]

[0107] Table 7

[0108]

[0109] The embodiment of the present application indicates through the unit type field that the data type of a data unit is a point cloud attribute slice of an unspecified attribute type, or a point cloud attribute slice of a specified attribute type. When the file is decoded and presented, the attribute data of all attribute types of the point cloud can be accessed separately, or the attribute data of the specified attribute type can be accessed separately.

[0110] In one embodiment of the present application, the data unit in the point cloud code stream can be encapsulated in the TLV code stream format, that is, one data unit avs_pcc_unit is defined as one tlv unit. The TLV code stream format, that is, the Type-length-value bytestream format, refers to a structure consisting of the data type Type, the data length Length and the data value Value. For specific information about the TLV code stream format, please refer to the standard ISO / IEC 23090-9.

[0111] Table 8 shows the syntax structure of a data unit encapsulated based on the TLV code stream format in an embodiment of the present application.

[0112] Table 8

[0113]

[0114] Among them, tlv_type corresponds to the unit type field avs_pcc_unit_type in the above point cloud sample definition, tlv_num_payload_bytes corresponds to the number of bytes of the data unit avs_pcc_unit_length in the above point cloud sample definition, and the tlv_payload_byte set is the set of data units avs_pcc_unit.

[0115] For the semantic description of different values ​​of the unit type field in the data unit, please refer to the meaning in the above Table 5.

[0116] In one embodiment of the present application, the point cloud file includes one or more attribute component tracks obtained by encapsulating multiple sets of attribute data.

[0117] When multiple groups of attribute data are encapsulated into an attribute component track, the unit type field indicates that the data type of the data unit in the attribute component track is a point cloud attribute header or a point cloud attribute slice with an unspecified attribute type; for example, an embodiment of the present application can encapsulate N groups of attribute data into an attribute component track, at which time the data unit in the attribute component track may include two data types, namely: a point cloud attribute header attribute_header, or a point cloud attribute slice attribute_slice with an unspecified attribute type.

[0118] When multiple groups of attribute data are encapsulated into multiple attribute component tracks, the unit type field indicates that the data type of the data unit in the attribute component track is a point cloud attribute header or a point cloud attribute slice of a specified attribute type. For example, an embodiment of the present application can encapsulate N groups of attribute data into multiple attribute component tracks, where each attribute component track can contain M groups of attribute data, where M is an integer greater than or equal to 1 and less than N; that is, an attribute component track contains a portion of attribute data of a specified attribute type. Taking the two attribute types of color and reflectivity as examples, the data unit in one attribute component track can include two data types, namely: point cloud attribute header attribute_header, or point cloud color data attribute_slice_color; the data unit in another attribute component track can include two data types, namely: point cloud attribute header attribute_header, or point cloud reflectivity data attribute_slice_ref.

[0119] In the embodiments of the present application, when adopting different point cloud track encapsulation modes, the unit type field is used to indicate whether the data type of a data unit is a point cloud attribute header or a point cloud attribute slice, and whether the point cloud attribute slice is a specified attribute type, which can further optimize the flexibility of data access selection.

[0120] In one embodiment of the present application, when the data type of the data unit is a point cloud attribute slice of a specified attribute type, the data structure of the point cloud attribute slice includes a slice data type field, and the slice data type field is used to indicate the specified attribute type.

[0121] In one embodiment of the present application, when the data type of the data unit is a point cloud attribute slice of a specified attribute type, the data unit includes at least one of a point cloud attribute slice header and a point cloud attribute data code stream;

[0122] The data structure of the point cloud attribute slice header includes a slice data type field and a slice identifier field, and the slice identifier field is used to indicate the identifier of the point cloud attribute slice;

[0123] The data structure of the point cloud attribute data code stream includes a slice data type field and a parameter set flag field. The parameter set flag field is used to indicate the parameter set used when decoding the point cloud attribute data code stream.

[0124] Table 9 shows the data structure of the point cloud attribute slice header attribute_slice_header in one embodiment of the present application.

[0125] Table 9

[0126]

[0127]

[0128] Table 10 shows the data structure of the point cloud attribute slice data bitstream general_attribute_data_bitstream in one embodiment of the present application.

[0129] Table 10

[0130]

[0131] Wherein, slice_id indicates the identifier of the point cloud attribute slice.

[0132] reflQPoffset is used to control the reflectivity attribute quantization parameter, and the value range may be, for example, -32 to 32.

[0133] slice_data_type indicates the type of data contained in the current point cloud attribute slice. For example, a value of 0 indicates that the current point cloud attribute slice contains color data attribute_data_color; a value of 1 indicates that the current point cloud attribute slice contains reflectivity data attribute_data_reflectance; a value of 2 indicates that the current point cloud attribute slice contains mixed color and reflectivity data attribute_data_color_ref.

[0134] para_set_index_color indicates the parameter set index used when decoding color data of a point cloud attribute slice containing mixed color and reflectance data. The value of this field can be, for example, 0 to attribute_info_num_set_minus1[0].

[0135] para_set_index_ref indicates the parameter set index used when decoding reflectance data of a point cloud attribute slice containing mixed color and reflectance data. The value of this field can be, for example, 0 to attribute_info_num_set_minus1[1].

[0136] para_set_index indicates the parameter set number used when decoding the current point cloud attribute slice. If the current point cloud attribute slice is color data, the value of this field can be 0 to attribute_info_num_set_minus1[0]; if the current point cloud attribute slice is reflectivity data, the value of this field can be 0 to attribute_info_num_set_minus1[1].

[0137] In one embodiment of the present application, the point cloud file also includes a geometry component track obtained by encapsulating geometric data; the unit type field indicates that the data type of the data unit in the geometry component track is at least one of a point cloud sequence header sequence_header, a point cloud geometry header geometry_header, a point cloud frame header frame_header, or a point cloud geometry slice geometry_slice.

[0138] Figure 3 A point cloud file obtained by performing multi-track packaging based on component type in one embodiment of the present application is shown.

[0139] like Figure 3 As shown, the point cloud file includes three point cloud tracks obtained based on component type encapsulation, wherein AVS-PCCTrack 1 is a geometry component track, AVS-PCCTrack 2 is an attribute component track 1 corresponding to one attribute type (e.g., color attribute), and AVS-PCCTrack 3 is an attribute component track 2 corresponding to another attribute type (e.g., reflectivity attribute). In some other optional implementations, AVS-PCCTrack 2 and AVS-PCCTrack 3 may also be attribute component tracks corresponding to multiple attribute types, for example, AVS-PCCTrack 2 includes attribute data of two different attribute types, and AVS-PCCTrack 3 includes attribute data of another two different types.

[0140] In one embodiment of the present application, multiple groups of attribute data and geometric data can be encapsulated in the same point cloud track, that is, point cloud file encapsulation can be performed based on the single-track mode.

[0141] Figure 4 Fig. shows a point cloud file obtained by single-track encapsulation in an example of the present application.

[0142] As Figure 4 shown, the point cloud file includes only one point cloud track, that is, the AVS-PCC track. The point cloud samples in this point cloud track can include data units such as parameter information, geometric data, and attribute data.

[0143] In one embodiment of the present application, the point cloud file includes a point cloud slice track obtained by encapsulating the attribute data of multiple point cloud slices.

[0144] When all the attribute data of multiple point cloud slices is encapsulated into one point cloud slice track, the unit type field indicates that the data type of the data unit in the point cloud slice track is a point cloud attribute slice with an unspecified attribute type; for example, the data corresponding to J point cloud slices is encapsulated into one point cloud slice track (assuming there are K point cloud slices in the point cloud frame, J < K), and each point cloud slice track contains N groups of attribute component data corresponding to the corresponding point cloud slice (i.e., all the attribute data).

[0145] When part of the attribute data of multiple point cloud slices is encapsulated into one point cloud slice track, the unit type field indicates that the data type of the data unit in the point cloud slice track is a point cloud attribute slice with a specified attribute type; for example, the data corresponding to J point cloud slices is encapsulated into one point cloud slice track (assuming there are K point cloud slices in the point cloud frame, J < K), and each point cloud slice track contains only M groups of attribute component data corresponding to the corresponding point cloud slice (1 <= M < N, that is, part of the attribute data)

[0146] In one embodiment of the present application, the point cloud file further includes a point cloud slice track obtained by encapsulating the geometric data of multiple point cloud slices; the unit type field indicates that the data type of the data unit in the point cloud slice track is a point cloud geometry slice.

[0147] In one embodiment of the present application, the point cloud file further includes a point cloud slice base track obtained by encapsulating the parameter information of multiple point cloud slices; the unit type field indicates that the data type of the data unit in the point cloud slice base track is at least one of a point cloud sequence header, a point cloud geometry header, a point cloud attribute header, or a point cloud frame header.

[0148] Figure 5 Fig. shows a point cloud file for multi-track encapsulation based on point cloud slices in an embodiment of the present application.

[0149] As Figure 5As shown, in the mode of multi-track encapsulation based on point cloud slices, the point cloud file includes a point cloud slice base track slicebasetrack, a geometric component track associated with the point cloud slice base track slicebasetrack, and an attribute component track associated with the geometric component track.

[0150] For example, the point cloud file corresponds to three point cloud slices, namely slice1, slice2, and slice3.

[0151] Among them, the samples of the point cloud slice base track slicebasetrack include a geometric header and an attribute header.

[0152] slicetrack1 is the geometric component track corresponding to point cloud slices slice1 and slice2, and the samples in this track include the geometric headers and geometric data of point cloud slices slice1 and slice2.

[0153] slicetrack2 is the attribute component track corresponding to point cloud slices slice1 and slice2, and the samples in this track include the attribute headers and attribute data of point cloud slices slice1 and slice2.

[0154] slicetrack3 is the geometric component track corresponding to point cloud slice slice3, and the samples in this track include the geometric header and geometric data of point cloud slice slice3.

[0155] slicetrack4 is the attribute component track corresponding to point cloud slice slice4, and the samples in this track include the attribute header and attribute data of point cloud slice slice4.

[0156] In an embodiment of the present application, the attribute data of multiple point cloud slices and the geometric data of multiple point cloud slices can be encapsulated in the same point cloud slice track. For example, the data corresponding to J point cloud slices is encapsulated into one point cloud slice track (assuming there are K point cloud slices in the point cloud frame, J < K), and the geometric data and attribute data corresponding to each point cloud slice are included in the point cloud slice track.

[0157] Figure 6 Shows a point cloud file obtained by simultaneously encapsulating the geometric data and attribute data of point cloud slices in an embodiment of the present application.

[0158] As Figure 6 shown, in the mode of simultaneously encapsulating the geometric data and attribute data of point cloud slices, the point cloud file includes a point cloud slice base track slicebasetrack, and one or more point cloud slice tracks associated with the point cloud slice base track slicebasetrack.

[0159] For example, the point cloud file corresponds to three point cloud slices, namely slice1, slice2 and slice3.

[0160] Among them, the sample of the point cloud slice base track slicebasetrack includes a geometry head and an attribute head.

[0161] Slicetrack1 is a point cloud slice track corresponding to point cloud slices slice1 and slice2. The samples in this track include the geometry header, geometry data, attribute header, and attribute data of point cloud slices slice1 and slice2.

[0162] Slicetrack2 is a point cloud slice track corresponding to point cloud slice slice3. The samples in this track include the geometry header, geometry data, attribute header, and attribute data of point cloud slice slice3.

[0163] In one embodiment of the present application, a point cloud file includes one or more point cloud tracks obtained by encapsulating a point cloud code stream, the point cloud track includes multiple point cloud samples, the point cloud sample includes one or more sub-samples; the data box of the sub-sample includes a type flag field, and the type flag field is used to indicate the data type of the sub-sample.

[0164] A subsample is a data encapsulation unit in a point cloud sample. Different types of subsamples can be obtained in a point cloud sample based on different division methods. For example, subsamples with different data capacities can be obtained in a point cloud sample based on different dimensions such as data units, spatial blocks, or point cloud subframes.

[0165] In one embodiment of the present application, if the value of the type flag field is a preset first value (for example, a value of 0), the data type of the subsample is the data type indicated by the unit type field; if the value of the type flag field is a second value different from the first value (for example, other optional values ​​different from 0), the data type of the subsample is not the data type indicated by the unit type field.

[0166] In one embodiment of the present application, if the value of the type field is a first numerical value, and the data type indicated by the unit type field is a type related to a point cloud piece, the data box of the subsample also includes a piece identification field, and the piece identification field is used to indicate an identifier of the point cloud piece corresponding to the subsample.

[0167] Based on the sample definition in the embodiment of the present application, a data structure of a sub-sample may be defined accordingly.

[0168] Each point cloud sample can be divided into one or more point cloud sub-samples. SubSampleInformationBox is used in the encapsulation of point cloud data. The sub-sample is defined according to the value of the class flag field (flags) of the sub-sample information data. The type flag field specifies the data type of the sub-sample information in this data box.

[0169] When the type flag field takes a value of 0, it indicates that the subsample is a subsample based on the point cloud data type. That is, a subsample contains only one data type defined by avs_pcc_unit_type. The type flag field may also have other reserved flag values.

[0170]

[0171] Among them, avs_pcc_unit_type indicates the data type of the point cloud contained in the subsample, and the value meaning and value range are the same as the data definition in the sample.

[0172] In one embodiment of the present application, when the data unit in the point cloud code stream includes a point cloud attribute slice of a specified attribute type, the data box of the point cloud code stream further includes: setting an attribute flag field default_attr_index, which is used to indicate whether the parameter set referenced when decoding the point cloud attribute slice is the target parameter set in the point cloud attribute header. The target parameter set can be a pre-specified parameter set in the point cloud header.

[0173] In one embodiment of the present application, the point cloud attribute header of the point cloud code stream includes one or more parameter sets corresponding to the specified attribute type; the target parameter set is the only parameter set corresponding to the specified attribute type in the point cloud attribute header, or the parameter set of the specified position order corresponding to the specified attribute type in the point cloud attribute header. For example, if the current point cloud attribute slice is point cloud color data, the target parameter set can be the only parameter set corresponding to the color attribute in the point cloud attribute header, or it can be the first parameter set corresponding to the color attribute in the point cloud attribute header.

[0174] In one embodiment of the present application, when the data unit in the point cloud code stream includes a point cloud attribute slice of a specified attribute type, the data box of the point cloud code stream also includes: a parameter set flag field attr_para_set_index, which is used to indicate the position order of the parameter set referenced when decoding the point cloud attribute slice in the point cloud attribute header.

[0175] In one embodiment of the present application, the position order is the order of arrangement of sets in a plurality of parameter sets corresponding to a specified attribute type.

[0176] In one embodiment of the present application, when the data unit in the point cloud code stream includes a point cloud attribute slice of a specified attribute type, the data box of the point cloud code stream also includes: a dynamic attribute flag field dynamic_attr_index, which is used to indicate whether the position order of the parameter set referenced when decoding the point cloud attribute slice in the point cloud attribute header is a fixed value.

[0177] In one embodiment of the present application, when the data unit in the point cloud code stream includes a point cloud attribute slice of a specified attribute type, the data box of the point cloud code stream further includes at least one of the following fields:

[0178] The attribute quantity field attr_num is used to indicate the number of types of point cloud attribute slices contained in the point cloud code stream; for example, if the point cloud code stream includes two attribute types, color and reflectivity, the attribute quantity field attr_num should take a value of 2.

[0179] The attribute type field attr_type is used to indicate the attribute type of the point cloud attribute slice contained in the point cloud code stream; for example, when the value of this field is 0, it indicates the color attribute, and when the value is 1, it indicates the reflectivity attribute.

[0180] The attribute identification field attr_id is used to indicate the identifier of the point cloud attribute slice contained in the point cloud code stream.

[0181] In one embodiment of the present application, the attribute parameter set information data box in the point cloud code stream may be defined as follows.

[0182]

[0183]

[0184] attr_num indicates the number of attribute components contained in the track.

[0185] attr_type indicates the type of attribute component contained in the track. A value of 0 indicates a color attribute; a value of 1 indicates a reflectivity attribute.

[0186] attr_id indicates the identifier of the attribute component data.

[0187] When the value of default_attr_index is 1, it indicates that the parameter set referenced when decoding the current attribute component data is the first parameter set of this type of attribute in the attribute header; when the value is 0, it indicates that the parameter set referenced when decoding the current attribute component data is not the first parameter set of this type of attribute in the attribute header.

[0188] In another implementation method, the meaning of this field can also be defined as follows:

[0189] When the value of default_attr_index is 1, it indicates that the parameter set referenced when decoding the current attribute component data is the only parameter set of this type of attribute in the attribute header; when the value is 0, it indicates that the parameter set referenced when decoding the current attribute component data is not the only parameter set of this type of attribute in the attribute header.

[0190] attr_para_set_index indicates the order of the parameter set referenced in the attribute header when decoding the current attribute component data. (In another implementation method, it can be further defined that the order is the order under the corresponding attribute type)

[0191] When dynamic_attr_index is 0, it indicates that the order of the parameter set referenced when decoding the current attribute component data in the attribute header (under this type of attribute) remains unchanged. When it is 1, it indicates that the order of the parameter set referenced when decoding the current attribute component data in the attribute header (under this type of attribute) will change.

[0192] In one embodiment of the present application, a point cloud code stream includes a parameter header sample group consisting of multiple parameter header samples, and the parameter header samples include a point cloud geometry header or a point cloud attribute header; when a data unit in the point cloud code stream includes a point cloud attribute slice of a specified attribute type, the data box of the parameter header sample group includes: an attribute identification field, used to indicate an identifier of the point cloud attribute slice; a parameter set flag field, used to indicate the position order of the parameter set referenced when decoding the point cloud attribute slice in the point cloud attribute header.

[0193] In one embodiment of the present application, the position order is the order of arrangement of sets in a plurality of parameter sets corresponding to a specified attribute type.

[0194] In one embodiment of the present application, the data box of the parameter header sample group further includes:

[0195] The parameter set information flag field is used to indicate whether the parameter set referenced when decoding the point cloud attribute slice is the target parameter set in the point cloud attribute header.

[0196] In one embodiment of the present application, if there is any point cloud sample in the point cloud code stream whose parameter set information flag field value is a preset value, then the parameter set information flag field value of the synchronization sample in the point cloud code stream is the preset value; the parameter set information flag field whose value is the preset value indicates that the parameter set referenced when decoding the point cloud sample is the target parameter set in the point cloud attribute header, and the synchronization sample is a sample that can be directly decoded when decoding the point cloud file.

[0197] In one embodiment of the present application, the data box of the parameter header sample group may be defined as follows.

[0198]

[0199] attr_id indicates the identifier of the attribute component data.

[0200] attr_para_set_index indicates the order of the parameter set in the attribute header (under the attribute of this type) that is referenced when decoding the corresponding attribute component data (i.e., the attribute component data corresponding to the corresponding attr_id). (In another implementation method, it can be further defined that the order is the order under the corresponding attribute type)

[0201] The attr_num field takes the value of the corresponding field in the attribute component track.

[0202] At the same time, if there is a sample in the point cloud code stream whose attr_para_set_info_flag value is 1, the attr_para_set_info_flag value of any synchronized sample of the track must be 1.

[0203] In one embodiment of the present application, when the data unit in the point cloud code stream includes a point cloud attribute slice of a specified attribute type, the data structure of the point cloud attribute slice includes: a parameter set flag field, which is used to indicate the parameter set corresponding to the specified attribute type that is referenced when decoding the point cloud attribute slice.

[0204] Taking color attributes and reflectivity attributes as examples, the data structures of point cloud color data attribute_slice_color and point cloud reflectivity data attribute_slice_ref in the embodiments of the present application are respectively shown in Tables 11 and 12 below.

[0205] Table 11

[0206]

[0207] Table 12

[0208]

[0209] The above embodiments introduce the processing method implemented by the present application at the decoding end of the point cloud file.

[0210] Figure 7 The flowchart of the step of the method for processing a point cloud file in one embodiment of the present application is shown. The present application embodiment takes the method for encoding a point cloud file executed by a client device equipped with a point cloud encoding device as an example. Figure 7 As shown, the encoding method of the point cloud file includes the following steps S710 to S720.

[0211] S710: Acquire point cloud media content;

[0212] S720: Encodes and encapsulates the point cloud media content to obtain a point cloud file carrying a point cloud bitstream; the data box of the point cloud bitstream includes a unit type field, and the unit type field is used to indicate the data type of the data unit in the point cloud bitstream.

[0213] The method for processing the point cloud file according to the embodiments of the present application is implemented at the encoding end. The specific implementation manner of this method corresponds to each of the above embodiments at the decoding end and will not be elaborated here.

[0214] The following takes a specific application scenario as an example to illustrate the network transmission process of point cloud data between the server at the encoding end and the client at the decoding end according to the embodiments of the present application.

[0215] The method for encapsulating the point cloud file proposed by the present invention includes the following implementation steps:

[0216] 1. The server obtains a point cloud bitstream, and the point cloud bitstream includes geometric data and N groups of attribute data (N>0);

[0217] 2. The server encapsulates the point cloud bitstream into one or more file tracks in one of the following ways:

[0218] 1) Encapsulation based on component type: encapsulate the geometric data into a point cloud geometry component track, and encapsulate the N groups of attribute data into an attribute component track. The avs point cloud units included in the geometric component track sample can only have avs_pcc_unit_type values of 0 to 1, 3 to 4. The avs point cloud units included in the attribute component track sample can only have avs_pcc_unit_type values of 2 or 5.

[0219] 2) Encapsulation based on component type: encapsulate the geometric data into a point cloud geometry component track, and encapsulate the N groups of attribute data into M attribute component tracks (1 <= M < N). The avs point cloud units included in the geometric component track sample can only have avs_pcc_unit_type values of 0 to 1, 3 to 4. The avs point cloud units included in the attribute component track sample can only have avs_pcc_unit_type values of 2 or 6 to 7.

[0220] 3) Encapsulation is performed based on point cloud patches. The data corresponding to J point cloud patches is encapsulated into a point cloud patch track (assuming there are K point cloud patches in the point cloud frame, J < K), and each point cloud patch track only contains the geometric component data corresponding to the respective point cloud patch or N groups of attribute component data. The avs point cloud units included in the point cloud patch base track sample can only have avs_pcc_unit_type values ranging from 0 to 3. If the point cloud patch track corresponds to geometric components, the avs point cloud units included in the sample can only have avs_pcc_unit_type values of 4; if the point cloud patch track corresponds to attribute components, the avs point cloud units included in the sample can only have avs_pcc_unit_type values of 5.

[0221] 4) Encapsulation is performed based on point cloud patches. The data corresponding to J point cloud patches is encapsulated into a point cloud patch track (assuming there are K point cloud patches in the point cloud frame, J < K), and each point cloud patch track only contains the geometric component data corresponding to the respective point cloud patch or M groups of attribute component data (1 <= M < N). The avs point cloud units included in the point cloud patch base track sample can only have avs_pcc_unit_type values ranging from 0 to 3. If the point cloud patch track corresponds to geometric components, the avs point cloud units included in the sample can only have avs_pcc_unit_type values of 4; if the point cloud patch track corresponds to attribute components, the avs point cloud units included in the sample can only have avs_pcc_unit_type values of 6 or 7.

[0222] 5) Encapsulation is performed based on point cloud patches. The data corresponding to J point cloud patches is encapsulated into a point cloud patch track (assuming there are K point cloud patches in the point cloud frame, J < K), and the point cloud patch track contains the geometric and attribute data corresponding to each point cloud patch. The avs point cloud units included in the point cloud patch base track sample can only have avs_pcc_unit_type values ranging from 0 to 3. The avs point cloud units included in the point cloud patch track sample can only have avs_pcc_unit_type values of 4, 5 or 4, 6 - 7.

[0223] 6) Single-track encapsulation, where the point cloud bitstream is encapsulated into a file track. The avs point cloud units included in the point cloud track sample can only have avs_pcc_unit_type values ranging from 0 - 4, 5 or 0 - 4, 6 - 7.

[0224] 3. While the server adopts the two different encapsulation methods in 2, corresponding metadata information is added to indicate the necessary information for decoding the file track, including:

[0225] 1) Define sample and subsample information.

[0226] 2) Define the parameter set information data box.

[0227] 3) Modify the metadata information indicated by the parameter header sample group.

[0228] 4. According to the transmission method between the server and the client,

[0229] 1) The server transmits the point cloud file F directly to the client;

[0230] 2) The server slices the point cloud file F to obtain a set Fs. Each Fs slice contains the corresponding slices of X file tracks. According to the user's needs, the server transmits the file track data required by the user in the corresponding slice to the user.

[0231] 5. According to the transmission method between the server and the client,

[0232] 1) The client receives the complete file F1. When decapsulating, decoding and presenting the file, the client selects all or part of the tracks in F1 for decoding and presentation based on the performance of the user's device and the requirements of the presentation effect, combined with the corresponding metadata information in the file.

[0233] 2) The client receives the slice file Fs. The client decapsulates and decodes the received file slices and finally presents the media content.

[0234] In the above decoding process, the client can selectively extract the corresponding point cloud slices and parameter headers for decoding according to the information in the samples and subsamples. Among them, for the parameter information required for decoding the attribute data, the corresponding parameters can be found according to the order of the parameter set corresponding to the attribute data in the attribute header.

[0235] Based on the introduction of the above application scenarios, it can be seen that the solution proposed in the embodiment of the present application can flexibly encapsulate the point cloud bitstream into one or more file tracks and support partial access to samples within the track.

[0236] It should be noted that although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0237] The following introduces an embodiment of the device of the present application, which can be used to execute the method for processing point cloud files in the above-mentioned embodiment of the present application. Figure 8 The structure block diagram of the point cloud file processing device of the decoding end provided by the embodiment of the present application is schematically shown. Figure 8 As shown, the decoding device 800 of the point cloud file may include:

[0238] A first acquisition module 810 is configured to acquire a point cloud file carrying a point cloud code stream, wherein a data box of the point cloud code stream includes a unit type field, and the unit type field is used to indicate a data type of a data unit in the point cloud code stream;

[0239] The decoding module 820 is configured to decapsulate and decode the point cloud file according to the unit type field to obtain point cloud media content corresponding to the data unit.

[0240] Fig. 9 The structure block diagram of the point cloud file processing device of the encoding end provided by the embodiment of the present application is schematically shown. Fig. 9 As shown, the point cloud file encoding device 900 may include:

[0241] The second acquisition module 910 is configured to acquire point cloud media content;

[0242] The encoding module 920 is configured to encode and encapsulate the point cloud media content to obtain a point cloud file carrying a point cloud code stream; the data box of the point cloud code stream includes a unit type field, and the unit type field is used to indicate the data type of the data unit in the point cloud code stream.

[0243] The specific details of the point cloud file processing device provided in each embodiment of the present application have been described in detail in the corresponding method embodiment and will not be repeated here.

[0244] Fig.10 The structure block diagram of a computer system for implementing an electronic device according to an embodiment of the present application is schematically shown.

[0245] It should be noted that Fig.10 The computer system 1000 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0246] like Fig.10As shown, the computer system 1000 includes a central processing unit 1001 (CPU), which can perform various appropriate actions and processes according to the program stored in the read-only memory 1002 (ROM) or the program loaded from the storage part 1008 to the random access memory 1003 (RAM). Various programs and data required for system operation are also stored in the random access memory 1003. The central processing unit 1001, the read-only memory 1002 and the random access memory 1003 are connected to each other through a bus 1004. The input / output interface 1005 (Input / Output interface, i.e., I / O interface) is also connected to the bus 1004.

[0247] The following components are connected to the input / output interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the input / output interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage section 1008 as needed.

[0248] In particular, according to an embodiment of the present application, the process described in each method flow chart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program contains a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the central processor 1001, various functions defined in the system of the present application are executed.

[0249] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0250] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0251] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0252] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0253] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary technical means in the art that are not disclosed in the present application.

[0254] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for processing a point cloud file, characterized in that: include: Acquire a point cloud file encapsulating a point cloud code stream, wherein a data box of the point cloud code stream includes a unit type field, and the unit type field is used to indicate a data type of a data unit in the point cloud code stream; The point cloud file is decapsulated and decoded according to the unit type field to obtain point cloud media content corresponding to the data unit.

2. The method for processing a point cloud file according to claim 1, characterized in that: The data type of the data unit includes a point cloud attribute slice of an unspecified attribute type or a point cloud attribute slice of a specified attribute type.

3. The method for processing point cloud files according to claim 2, characterized in that: The point cloud file includes one or more attribute component tracks obtained by encapsulating multiple sets of attribute data; When the multiple groups of attribute data are encapsulated into one attribute component track, the unit type field indicates that the data type of the data unit in the attribute component track is a point cloud attribute header or a point cloud attribute slice with unspecified attribute type; When the multiple groups of attribute data are encapsulated into a plurality of the attribute component tracks, the unit type field indicates that the data type of the data unit in the attribute component track is a point cloud attribute header or a point cloud attribute slice of a specified attribute type.

4. The method for processing a point cloud file according to claim 3, characterized in that: The point cloud file also includes a geometry component track obtained by encapsulating geometry data; the unit type field indicates that the data type of the data unit in the geometry component track is at least one of a point cloud sequence header, a point cloud geometry header, a point cloud frame header or a point cloud geometry piece.

5. The method for processing point cloud files according to claim 4, characterized in that: The multiple sets of attribute data and the geometric data are encapsulated in the same point cloud track.

6. The method for processing point cloud files according to claim 2, characterized in that: When the data type of the data unit is a point cloud attribute slice of the specified attribute type, the data structure of the point cloud attribute slice includes a slice data type field, and the slice data type field is used to indicate the specified attribute type.

7. The method for processing a point cloud file according to claim 6, characterized in that: When the data type of the data unit is a point cloud attribute slice of the specified attribute type, the data unit includes at least one of a point cloud attribute slice header and a point cloud attribute data code stream; The data structure of the point cloud attribute slice header includes the slice data type field and a slice identifier field, and the slice identifier field is used to indicate the identifier of the point cloud attribute slice; The data structure of the point cloud attribute data code stream includes the slice data type field and a parameter set flag field, and the parameter set flag field is used to indicate the parameter set used when decoding the point cloud attribute data code stream.

8. The method for processing point cloud files according to claim 2, characterized in that: The point cloud file includes a point cloud track obtained by encapsulating attribute data of a plurality of point cloud slices; When all attribute data of the plurality of point cloud slices are encapsulated into one point cloud slice track, the unit type field indicates that a data type of a data unit in the point cloud slice track is a point cloud attribute slice of unspecified attribute type; When partial attribute data of the plurality of point cloud slices are encapsulated into one point cloud slice track, the unit type field indicates that the data type of the data unit in the point cloud slice track is a point cloud attribute slice of a specified attribute type.

9. The method for processing a point cloud file according to claim 8, characterized in that: The point cloud file also includes a point cloud slice track obtained by encapsulating the geometric data of the multiple point cloud slices; the unit type field indicates that the data type of the data unit in the point cloud slice track is a point cloud geometric slice.

10. The method for processing point cloud files according to claim 9, characterized in that: The attribute data of the plurality of point cloud slices and the geometric data of the plurality of point cloud slices are encapsulated in the same point cloud slice track.

11. The method for processing a point cloud file according to claim 8, characterized in that: The point cloud file also includes a point cloud slice basic track obtained by encapsulating parameter information of the multiple point cloud slices; the unit type field indicates that the data type of the data unit in the point cloud slice basic track is at least one of a point cloud sequence header, a point cloud geometry header, a point cloud attribute header or a point cloud frame header.

12. The method for processing a point cloud file according to any one of claims 1 to 11, characterized in that: The point cloud file includes one or more point cloud tracks obtained by encapsulating the point cloud code stream, the point cloud track includes multiple point cloud samples, and the point cloud sample includes one or more sub-samples; the data box of the sub-sample includes a type flag field, and the type flag field is used to indicate the data type of the sub-sample.

13. The method for processing a point cloud file according to claim 12, characterized in that: If the value of the type flag field is a preset first value, the data type of the subsample is the data type indicated by the unit type field; If the value of the type flag field is a second value different from the first value, the data type of the subsample is not the data type indicated by the unit type field.

14. The method for processing a point cloud file according to claim 13, characterized in that: If the value of the type field is the first numerical value, and the data type indicated by the unit type field is a type related to a point cloud piece, then the data box of the subsample also includes a piece identification field, and the piece identification field is used to indicate an identifier of the point cloud piece corresponding to the subsample.

15. The method for processing a point cloud file according to any one of claims 1 to 11, characterized in that: When the data unit in the point cloud code stream includes a point cloud attribute slice of a specified attribute type, the data box of the point cloud code stream further includes: The attribute flag field is set to indicate whether the parameter set referenced when decoding the point cloud attribute slice is the target parameter set in the point cloud attribute header.

16. The method for processing a point cloud file according to claim 15, characterized in that: The point cloud attribute header of the point cloud code stream includes one or more parameter sets corresponding to the specified attribute type; the target parameter set is the only parameter set corresponding to the specified attribute type in the point cloud attribute header, or the parameter set in a specified position order corresponding to the specified attribute type in the point cloud attribute header.

17. The method for processing a point cloud file according to claim 16, characterized in that: When the data unit in the point cloud code stream includes a point cloud attribute slice of a specified attribute type, the data box of the point cloud code stream further includes: The parameter set flag field is used to indicate the position order of the parameter set referenced when decoding the point cloud attribute slice in the point cloud attribute header.

18. The method for processing a point cloud file according to claim 17, characterized in that: The position order is the order of arrangement of sets in a plurality of parameter sets corresponding to the specified attribute type.

19. The method for processing a point cloud file according to claim 15, characterized in that: When the data unit in the point cloud code stream includes a point cloud attribute slice of a specified attribute type, the data box of the point cloud code stream further includes: The dynamic attribute flag field is used to indicate whether the position order of the parameter set referenced when decoding the point cloud attribute slice in the point cloud attribute header is a fixed value.

20. The method for processing a point cloud file according to claim 15, characterized in that: When the data unit in the point cloud codestream includes a point cloud attribute slice of a specified attribute type, the data box of the point cloud codestream further includes at least one of the following fields: The attribute quantity field is used to indicate the number of types of the point cloud attribute slices contained in the point cloud code stream; The attribute type field is used to indicate the attribute type of the point cloud attribute slice contained in the point cloud code stream; The attribute identification field is used to indicate the identifier of the point cloud attribute slice contained in the point cloud code stream.

21. The method for processing a point cloud file according to any one of claims 1 to 11, characterized in that: The point cloud codestream includes a parameter header sample group consisting of a plurality of parameter header samples, wherein the parameter header sample includes a point cloud geometry header or a point cloud attribute header; when a data unit in the point cloud codestream includes a point cloud attribute slice of a specified attribute type, a data box of the parameter header sample group includes: An attribute identification field, used to indicate an identifier of the point cloud attribute slice; The parameter set flag field is used to indicate the position order of the parameter set referenced when decoding the point cloud attribute slice in the point cloud attribute header.

22. The method for processing a point cloud file according to claim 21, characterized in that: The position order is the order of arrangement of sets in a plurality of parameter sets corresponding to the specified attribute type.

23. The method for processing a point cloud file according to claim 21, characterized in that: The data box of the parameter header sample group also includes: The parameter set information flag field is used to indicate whether the parameter set referenced when decoding the point cloud attribute slice is the target parameter set in the point cloud attribute header.

24. The method for processing a point cloud file according to claim 23, characterized in that: If the parameter set information flag field value of any point cloud sample in the point cloud code stream is a preset value, the parameter set information flag field value of the synchronization sample in the point cloud code stream is the preset value; The parameter set information flag field whose value is the preset value indicates that the parameter set referenced when decoding the point cloud sample is the target parameter set in the point cloud attribute header, and the synchronization sample is a sample that can be directly decoded when decoding the point cloud file.

25. The method for processing a point cloud file according to any one of claims 1 to 11, characterized in that: When the data unit in the point cloud code stream includes a point cloud attribute slice of a specified attribute type, the data structure of the point cloud attribute slice includes: The parameter set flag field is used to indicate the parameter set corresponding to the specified attribute type that is referenced when decoding the point cloud attribute slice.

26. A method for processing a point cloud file, characterized in that: include: Get point cloud media content; Encoding and packaging the point cloud media content to obtain a point cloud file carrying a point cloud code stream; The data box of the point cloud codestream includes a unit type field, and the unit type field is used to indicate the data type of the data unit in the point cloud codestream.

27. A point cloud file processing device, characterized in that: include: A first acquisition module is configured to acquire a point cloud file carrying a point cloud code stream, wherein a data box of the point cloud code stream includes a unit type field, and the unit type field is used to indicate a data type of a data unit in the point cloud code stream; The decoding module is configured to decapsulate and decode the point cloud file according to the unit type field to obtain point cloud media content corresponding to the data unit.

28. A point cloud file processing device, characterized in that: include: A second acquisition module is configured to acquire point cloud media content; The encoding module is configured to encode and encapsulate the point cloud media content to obtain a point cloud file carrying a point cloud code stream; the data box of the point cloud code stream includes a unit type field, and the unit type field is used to indicate the data type of the data unit in the point cloud code stream.

29. A computer readable medium, characterized in that The computer-readable medium stores a computer program, and when the computer program is executed by a processor, the method for processing a point cloud file according to any one of claims 1 to 26 is implemented.

30. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the executable instructions to implement the point cloud file processing method as described in any one of claims 1 to 26.