Data processing method and device for point cloud media, storage medium and electronic device

Abstract

The application discloses a data processing method and device of a point cloud medium, a storage medium and an electronic device. The method comprises the following steps: acquiring a point cloud code stream of a point cloud medium, wherein the point cloud code stream comprises a plurality of target attribute data and attribute dependent combination indication information, the attribute dependent combination indication information is used for indicating a coding and decoding dependent combination relationship between the plurality of target attribute data; and the point cloud code stream is decoded according to the attribute dependent combination indication information to present the point cloud medium, so that the combination and the coding and decoding dependent relationship between the attribute data of the point cloud can be flexibly organized, and the decoding accuracy of the point cloud code stream can be improved.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and in particular relates to a data processing method, apparatus, storage medium and electronic device for point cloud media. Background Technology

[0002] Point clouds are collections of points obtained by collecting data from the surface of an object using measuring instruments. Point clouds have wide applications in autonomous driving, high-precision maps, and virtual reality. However, since point clouds often consist of tens of thousands to hundreds of millions of three-dimensional points, they pose significant challenges to data storage, processing, transmission, and display. Current storage resources and transmission bandwidth are insufficient to handle these demands. Therefore, encoding and compressing point clouds to obtain point cloud bitstreams, and then using these bitstreams for storage or transmission, is essential.

[0003] Currently, common point cloud encoding and compression methods include geometry-based point cloud compression (GPCC). However, existing GPCC technology cannot flexibly organize the encoding and decoding dependencies and combinations between point cloud attribute data, which is not conducive to the correct decoding of point cloud bitstreams. Summary of the Invention

[0004] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a data processing method, apparatus, storage medium, and electronic device for point cloud media, which can flexibly organize the encoding and decoding dependencies and combination relationships between the attribute data of point clouds, thereby improving the decoding accuracy of point cloud bitstreams.

[0005] Firstly, this application provides a data processing method for point cloud media, including:

[0006] Obtain the point cloud bitstream of point cloud media, wherein the point cloud bitstream includes multiple target attribute data and attribute dependency combination indication information, and the attribute dependency combination indication information is used to indicate the encoding and decoding dependency combination relationship between the multiple target attribute data;

[0007] The point cloud stream is decoded to present the point cloud media based on the attribute dependency combination indication information.

[0008] Secondly, this application provides a data processing method for point cloud media, including:

[0009] Acquire point cloud media;

[0010] The point cloud media is encoded and compressed to obtain a point cloud bitstream. The point cloud bitstream includes multiple target attribute data and attribute dependency combination indication information. The attribute dependency combination indication information is used to indicate the encoding and decoding dependency combination relationship between the multiple target attribute data.

[0011] In some embodiments, the plurality of target attribute data includes a plurality of first target attribute data of the same attribute type, and the attribute dependency combination indication information is used to indicate the encoding / decoding dependency combination relationship among the plurality of first target attribute data.

[0012] In some embodiments, the plurality of target attribute data further includes a plurality of second target attribute data of the same attribute type, wherein the first target attribute data and the second target attribute data belong to different attribute types, and the attribute dependency combination indication information is further used to indicate the encoding / decoding dependency combination relationship between the first target attribute data and the second target attribute data.

[0013] In some embodiments, the attribute dependency combination indication information is represented as a plurality of target fields, the plurality of target fields including a cross-type attribute prediction field, a cross-type attribute prediction group identifier field, a cross-type attribute prediction order field, a same-type attribute prediction field, a same-type attribute prediction group identifier field, and a same-type attribute prediction order field;

[0014] Wherein, the cross-type attribute prediction field is used to indicate whether cross-type attribute prediction of the corresponding attribute data is allowed; the cross-type attribute prediction group identifier field is used to indicate the identifier of the cross-type attribute prediction group to which the corresponding attribute data belongs, the cross-type attribute prediction group being a group of attribute data that uses cross-type attribute prediction; the cross-type attribute prediction order field is used to indicate the prediction order of the corresponding attribute data in cross-type attribute prediction; the same-type attribute prediction field is used to indicate whether same-type attribute prediction of the corresponding attribute data is allowed; the same-type attribute prediction group identifier field is used to indicate the identifier of the same-type attribute prediction group to which the corresponding attribute data belongs, the same-type attribute prediction group being a group of attribute data that uses same-type attribute prediction; the same-type attribute prediction order field is used to indicate the prediction order of the corresponding attribute data in same-type attribute prediction.

[0015] In some embodiments, if the cross-type attribute prediction field is set to a first flag value, it indicates that cross-type attribute prediction of the corresponding attribute data is not allowed; if the cross-type attribute prediction field is set to a second flag value, it indicates that cross-type attribute prediction of the corresponding attribute data is allowed.

[0016] In some embodiments, if the value of the same type attribute prediction field is a first flag value, it indicates that same type attribute prediction of the corresponding attribute data is not allowed; if the value of the same type attribute prediction field is a second flag value, it indicates that same type attribute prediction of the corresponding attribute data is allowed.

[0017] In some embodiments, the point cloud stream includes a point cloud sequence, the point cloud sequence includes an attribute header and an attribute slice, the attribute header includes a set of parameters required for decoding the plurality of target attribute data, and the attribute slice includes the plurality of target attribute data; the cross-type attribute prediction field, the cross-type attribute prediction group identifier field, the same-type attribute prediction field, the same-type attribute prediction group identifier field, and the cross-type attribute prediction order field are located in the attribute header, and the same-type attribute prediction order field is located in the attribute slice.

[0018] In some embodiments, the point cloud sequence further includes a sequence header, which includes a set of parameters required for decoding the point cloud sequence; the plurality of target fields further include a first combination quantity identifier field and a second combination quantity identifier field, wherein the first combination quantity identifier field is used to indicate whether there are multiple sets of attribute data predicted using cross-type attribute in the point cloud sequence, and the second combination quantity identifier field is used to indicate whether there are multiple sets of attribute data predicted across reflectivity attribute in the point cloud sequence, wherein both the first combination quantity identifier field and the second combination quantity identifier field are located in the attribute header, or both are located in the sequence header.

[0019] In some embodiments, if the value of the first combination quantity identifier field is a second flag value, it indicates that there are multiple sets of attribute data predicted using cross-type attribute prediction; if the value of the first combination quantity identifier field is a first flag value, it indicates that there is at most one set of attribute data predicted using cross-type attribute prediction.

[0020] In some embodiments, if the value of the second combination quantity identifier field is a second flag value, it indicates that there are multiple sets of attribute data predicted using the transreflectivity attribute; if the value of the second combination quantity identifier field is a first flag value, it indicates that there is at most one set of attribute data predicted using the transreflectivity attribute.

[0021] In some embodiments, the plurality of target fields further include a first identifier indicator field and a second identifier indicator field located in the attribute header. The first identifier indicator field is used to indicate whether to set the identifier of the cross-type attribute prediction group to which the corresponding attribute data belongs; the second identifier indicator field is used to indicate whether to set the identifier of the same-type attribute prediction group to which the corresponding attribute data belongs.

[0022] In some embodiments, if the value of the first identifier indicating field is a first flag value, it indicates that the identifier of the cross-type attribute prediction group in which the corresponding attribute data is located is not set; if the value of the first identifier indicating field is a second flag value, it indicates that the identifier of the cross-type attribute prediction group in which the corresponding attribute data is located is set.

[0023] In some embodiments, if the value of the second identifier indicating field is a first flag value, it indicates that the identifier of the same type of attribute prediction group where the corresponding attribute data is located is not set; if the value of the second identifier indicating field is a second flag value, it indicates that the identifier of the same type of attribute prediction group where the corresponding attribute data is located is set.

[0024] In some embodiments, the plurality of target fields further include an attribute header information index field located in the attribute slice, the attribute header information index field being used to indicate the attribute header information index used by the attribute type of the attribute data in the corresponding attribute slice, the attribute header information index being used to obtain the corresponding parameter in the attribute header.

[0025] In some embodiments, the plurality of target fields further include a slice data type field and an attribute header information index field located in the attribute slice;

[0026] The slice data type field is used to indicate the attribute prediction type of the attribute data in the corresponding attribute slice, the attribute header information index field is used to indicate the attribute header information index used by the attribute type of the attribute data in the corresponding attribute slice, and the attribute header information index is used to obtain the corresponding parameters in the attribute header.

[0027] In some embodiments, if the piece data type field is a first flag value, the attribute prediction type indicates that the corresponding attribute piece includes color data that does not participate in cross-type attribute prediction; if the piece data type field is a second flag value, the attribute prediction type indicates that the corresponding attribute piece includes reflectance data that does not participate in cross-type attribute prediction or same-type attribute prediction; if the piece data type field is a third flag value, the attribute prediction type indicates that the corresponding attribute piece includes a set of color data and a set of reflectance data that have cross-type attribute prediction; if the piece data type field is a fourth flag value, the attribute prediction type indicates that the corresponding attribute piece includes at least one set of reflectance data that has same-type attribute prediction.

[0028] In some embodiments, if the piece data type field is a first flag value, the attribute prediction type indicates that the corresponding attribute piece includes a set of color data, and the set of color data is not allowed to be predicted across different attribute types; if the piece data type field is a second flag value, the attribute prediction type indicates that the corresponding attribute piece includes a set of reflectance data, and the set of reflectance data is not allowed to be predicted across different attribute types or within the same attribute type; if the piece data type field is a third flag value, the attribute prediction type indicates that the corresponding attribute piece includes a set of color data, and the set of color data is allowed to be predicted across different attribute types; if the piece data type field is a fourth flag value, it indicates that the corresponding attribute piece includes a set of reflectance data, and the set of reflectance data is allowed to be predicted across different attribute types or within the same attribute type.

[0029] In some embodiments, the point cloud stream includes a point cloud sequence, the point cloud sequence includes multiple point cloud data unit structures, the multiple target attribute data and the attribute dependency combination indication information are located in the multiple point cloud data unit structures, and each point cloud data unit structure includes a data unit type field, a data unit load length field and a data unit load field;

[0030] The data unit type field indicates the data type of the corresponding point cloud data unit structure, the data unit load length field indicates the load length of the corresponding point cloud data unit structure, and the data unit load field indicates the load of the corresponding point cloud data unit structure.

[0031] In some embodiments, when the data type is a sequence header type, the data unit payload field is used to indicate the bit stream corresponding to the sequence header; when the data type is an attribute header type, the data unit payload field is used to indicate the bit stream corresponding to the attribute header; when the data type is an attribute slice type, the data unit payload field is used to indicate the bit stream corresponding to the attribute slice; when the data type is an attribute slice block type, the data unit payload field is used to indicate the bit stream corresponding to the attribute slice block.

[0032] The attribute slice includes at least one attribute slice block.

[0033] In some embodiments, when the data type is the attribute piece block type, the corresponding point cloud data unit structure further includes a block combination order field, which is used to indicate the combination order of the corresponding attribute piece blocks when they are combined into the corresponding attribute piece.

[0034] In some embodiments, decoding the point cloud code stream according to the attribute dependency combination indication information includes:

[0035] Based on the attribute dependency combination indication information, at least one target same type attribute prediction group and a first prediction order of the target attribute data in the target same type attribute prediction group are determined from the plurality of target attribute data;

[0036] The target attribute prediction group is decoded according to the first prediction order.

[0037] In some embodiments, decoding the point cloud code stream according to the attribute dependency combination indication information further includes:

[0038] Based on the attribute dependency combination indication information, at least one target cross-type attribute prediction group and a second prediction order of the target attribute data in the target cross-type attribute prediction group are determined from the plurality of target attribute data;

[0039] The target cross-type attribute prediction group is decoded according to the second prediction order.

[0040] Thirdly, this application provides a data processing apparatus for point cloud media, comprising:

[0041] The first acquisition module is used to acquire the point cloud code stream of the point cloud media. The point cloud code stream includes multiple target attribute data and attribute dependency combination indication information. The attribute dependency combination indication information is used to indicate the encoding and decoding dependency combination relationship between the multiple target attribute data.

[0042] The decoding module is used to decode the point cloud bitstream according to the attribute dependency combination indication information to present the point cloud media.

[0043] Fourthly, this application provides a data processing apparatus for point cloud media, comprising:

[0044] The second acquisition module is used to acquire point cloud media;

[0045] The encoding and compression module is used to encode and compress the point cloud media to obtain a point cloud code stream. The point cloud code stream includes multiple target attribute data and attribute dependency combination indication information. The attribute dependency combination indication information is used to indicate the encoding and decoding dependency combination relationship between the multiple target attribute data.

[0046] Fifthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the data processing method for point cloud media described in any of the preceding claims.

[0047] In a sixth aspect, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the data processing method for point cloud media described in any of the preceding claims.

[0048] The point cloud media data processing method, apparatus, storage medium, and electronic device provided in this application acquire the point cloud code stream of the point cloud media, wherein the point cloud code stream includes multiple target attribute data and attribute dependency combination indication information, the attribute dependency combination indication information being used to indicate the encoding and decoding dependency combination relationship between the multiple target attribute data; and decodes the point cloud code stream according to the attribute dependency combination indication information to present the point cloud media, thereby flexibly organizing the combination and encoding and decoding dependency relationship between the attribute data of the point cloud, which is beneficial to improving the decoding accuracy of the point cloud code stream. Attached Figure Description

[0049] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0050] Figure 1a This is a schematic diagram illustrating the degrees of freedom of 3DoF provided in an embodiment of this application;

[0051] Figure 1b This is a schematic diagram illustrating a 3DoF+ degree of freedom provided in an embodiment of this application;

[0052] Figure 1c This is a schematic diagram illustrating a 6DoF degree of freedom provided in an embodiment of this application;

[0053] Figure 2 This is a schematic diagram of the data processing framework of the point cloud media data processing system provided in the embodiments of this application;

[0054] Figure 3 This is a flowchart illustrating the data processing method for point cloud media provided in an embodiment of this application;

[0055] Figure 4 This is a flowchart illustrating another cloud media data processing method provided in an embodiment of this application;

[0056] Figure 5 This is a schematic diagram of the structure of the point cloud media data processing device provided in the embodiments of this application;

[0057] Figure 6 This is a schematic diagram of the structure of another cloud media data processing device provided in the embodiments of this application;

[0058] Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0059] Figure 8 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0060] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0061] The following is a description of some of the technical terms used in this application:

[0062] I. Point Clouds

[0063] A point cloud is a set of randomly distributed discrete points in space that represent the spatial structure and surface properties of a three-dimensional object or scene. Each point in a point cloud includes at least geometric data, which represents the point's three-dimensional position information. Depending on the application scenario, points in a point cloud may also include one or more sets of attribute data. Each set of attribute data reflects a specific attribute possessed by the point, such as color, material, or other information. Typically, each point in a point cloud has the same number of sets of attribute data.

[0064] Point clouds can flexibly and conveniently represent the spatial structure and surface properties of three-dimensional objects or scenes, and therefore have a wide range of applications. They can be used in virtual reality (VR) games, computer-aided design (CAD), geographic information systems (GIS), autonomous navigation systems (ANS), digital cultural heritage, free-viewpoint broadcasting, 3D immersive remote presentation, and 3D reconstruction of biological tissues and organs.

[0065] Point clouds are primarily acquired through the following methods: computer generation, 3D laser scanning, and 3D photogrammetry. Specifically, point clouds can be obtained by acquiring visual scenes of the real world using acquisition devices (a set of cameras or a camera system with multiple lenses and sensors). 3D laser scanning can obtain point clouds of static 3D objects or scenes in the real world, acquiring millions of point clouds per second; 3D photogrammetry can obtain point clouds of dynamic 3D objects or scenes in the real world, acquiring tens of millions of point clouds per second. Furthermore, in the medical field, point clouds of biological tissues and organs can be obtained through magnetic resonance imaging (MRI), computed tomography (CT), and electromagnetic positioning information. Point clouds can also be directly generated by computers based on virtual 3D objects and scenes. With the continuous accumulation of large-scale point cloud data, the efficient storage, transmission, publication, sharing, and standardization of point cloud data have become crucial for point cloud applications.

[0066] II. Point Cloud Media

[0067] Point cloud media is a typical example of 6DoF immersive media. Immersive media refers to media files that provide immersive content, allowing viewers to experience visual, auditory, and other sensory sensations reminiscent of the real world. Based on the degree of freedom viewers have when consuming media content, immersive media can be categorized as: 6DoF (Degree of Freedom) immersive media, 3DoF immersive media, and 3DoF+ immersive media. Please refer to [further details omitted]. Figure 1a , Figure 1a This is a schematic diagram illustrating a 3DoF (3 Degrees of Freedom) demonstration provided in an embodiment of this application. 3DoF refers to three degrees of freedom, meaning that the viewer of immersive media is fixed at the center point in a three-dimensional space, and the viewer's head rotates along the X, Y, and Z axes to view the images provided by the media content. Please refer to [link / reference]. Figure 1b , Figure 1b This is a schematic diagram illustrating a 3DoF+ degree of freedom provided in an embodiment of this application. 3DoF+ refers to the fact that, in addition to the three degrees of freedom, the viewer's head in immersive media also possesses limited freedom of movement along the X, Y, and Z axes. Please refer to... Figure 1c , Figure 1c This is a schematic diagram illustrating a 6DoF degree of freedom provided in an embodiment of this application. 6DoF refers to the fact that, in addition to the three degrees of freedom, the viewer's head in immersive media also has the freedom to move freely along the X-axis, Y-axis, and Z-axis.

[0068] It should be noted that the data processing method for point cloud media provided in this application embodiment is applied to a data processing system for point cloud media. This data processing system may include content production equipment and content presentation equipment. Content production equipment can refer to the computer equipment used by the provider of point cloud media (e.g., the content creator of point cloud media). This computer equipment can be a terminal (e.g., a PC, a smart mobile device, such as a smartphone) or a server. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Content presentation equipment can refer to the computer equipment used by the user of point cloud media (e.g., the viewer of point cloud media, i.e., the business target). This computer equipment can be a terminal (e.g., a PC, a smart mobile device, such as a smartphone), VR device (e.g., a VR headset, VR glasses), smart home appliances, vehicle terminals, aircraft, etc.), and this computer equipment integrates a client. The client here can be a client capable of displaying data information such as text, images, audio, and video, including but not limited to multimedia clients (e.g., video clients), social clients (e.g., instant messaging clients), news applications (e.g., news clients), entertainment clients (e.g., game clients), shopping clients, in-vehicle clients, browsers, etc. This client can be a standalone client or an embedded sub-client integrated into another client (e.g., a social client); there is no limitation on this.

[0069] Content creation equipment and content presentation equipment can be the same device or different devices. Both include multiple modules, each performing different functions. These modules can be integrated into the same electronic device or located in different electronic devices. Content creation equipment can be used to acquire, encode, and encapsulate point cloud media; correspondingly, content presentation equipment can be used to decapsulate, decode, render, and display encapsulated files. Please see [link to relevant documentation]. Figure 2 , Figure 2 This is a schematic diagram of the data processing framework of the point cloud media data processing system provided in the embodiments of this application.

[0070] exist Figure 2In this process, on the content production device side, the real-world visual scene A is captured by a set of cameras or a camera device with multiple lenses and sensors. The capture result is point cloud media B, which is a frame sequence composed of a large number of point cloud frames. One or more point cloud frames are encoded into a point cloud bitstream E, including encoded geometric bitstreams and attribute bitstreams. Then, the point cloud bitstream E is encapsulated according to a specific media container file format (such as ISOBMFF) to obtain an initialization segment and a sequence of media segments (Fs) for streaming transmission, or a media file (F) for file playback. During the encapsulation process, metadata is also included in the file F or the media segment Fs. Afterward, the segment Fs can be transmitted to the content presentation device using a transmission mechanism. This transmission mechanism may include, but is not limited to, the DASH protocol, HLS protocol, SMTP protocol, TCP protocol, etc.

[0071] On the content presentation device side, the sequence (Fs) or media file (F) is decapsulated to obtain the point cloud bitstream E' and its metadata is parsed. Then, the point cloud bitstream E' is decoded to generate point cloud media D'. After that, the point cloud media D' is rendered, and the corresponding visual scene A' is presented. During the data processing on the content presentation device side, the decoded point cloud media D' can be rendered and displayed on the screen of a head-mounted display or any other display device based on the current viewer's viewing position, viewing direction, or the viewpoint determined by various types of sensors (such as head, position, or eye-tracking sensors). Partial access to the decoded point cloud media through the current viewing position or viewing direction can be used to optimize the point cloud media processing.

[0072] It is understood that the data processing technology (mainly including encoding and decoding technologies) of point cloud media involved in this application can be implemented based on cloud technology; for example, using a cloud server as a content production device. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or local area network to realize data computing, storage, processing, and sharing.

[0073] This application provides a data processing method, apparatus, storage medium, and electronic device for point cloud media.

[0074] Please see Figure 3 , Figure 3 This is a flowchart illustrating a point cloud media data processing method provided in an embodiment of this application. This point cloud media data processing method can be applied to content presentation devices and specifically includes the following steps 101 and 102, wherein:

[0075] 101. Obtain the point cloud bitstream of point cloud media, wherein the point cloud bitstream includes multiple target attribute data and attribute dependency combination indication information, wherein the attribute dependency combination indication information is used to indicate the encoding and decoding dependency combination relationship between the multiple target attribute data.

[0076] The point cloud bitstream is the bitstream structure of the encoded and compressed point cloud media. It can be extracted by the content presentation device after decapsulating the encapsulated file. The point cloud media includes attribute data and geometric data. The attribute data can include various types, such as color attributes and reflectivity attributes. The encoding and compression methods can include geometry-based point cloud compression (GPCC).

[0077] Encoding-decoding dependency composition relationships include both encoding-decoding dependencies and composition relationships. That is, they indicate both the combination of target attribute data and the encoding-decoding dependencies between target attribute data within the composition. The content of the attribute dependency composition indication information depends on the actual situation of the target attribute data.

[0078] Specifically, the multiple target attribute data may include multiple first target attribute data of the same attribute type. In this case, the attribute dependency combination indication information can be used only to indicate the encoding / decoding dependency combination relationship between the multiple first target attribute data. The multiple target attribute data may also include multiple second target attribute data of the same attribute type, where the first target attribute data and the second target attribute data belong to different attribute types. In this case, the attribute dependency combination indication information can also be used to indicate the encoding / decoding dependency combination relationship between the first target attribute data and the second target attribute data. That is, the target attribute data in the point cloud code stream may belong to only the same attribute type or to different attribute types. The attribute dependency combination indication information can indicate the encoding / decoding dependency combination relationship between attribute data of the same attribute type, or it can indicate the encoding / decoding dependency combination relationship between attribute data of different attribute types.

[0079] For example, target attribute data can include color data belonging to the color attribute and reflectance data belonging to the reflectance attribute. In this case, the attribute dependency combination indication information can not only indicate the encoding and decoding dependency combination relationship between reflectance data, but also further indicate the encoding and decoding dependency combination relationship between color data and reflectance data.

[0080] 102. Decode the point cloud stream according to the attribute dependency combination indication information to present the point cloud media.

[0081] Decoding the point cloud stream mainly involves two parts: decoding the geometric bitstream (i.e., target geometric data) of the point cloud media, and decoding the attribute bitstream (i.e., target attribute data) of the point cloud media. Attribute dependency combination indication information is mainly used to decode the attribute bitstream. The decoding process of the point cloud stream is the reverse of the encoding and compression process. By combining attribute dependency combination indication information with the decoding of the attribute bitstream, the decoding accuracy of the attribute bitstream in the point cloud stream can be improved. After decoding the point cloud stream, the content presentation device can further render and display the decoded data to present the point cloud media to the user (see above). Figure 2 ).

[0082] In some embodiments, when decoding the attribute bitstream of point cloud media, step 102 above may specifically include:

[0083] Based on the attribute dependency combination indication information, at least one target same type attribute prediction group and a first prediction order of the target attribute data in the target same type attribute prediction group are determined from the plurality of target attribute data;

[0084] The target attribute prediction group is decoded according to the first prediction order.

[0085] Among them, the target same type attribute prediction group includes multiple target attribute data of the same attribute type and with encoding and decoding dependencies. For example, if the target attribute data includes color data of color attribute and reflectance data of reflectance attribute, then the target same type attribute prediction group can include multiple reflectance data with encoding and decoding dependencies. In this case, the attribute dependency combination indication information can indicate which reflectance data belong to the same target same type attribute prediction group and indicate the first prediction order of each reflectance data in the corresponding target same type attribute prediction group.

[0086] Furthermore, step 102 above may also include:

[0087] Based on the attribute dependency combination indication information, at least one target cross-type attribute prediction group and a second prediction order of the target attribute data in the target cross-type attribute prediction group are determined from the plurality of target attribute data;

[0088] The target cross-type attribute prediction group is decoded according to the second prediction order.

[0089] The target cross-type attribute prediction group includes multiple target attribute data of different attribute types and with encoding / decoding dependencies, such as color data of color attribute and reflectance data of reflectance attribute. In this case, the attribute dependency combination indication information can also indicate which color data and which reflectance data belong to the same target cross-type attribute prediction group, and indicate the second prediction order of each reflectance data in the corresponding target cross-type attribute prediction group.

[0090] To support the data processing methods in the embodiments of this application, several descriptive fields have been added at the system layer, and corresponding high-level syntactic data structures have been defined, based on existing technologies. The data processing methods described above will be explained in detail below with reference to various syntactic tables.

[0091] In this embodiment, the point cloud bitstream may include a start code and a point cloud sequence. The start code is a specific set of bit strings, which can be used to extract the corresponding content of each part of the point cloud sequence. In a point cloud bitstream conforming to this embodiment, these bit strings should not appear under any circumstances other than the start code.

[0092] The start code consists of a start code prefix and a start code value. The start code prefix is ​​the bit string '0000 0000 0000 000000000001'. All start codes should be byte-aligned.

[0093] The start code value is an 8-bit integer used to represent the type of the start code. Please refer to Table 1 below. Table 1 is a table of start code types provided in the embodiments of this application.

[0094] Table 1

[0095] start code type Start code value (hexadecimal) Slice start code (slice_start_code) 00～7F reserve 80～8E Slice end code (slice_end_code) 8F reserve 90~AF Point cloud sequence start code (sequence_start_code) B0 Point cloud sequence end code (sequence_end_code) B1 Point cloud frame start code (frame_start_code) B2 reserve B3~B8 System start code B9~FF

[0096] In this embodiment, the point cloud sequence (PCC_sequence) is the highest-level syntax structure of the point cloud bitstream. The point cloud sequence includes sequence header information (referred to as sequence_header), point cloud frame geometry header information (referred to as geometry_header), point cloud frame attribute header information (referred to as attribute_header), and one or more point cloud frames. Each point cloud frame may contain point cloud frame header information (referred to as point cloud frame header) and one or more data units, where each data unit includes a point cloud slice. The sequence header includes the set of parameters required for decoding the point cloud sequence. The geometry header includes the set of parameters required for decoding the geometric data of the point cloud. The attribute header includes the set of parameters required for decoding the attribute data of the point cloud (such as the target attribute data in this point cloud bitstream).

[0097] The point cloud slices here can include geometric slice information (or simply geometry_slice) and attribute slice information (or simply attribute_slice). Each point cloud slice consists of a slice header and a data bitstream (geometry_data_bitstream).

[0098] A geometry slice consists of two parts: a geometry slice header and a geometry data bitstream. The geometry data bitstream contains the geometry data of the corresponding geometry slice. An attribute slice consists of two parts: an attribute slice header and an attribute data bitstream (general_attribute_data_bitstream). The attribute data bitstream contains the attribute data of the corresponding attribute slice, such as the target attribute data in this point cloud bitstream.

[0099] It should be noted that, for each content part of the above point cloud sequence, the embodiments of this application mainly add several descriptive fields to the sequence header, attribute header and attribute slice to support the real-time steps of the point cloud media data processing method of this application. The extended fields mainly involve two parts of indication content: indicating the encoding and decoding dependency relationship and combination relationship between attribute data, and indicating the prediction order of attribute data in the corresponding combination.

[0100] 1. Indicate the encoding / decoding dependencies and combination relationships between indicator attribute data, as well as the prediction order of reflectance data in cross-type attribute prediction.

[0101] In this embodiment of the application, the attribute dependency combination indication information is represented as multiple target fields, which may include a cross-type attribute prediction field, a cross-type attribute prediction group identifier field, a cross-type attribute prediction order field, a same-type attribute prediction field, a same-type attribute prediction group identifier field, and a same-type attribute prediction order field.

[0102] Wherein, the cross-type attribute prediction field is used to indicate whether cross-type attribute prediction of the corresponding attribute data is allowed; the cross-type attribute prediction group identifier field is used to indicate the identifier of the cross-type attribute prediction group to which the corresponding attribute data belongs, the cross-type attribute prediction group being a group of attribute data that uses cross-type attribute prediction; the cross-type attribute prediction order field is used to indicate the prediction order of the corresponding attribute data in cross-type attribute prediction; the same-type attribute prediction field is used to indicate whether same-type attribute prediction of the corresponding attribute data is allowed; the same-type attribute prediction group identifier field is used to indicate the identifier of the same-type attribute prediction group to which the corresponding attribute data belongs, the same-type attribute prediction group being a group of attribute data that uses same-type attribute prediction; the same-type attribute prediction order field is used to indicate the prediction order of the corresponding attribute data in same-type attribute prediction.

[0103] Specifically, both the cross-type attribute prediction field and the same-type attribute prediction field mentioned above are binary variables, meaning they each have only two possible value ranges. They can have the same value range, such as both taking values ​​from a first flag value and a second flag value, or they can have different value ranges; this is not limited here. In some embodiments, the first flag value can be 0, and the second flag value can be 1.

[0104] In some embodiments, the cross-type attribute prediction field, the cross-type attribute prediction group identifier field, the same-type attribute prediction field, the same-type attribute prediction group identifier field, and the cross-type attribute prediction order field are located in the attribute header, and the same-type attribute prediction order field is located in the attribute slice.

[0105] For example, please refer to Tables 2, 3 and 4 below. Table 2 is a syntax definition table of the attribute header provided in the embodiments of this application. Table 3 is another syntax definition table of the attribute header provided in the embodiments of this application. Table 4 is a syntax definition table of the general attribute data bitstream in the attribute slice provided in the embodiments of this application.

[0106] Table 2

[0107]

[0108]

[0109]

[0110] It should be noted that Tables 2, 3, and 4 use the following examples to illustrate attribute dependency combination indication information: the same-type attribute prediction field indicates whether same-type attribute prediction of the corresponding reflectance data is allowed; the same-type attribute prediction group identifier field indicates the identifier of the same-type attribute prediction group to which the corresponding reflectance data belongs; the same-type attribute prediction group is a group of reflectance data using cross-reflectance attribute prediction; and the same-type attribute prediction order field indicates the prediction order of the corresponding reflectance data in same-type attribute prediction. In these examples, the same-type attribute prediction field is also referred to as the cross-reflectance attribute prediction field, the same-type attribute prediction group identifier field is also referred to as the cross-reflectance attribute prediction group identifier field, and the same-type attribute prediction group is also referred to as the cross-reflectance data group. Furthermore, Tables 2, 3, and 4 only exemplify the syntax content of the indication information for adding encoding / decoding dependency combination relationships in the attribute header; they are not limited to a complete syntax definition table.

[0111] The attribute presence flag field (attributePresentFlag[attrIdx]) is a binary variable whose value range includes only two values, such as the first flag value and the second flag value mentioned above. For example, when the attribute presence flag field is the first flag value, it means that the current point cloud bitstream does not contain the attrIdx attribute encoding; when the attribute presence flag field is the second flag value, it means that the current point cloud bitstream contains the attrIdx attribute encoding.

[0112] attrIdx is an integer between 0 and 15, the meaning of which is explained in Table 3 below.

[0113] Table 3

[0114] Attribute index attr_idx Attribute type description 0 Color attribute 1 Reflectance property 2..15 reserve

[0115] The multi-attribute information enable flag (sps_multi_set_flag) is a binary variable that takes only two values, such as the first and second flag values ​​mentioned above. For example, if the value is the second flag value, it indicates that multiple sets of attribute information can be used for the same attribute data; if the value is the first flag value, it indicates that multiple sets of attribute information can be used for the same attribute data, and only one set of attribute information can be used for the same attribute data. When sps_multi_set_flag does not appear in the bitstream, its default value is zero.

[0116] The attribute information enable flag field (multi_set_flag[attrIdx]) is a binary variable that can only take two values, such as the first flag value and the second flag value mentioned above. When the attribute information enable flag is the second flag value, it indicates that the attribute type determined by the attribute index attrIdx can support the use of multiple sets of attribute information; when the attribute information enable flag is the first flag value, it indicates that the attribute type determined by the attribute index attrIdx can not support the use of multiple sets of attribute information, and the attribute data of this attribute type can only use one set of attribute information. When multi_set_flag[attrIdx] does not appear in the bitstream, its default value is zero.

[0117] The attribute information count minus one (attribute_info_num_set_minus1[attrIdx]) is an unsigned integer. Adding 1 indicates the number of attribute information items supported by the attribute type determined by the attribute index attrIdx in this point cloud bitstream. This is a number between 0 and 127. When attribute_info_num_set_minus1[attrIdx] does not appear in the bitstream, its default value is -1.

[0118] The attribute data count minus one (attribute_data_num_set_minus1[attrIdx]) is an unsigned integer. Adding 1 indicates the number of attribute data supported by the attribute type determined by the attribute index attrIdx in this point cloud bitstream. This is a number between 0 and 127. When attribute_data_num_set_minus1[attrIdx] does not appear in the bitstream, its default value is -1.

[0119] `maxNumAttributesMinus1` is an unsigned integer. Adding 1 indicates the maximum number of attribute types supported by this point cloud bitstream. The value of `maxNumAttributesMinus1` is an integer between 0 and 15. When `maxNumAttributesMinus1` does not appear in the bitstream, it defaults to -1.

[0120] `outputBitDepthMinus1` is an unsigned integer used to control the attribute output bit depth; this value is between 0 and 15. `outputBitDepth = outputBitDepthMinus1 + 1`. If this syntax element is not in the bitstream, the default value is zero.

[0121] The adaptive exponential Golomb coding sliding window size (log2_golomb_group_size) is an unsigned integer representing the logarithmic value of the adaptive exponential Golomb coding sliding window size for the attribute prediction residuals or transform coefficients. The adaptive exponential Golomb coding sliding window size is expressed as golomb_sliding_window_size = 2. log2 _golomb_group_size .

[0122] The attribute transformation algorithm flag field (transform) is an unsigned integer used to control the attribute encoding method. The attribute transformation algorithm flag field can represent three attribute encoding methods. For example, when the attribute transformation algorithm flag field takes the first flag value mentioned above, it indicates that a predictive method is used for attribute encoding; when the attribute transformation algorithm flag field takes the second flag value mentioned above, it indicates that a multi-level transformation is used for attribute encoding; and when the attribute transformation algorithm flag field takes the third flag value, it indicates that a predictive transformation method is used for attribute encoding. The first, second, and third flag values ​​can be 0, 1, and 2, respectively.

[0123] The attribute transformation precision value (kFracBits) is an unsigned integer used to represent the shift precision of fixed-point operations during attribute transformation.

[0124] The `transformSegmentSize` is an unsigned integer representing the number of points used for the attribute transformation, i.e., the wavelet transform using `transformSegmentSize` points. `transformSegmentSize = 0` means that all points in the slice are used for the wavelet transform.

[0125] Attribute transformation coefficient quantization parameter difference (attrTransformQpDelta): This attribute transformation coefficient quantization parameter difference is an unsigned integer used to represent the difference between the attribute transformation coefficient quantization parameter and the attribute residual quantization parameter. Attribute transformation coefficient quantization parameter attrTransformQp = attrQuantParam(attribute quantization parameter) + attrTransformQpDelta.

[0126] The attribute residual quadratic prediction field (cross_component_pred) is a binary variable used to indicate whether attribute residual quadratic prediction is allowed. When the attribute residual quadratic prediction field takes the first flag value (e.g., 1), it means that attribute residual quadratic prediction is allowed. When the attribute residual quadratic prediction field takes the second flag value (e.g., 0), it means that attribute residual quadratic prediction is not allowed.

[0127] Residual encoding order switch field (orderSwitch): This residual encoding order switch field is a binary variable; when the value of the residual encoding order switch field is the first flag value (such as 1), it indicates that the residual encoding order is UYV / GRB order; when the value of the residual encoding order switch field is the second flag value (such as 0), it indicates that the residual encoding order is RGB / YUV order.

[0128] Chroma channel Cb quantization parameter offset (chromaQpOffsetCb): This chroma channel Cb quantization parameter offset is a signed integer used to control the quantization parameter of the Cb channel, with a value range of 16 to 16. If chromaQpOffsetCb does not exist in the current attribute header information, then the value of chromaQpOffsetCb is 0. That is, chromaQpCb = Clip3(minQP, maxQP, attribute_qp + chromaQpOffsetCb). Note that the quantization parameter lumaQp for the luma channel is attribute_qp, with the minimum supported quantization parameter being minQP = 0 and the maximum supported quantization parameter being maxQP = 63.

[0129] Chroma channel quantization parameter offset (chromaQpOffsetCr): This chroma channel quantization parameter offset is a signed integer. It controls the quantization parameter of the Cr channel, and its value ranges from 16 to 16. If chromaQpOffsetCr does not exist in the current attribute header information, then the value of chromaQpOffsetCr is 0. That is, chromaQpCr = Clip3(minQP, maxQP, attribute_qp + chromaQpOffsetCr). Note that the quantization parameter lumaQp of the luma channel is attribute_qp, and the minimum supported quantization parameter is minQP = 0, and the maximum supported quantization parameter is maxQP = 63.

[0130] Nearest neighbor prediction parameter 1 (nearestPredParam1): This nearest neighbor prediction parameter 1 is an unsigned integer used to control the threshold for nearest neighbor prediction.

[0131] Nearest neighbor prediction parameter 2 (nearestPredParam2): This nearest neighbor prediction parameter 2 is an unsigned integer used to control the threshold for nearest neighbor prediction; the threshold is expressed as attrQuantParam*nearestPredParam1+nearestPredParam1.

[0132] Spatial bias coefficient (axisBias): This spatial bias coefficient is an unsigned integer used to control the offset in the Z direction during the calculation of attribute prediction values.

[0133] Maximum number of neighbor points selected for prediction (maxNumOfPredictNeighbours): This maximum number of neighbor points selected for prediction is an unsigned integer used to limit the number of neighbor points selected during attribute prediction. In bitstreams conforming to this section, maxNumOfPredictNeighbours should not exceed 16.

[0134] The parity-based attribute information flag (parityBasedLevelCodingEnable) is a binary variable used to control whether parity-based attribute residual or transformation coefficient encoding is enabled. For example, a value of the second flag indicates that it is enabled, and a value of the first flag indicates that it is disabled.

[0135] Color Reorder Mode: The color reorder mode is an unsigned integer used to represent the reordering mode of the current color information selection. When the color reorder mode field is the first flag value (e.g., 0), it indicates the original point cloud input order; when the color reorder mode field is the second flag value (e.g., 1), it indicates Hilbert reordering; when the color reorder mode field is the third flag value (e.g., 2), it indicates Morton reordering.

[0136] Reflectance Reordering Mode (colorReorderMode): The reflectance reordering mode is an unsigned integer. When the reflectance reordering mode is the first flag value (e.g., 0), it indicates the original point cloud input order; when the reflectance reordering mode field is the second flag value (e.g., 1), it indicates Hilbert reordering; when the reflectance reordering mode field is the third flag value (e.g., 2), it indicates Morton reordering.

[0137] Maximum cache limit parameter (maxNumofCoeff): This maximum cache limit parameter is an unsigned integer used to calculate the maximum number of transformation parameters that can be cached in attribute transformation coding.

[0138] Maximum Delay Limit Parameter (coeffLengthControl): This maximum delay limit parameter is an unsigned integer used to limit the maximum number of delays for transformation parameters in attribute transformation coding. Specifically, the maximum number of delay points is calculated as maxNumofCoeff * coeffLengthControl.

[0139] Reflectance group prediction flag field (refGroupPred): This reflectance group prediction flag field is a binary variable used to control whether the reflectance group prediction mode of the prediction transformation is enabled; when the reflectance group prediction flag field is the first flag value (e.g., 1), it indicates that group prediction is enabled; when the reflectance group prediction flag field is the first flag value (e.g., 0), it indicates that group prediction is disabled.

[0140] Initial Predictive Transform Ratio (initPredTransRatio): This initial predictive transform ratio is a signed integer used in the multi-level transform algorithm for attribute compression (transform=1) to control the size of the initial distance threshold used when constructing the predictive transform tree.

[0141] Transform Residual Layer Flag Field (transResLayer): This transform residual layer flag field is a binary variable used in the multi-layer transform algorithm for attribute compression (transform=1) to control whether attribute residual compensation is used. When transResLayer is the first flag value (e.g., 1), it indicates that attribute residual compensation is used; when transResLayer is the second flag value (e.g., 0), it indicates that attribute residual compensation is not used.

[0142] Color Golomb Order (ColorGolombNum): This color Golomb order is an unsigned integer used to represent the order K of the K-order exponent Golomb used when decoding the current color prediction residual or transform coefficients, where K = ColorGolombNum.

[0143] Reflectance index Golomb order (RefGolombNum): This reflectance index Golomb order is an unsigned integer used to represent the order K of the K-order exponential Golomb used when decoding the current reflectance prediction residual or transform coefficients, where K = ColorGolombNum.

[0144] The initial color prediction transformation ratio (colorInitPredTransRatio) is a signed integer used in the multi-level transformation algorithm for attribute compression (transform=1) to control the initial distance threshold used for color when constructing the prediction transformation tree.

[0145] The initial reflectance prediction transformation ratio (refInitPredTransRatio) is a signed integer. In multi-layer transformation algorithms used for attribute compression (transform=1), it controls the initial distance threshold used for reflectance when constructing the prediction transformation tree.

[0146] The point cloud adaptive quantization tool flag (colorQPAdjustFlag) is a binary variable. For example, a value of the first flag mentioned above indicates that the point cloud adaptive quantization tool is turned off; a value of the second flag mentioned above indicates that the point cloud adaptive quantization tool is turned on.

[0147] The ratio of the number of points before and after point cloud geometric quantization (colorQPAdjustScalar) is an unsigned integer used to calculate the distance threshold of the point cloud adaptive quantization tool.

[0148] The reflectance property prediction precision value (predFixedPointFracBit) is an unsigned integer used to represent the shift precision of the fixed-point transformation operation during reflectance property prediction. predFixedPointFracBit = 0 indicates that fixed-point transformation is not used.

[0149] The attribute value change statistics window size (log2_pred_dist_weight_group_size) is an unsigned integer used to control the size of the attribute value change statistics window. The window size is calculated as N-2. log2 _pred_dist_weight_group_size .

[0150] The cross-type attribute prediction weight parameter 1 (crossAttrTypePredParam1) is a 15-bit unsigned integer used to control the weight parameter 1 used to calculate the geometric information distance and attribute information distance in cross-type attribute prediction.

[0151] The cross-type attribute prediction weight parameter 2 (crossAttrTypePredParam2) is a 21-bit unsigned integer used to control the weight parameter 2 used to calculate the geometric information distance and attribute information distance in cross-type attribute prediction.

[0152] The number of cross-reflectivity attribute predictions (crossRefNum) indicates the number of reflectivity data points in the cross-reflectivity attribute prediction reflectivity data group containing the corresponding reflectivity data.

[0153] It should be noted that `crossAttrTypePred[attrIdx][i]` is the aforementioned cross-type attribute prediction field, used to indicate whether cross-type attribute prediction of the corresponding attribute data is allowed. `crossAttrTypePred[attrIdx][i]` is a binary variable, and its value range includes only two values, such as the first flag value and the second flag value mentioned above. In this case, if the value of the cross-type attribute prediction field is the first flag value, it indicates that cross-type attribute prediction of the corresponding attribute data is not allowed; if the value of the cross-type attribute prediction field is the second flag value, it indicates that cross-type attribute prediction of the corresponding attribute data is allowed.

[0154] crossAttrTypeUnionID[attrIdx][i] is the cross-type attribute prediction group identifier field mentioned above, which is used to indicate the identifier of the cross-type attribute prediction group to which the corresponding attribute data belongs.

[0155] `attrEncodeOrder[i]` is the cross-type attribute prediction order field mentioned above, used to indicate the prediction order of the corresponding attribute data in cross-type attribute prediction. It is a binary variable. For example, if the value of the cross-type attribute prediction order field is the first flag value mentioned above, it indicates that color data is predicted first, and then reflectance data is predicted in cross-type attribute prediction; if the value of the cross-type attribute prediction order field is the second flag value mentioned above, it indicates that reflectance data is predicted first, and then color data is predicted in cross-type attribute prediction.

[0156] `cross_ref_pred` is the aforementioned cross-reflectivity attribute prediction field, used to indicate whether cross-reflectivity attribute prediction of the corresponding attribute data is allowed. It is a binary variable. For example, if the value of the cross-reflectivity attribute prediction field is the aforementioned first flag value, it indicates that cross-reflectivity attribute prediction of the corresponding attribute data is not allowed; if the value of the cross-reflectivity attribute prediction field is the aforementioned second flag value, it indicates that cross-reflectivity attribute prediction of the corresponding attribute data is allowed.

[0157] crossRefUnionID is the identifier field for the cross-reflectivity attribute prediction group mentioned above, used to indicate the identifier of the reflectivity data group that uses cross-reflectivity attribute prediction.

[0158] In some embodiments, only the fields crossAttrTypePred, crossAttrTypeUnionID, attrEncodeOrder, cross_ref_pred, and crossRefUnionID can be extended in the attribute header. Alternatively, more fields can be extended in the attribute header. For example, the plurality of target fields may also include a first combination quantity identifier field and a second combination quantity identifier field. The first combination quantity identifier field is used to indicate whether there are multiple sets of attribute data predicted using cross-type attribute in the point cloud sequence. The second combination quantity identifier field is used to indicate whether there are multiple sets of attribute data predicted using cross-reflectivity attribute in the point cloud sequence. Both the first combination quantity identifier field and the second combination quantity identifier field are located in the attribute header.

[0159] Please refer to Table 2 above. `multCrossAttrUnion` is the first combination quantity identifier field, used to indicate whether there are multiple sets of attribute data predicted using cross-type attribute methods in this point cloud sequence, or at most one set (i.e., 0 or 1) of attribute data predicted using cross-type attribute methods. It is a binary variable. For example, if the first combination quantity identifier field takes the second flag value, then there are multiple sets of attribute data predicted using cross-type attribute methods; if the first combination quantity identifier field takes the first flag value, then it indicates that there is at most one set of attribute data predicted using cross-type attribute methods.

[0160] `multCrossRefUnion` is the second combination quantity identifier field mentioned above, used to indicate whether there are multiple sets of attribute data predicted using the cross-reflectivity attribute in the point cloud sequence (i.e., multiple sets of cross-reflectivity data groups), or at most one set (i.e., 0 or 1) of attribute data predicted using the cross-reflectivity attribute. It is a binary variable; for example, if the value of the second combination quantity identifier field is the second flag value, it indicates that there are multiple sets of cross-reflectivity data; if the value of the second combination quantity identifier field is the first flag value, it indicates that there are 1 or 0 sets of cross-reflectivity data.

[0161] In other embodiments, the first and second combination quantity identifier fields may not be indicated in the attribute header, but in the sequence header. That is, the two fields, multCrossRefUnion and multCrossAttrUnion, are not set in the attribute header, but in the sequence header.

[0162] In some embodiments, instead of extending the first and second combined identifier fields in the attribute header, other fields can be defined to express a similar meaning. For example, the plurality of target fields may also include a first identifier indication field and a second identifier indication field located in the attribute header. The first identifier indication field is used to indicate whether to set the identifier of the cross-type attribute prediction group to which the corresponding attribute data belongs; the second identifier indication field is used to indicate whether to set the identifier of the same-type attribute prediction group to which the corresponding attribute data belongs. For details, please refer to Table 3 below.

[0163] Table 3

[0164]

[0165] The explanations of the other fields in Table 3 can be found in Table 2 above, and will not be repeated here. In Table 3, crossAttrUnionIdFlag[attrIdx][i] is the first identifier indicator field mentioned above. It is a binary variable. For example, if the value of the first identifier indicator field is the first flag value, it indicates that the identifier of the cross-type attribute prediction group where the corresponding attribute data belongs is not set; if the value of the first identifier indicator field is the second flag value, it indicates that the identifier of the cross-type attribute prediction group where the corresponding attribute data belongs is set.

[0166] In Table 3, crossRefUnionIdFlag[i] is the second identifier indicator field mentioned above. It is a binary variable. For example, if the value of the second identifier indicator field is the first flag value, it indicates that the identifier of the same type of attribute prediction group where the corresponding attribute data is located is not set; if the value of the second identifier indicator field is the second flag value, it indicates that the identifier of the same type of attribute prediction group where the corresponding attribute data is located is set.

[0167] 2. Indicator reflectance data in the prediction order across reflectance attributes

[0168] In this embodiment, attribute pieces can be deployed in three ways, and the prediction order of reflectance data in cross-reflectance attribute prediction can be indicated in the attribute pieces (as previously described, the prediction order of reflectance data in cross-type attribute prediction can be indicated in the attribute header).

[0169] For example, in some embodiments, all attribute data can be deployed in an attribute slice. In this case, the aforementioned same-type attribute prediction order field (i.e., the cross-reflectivity attribute prediction order field) is located in the attribute header of the attribute slice. The plurality of target fields may also include an attribute header information index field located in the attribute slice. The attribute header information index field is used to indicate the attribute header information index used by the attribute type of the attribute data in the corresponding attribute slice. The attribute header information index is used to obtain the corresponding parameters in the attribute header. The attribute header information index field maintains the general attribute data bitstream in the attribute slice. For details, please refer to Table 4.

[0170] Table 4

[0171]

[0172] Wherein, attribute_data_reflectance is the reflectance data. cross_ref_index is the cross-reflectance attribute prediction order field mentioned above. attributeID is the attribute header information index field mentioned above, used to indicate the attribute header information index used by the color attribute or reflectance attribute. attributeID is a number between 0 and attribute_info_num_set_minus1[attrIdx]. If attrIdx=0, attribute_info_num_set_minus1[0] is the number of attribute information supported by the color attribute in the attribute header minus one. If attrIdx=1, attribute_info_num_set_minus1[1] is the number of attribute information supported by the reflectance attribute in the attribute header minus one. Other fields in Table 4 can be found in the other tables mentioned above, and will not be repeated here.

[0173] Accordingly, when cross-reflectivity attribute prediction exists, the decoding order of the reconstructed reflectivity data in each cross-reflectivity attribute prediction attribute data group can be determined based on the value of the cross-reflectivity attribute prediction order field. In other embodiments, considering that in the general attribute data bitstream, when cross-reflectivity attribute prediction exists, the order in which different reflectivity data appears in the bitstream is consistent with its calculation order during reconstruction, the decoding order can also be set using the order in which the reflectivity data appears in the general attribute data bitstream.

[0174] In some embodiments, unlike the method described above of deploying all attribute data in one attribute slice, all attribute data can also be deployed in multiple attribute slices. For example, independently decodable attribute data can be deployed in separate attribute slices, while attribute data with encoding / decoding dependencies can be deployed in the same attribute slice. In this case, the multiple target fields also include slice data type fields and attribute header information index fields located in the attribute slices.

[0175] The slice data type field is used to indicate the attribute prediction type of the attribute data in the corresponding attribute slice, the attribute header information index field is used to indicate the attribute header information index used by the attribute type of the attribute data in the corresponding attribute slice, and the attribute header information index is used to obtain the corresponding parameters in the attribute header.

[0176] For example, please refer to Tables 5 and 6 below. Table 5 is the syntax definition table of the attribute slice header in the attribute slice provided in the embodiments of this application, and Table 6 is the syntax definition table of the general attribute data bitstream in the attribute slice provided in the embodiments of this application. It is easy to understand that Tables 5 and 6 are not limited to complete syntax definition tables.

[0177] Table 5

[0178]

[0179] Here, `slice id` is the slice identifier, used to indicate the corresponding attribute slice. `crossRefNum` indicates the number of reflectance data points within the attribute data group containing the corresponding reflectance data in the cross-reflectance attribute prediction. `reflQPoffset` is the offset of the reflectance attribute quantization parameter, used to control the reflectance attribute quantization parameter, with a value range of -32 to 32. `sliceDataType` is the slice data type field mentioned above, which can indicate four types. Its value range can include the first, second, third, and fourth flag values. For example, if the slice data type field is the first flag value, the attribute prediction type indicates that the corresponding attribute slice includes color data that does not participate in cross-type attribute prediction; if the slice data type field is the second flag value, the attribute prediction type indicates that the corresponding attribute slice includes reflectance data that does not participate in cross-type or same-type attribute prediction; if the slice data type field is the third flag value, the attribute prediction type indicates that the corresponding attribute slice includes a set of color data and a set of reflectance data that have cross-type attribute prediction; if the slice data type field is the fourth flag value, the attribute prediction type indicates that the corresponding attribute slice includes at least one set of reflectance data that has same-type attribute prediction. In some embodiments, the third flag value can be 2, and the fourth flag value can be 3.

[0180] attributeID, attributeIDColor, and attributeIDRef are all index fields of the attribute header information mentioned above. attributeID is the attribute header information index used by the color attribute or reflectance attribute. It is a number between 0 and attribute_info_num_set_minus1[sliceDataType], that is, a number between 0 and attribute_info_num_set_minus1[0], or a number between 0 and attribute_info_num_set_minus1[1]. Among them, attribute_info_num_set_minus1[0] is the number of attribute information supported by the color attribute in the attribute header minus one, and attribute_info_num_set_minus1[1] is the number of attribute information supported by the reflectance attribute in the attribute header minus one.

[0181] `attributeIDColor` is the attribute header information index used by the color attribute, which is a number between 0 and `attribute_info_num_set_minus1`[0]. `attributeIDRef` is the attribute header information index used by the reflectance attribute, which is a number between 0 and `attribute_info_num_set_minus1`[1].

[0182] Table 6

[0183]

[0184] Among them, attribute_data_reflectance is the reflectance data, and attribute_data_color is the color data. Explanations of each field in Table 6 can be found in Table 5 and the other tables mentioned above, and will not be repeated here.

[0185] For example, in some embodiments, when all attribute data is deployed in multiple attribute slices, in addition to the method described above, a set of attribute data with the same attribute type and the same encoding / decoding dependency can be deployed in each attribute slice. In this case, the multiple target fields still include the slice data type field and the attribute header information index field, but the indication content of the slice data type field is different.

[0186] Please refer to Tables 7 and 8. Table 7 is the syntax definition table of the attribute slice header in the attribute slice provided in the embodiments of this application, and Table 8 is the syntax definition table of the general attribute data bitstream in the attribute slice provided in the embodiments of this application. It should be noted that... Figure 7 and Figure 8 It is not limited to a complete grammar definition table.

[0187] Table 7

[0188]

[0189] Here, `sliceDataType` is the slice data type field mentioned above, which can indicate four types. For example, if the slice data type field is a first flag value, the attribute prediction type indicates that the corresponding attribute slice includes a set of color data, and the set of color data is not allowed to be predicted across different types of attributes; if the slice data type field is a second flag value, the attribute prediction type indicates that the corresponding attribute slice includes a set of reflectance data, and the set of reflectance data is not allowed to be predicted across different types of attributes or within the same type of attributes; if the slice data type field is a third flag value, the attribute prediction type indicates that the corresponding attribute slice includes a set of color data, and the set of color data is allowed to be predicted across different types of attributes; if the slice data type field is a fourth flag value, it indicates that the corresponding attribute slice includes a set of reflectance data, and the set of reflectance data is allowed to be predicted across different types of attributes or within the same type of attributes.

[0190] `attributeID` is the index field for the attribute header information mentioned above. It is used to index the attribute information supported by the color attribute or the reflectivity attribute in the attribute header. This is a number between 0 and `attribute_info_num_set_minus1[attrIdx]`. For example, when `sliceDataType` is 0 or 2, `attrIdx` is 0; when `sliceDataType` is 1 or 3, `attrIdx` is 1.

[0191] Table 8

[0192]

[0193] The explanations of each field in Table 8 can be found in Table 7 and the other tables mentioned above, and will not be repeated here.

[0194] In some embodiments, the sliceDataType value can be simplified by using the crossAttrTypePred and cross_ref_pred fields in the attribute header to determine whether it participates in cross-type attribute prediction or cross-reflectivity attribute prediction. In this case, sliceDataType can be a binary variable that only indicates two types. For example, if the slice data type field is a first flag value, it indicates that the corresponding attribute slice includes a set of color data; if the slice data type field is a second flag value, it indicates that the corresponding attribute slice includes a set of reflectivity data.

[0195] Furthermore, for the point cloud code stream in the embodiments of this application, the geometric patch in the point cloud sequence may include at least one, and the attribute patch may also include at least one (the specific number depends on the deployment method of the attribute data described above). In this technology, the geometric patch and the attribute patch can be further divided into multiple blocks, that is, the geometric patch is divided into at least one geometric block, and the attribute patch is divided into at least one attribute block.

[0196] Specifically, this can be achieved by improving the syntax definition of the bitstream structure of the point cloud code stream. Specifically, a unified bitstream syntax structure can be defined in the syntax table of the point cloud sequence. For example, the point cloud sequence can be defined as including multiple point cloud data unit structures, with multiple target attribute data and attribute dependency combination indication information located within these multiple point cloud data unit structures. The point cloud data unit structure is a unified type-length-value data structure. Simultaneously, by setting the data type of the point cloud data unit structure, a unified syntax expression can be achieved for different data types, such as sequence headers, geometric headers, attribute headers, point cloud frame headers, geometric slices, attribute slices, geometric separations, and attribute blocks.

[0197] For example, please refer to Table 9 below. Table 9 is a syntax definition table of the point cloud data unit structure provided in the embodiments of this application. It is not limited to a complete syntax definition table.

[0198] Table 9

[0199]

[0200]

[0201] Each point cloud data unit structure (tlv_encapsulation) includes a data unit type field (tlv_type), a data unit payload length field (tlv_num_payload_bytes), and a data unit payload field (tlv_payload_byte). The data unit type field indicates the data type of the corresponding point cloud data unit structure. The data unit payload length field indicates the payload length of the corresponding point cloud data unit structure; different types of point cloud data unit structures typically have different payloads (the corresponding bitstream in the point cloud sequence).

[0202] For example, when the data type is a sequence header type, the data unit payload field is used to indicate the bit stream corresponding to the sequence header; when the data type is an attribute header type, the data unit payload field is used to indicate the bit stream corresponding to the attribute header; when the data type is an attribute slice type, the data unit payload field is used to indicate the bit stream corresponding to the attribute slice; when the data type is an attribute slice block type, the data unit payload field is used to indicate the bit stream corresponding to the attribute slice block; when the data type is a geometry header type, the data unit payload field is used to indicate the bit stream corresponding to the geometry header; when the data type is a geometry slice type, the data unit payload field is used to indicate the bit stream corresponding to the geometry slice; when the data type is a point cloud frame header type, the data unit payload field is used to indicate the bit stream corresponding to the point cloud frame header; when the data type is a geometry slice block type, the data unit payload field is used to indicate the bit stream corresponding to the geometry slice block.

[0203] For example, the value range of tlv_type can be 0 to 7. For instance, when tlv_type is 0, it indicates that the corresponding point cloud data unit structure is a sequence header; when tlv_type is 1, it indicates that the corresponding point cloud data unit structure is a geometry header; when tlv_type is 2, it indicates that the corresponding point cloud data unit structure is an attribute header; when tlv_type is 3, it indicates that the corresponding point cloud data unit structure is a point cloud frame header; when tlv_type is 4, it indicates that the corresponding point cloud data unit structure is a geometry patch; when tlv_type is 5, it indicates that the corresponding point cloud data unit structure is an attribute patch; when tlv_type is 6, it indicates that the corresponding point cloud data unit structure is a geometry patch block; and when tlv_type is 7, it indicates that the corresponding point cloud data unit structure is an attribute patch block.

[0204] Furthermore, when the data type is the attribute slice block type, the corresponding point cloud data unit structure also includes a block combination order field (block_index) and the slice identifier (such as geometry_slice_id or attribute_slice_id) to which it belongs. The block combination order field is used to indicate the combination order of the corresponding attribute slice blocks when they are combined into the corresponding attribute slice.

[0205] In this embodiment, any geometric piece block may contain part or complete geometric data of the corresponding geometric piece, and a complete segment of geometric piece bitstream data structure corresponding to a geometric piece may be arbitrarily divided into one or more bitstream data structures. Similarly, any attribute piece block may contain part or complete geometric data of the corresponding attribute piece, and a complete segment of attribute piece bitstream data structure corresponding to an attribute piece may be arbitrarily divided into one or more bitstream data structures.

[0206] In some embodiments, the corresponding complete point cloud header (such as an attribute header or a geometry header) may be indicated in each block (geometric patch block or attribute patch block), or the complete point cloud header may be indicated only in the first block.

[0207] As described above, the data processing method for point cloud media provided in this application obtains the point cloud code stream of the point cloud media. The point cloud code stream includes multiple target attribute data and attribute dependency combination indication information. The attribute dependency combination indication information is used to indicate the encoding and decoding dependency combination relationship between the multiple target attribute data. Based on the attribute dependency combination indication information, the point cloud code stream is decoded to present the point cloud media. This allows for flexible organization of the combination and encoding and decoding dependency relationships between the attribute data of the point cloud, which is beneficial to improving the decoding accuracy of the point cloud code stream.

[0208] Based on the methods described in the above embodiments, this application also provides another data processing method for point cloud media. Please refer to [link to relevant documentation]. Figure 4 , Figure 4 This is a flowchart illustrating another point cloud media data processing method provided in this application embodiment. This point cloud media data processing method can be applied to content production equipment and specifically includes the following steps 201 and 202.

[0209] 201. Obtain point cloud media;

[0210] 202. Encode and compress the point cloud media to obtain a point cloud bitstream, the point cloud bitstream including multiple target attribute data and attribute dependency combination indication information, the attribute dependency combination indication information being used to indicate the encoding and decoding dependency combination relationship between the multiple target attribute data.

[0211] In some embodiments, the plurality of target attribute data includes a plurality of first target attribute data of the same attribute type, and the attribute dependency combination indication information is used to indicate the encoding / decoding dependency combination relationship among the plurality of first target attribute data.

[0212] In some embodiments, the plurality of target attribute data further includes a plurality of second target attribute data of the same attribute type, wherein the first target attribute data and the second target attribute data belong to different attribute types, and the attribute dependency combination indication information is further used to indicate the encoding / decoding dependency combination relationship between the first target attribute data and the second target attribute data.

[0213] In some embodiments, the attribute dependency combination indication information is represented as a plurality of target fields, the plurality of target fields including a cross-type attribute prediction field, a cross-type attribute prediction group identifier field, a cross-type attribute prediction order field, a same-type attribute prediction field, a same-type attribute prediction group identifier field, and a same-type attribute prediction order field;

[0214] Wherein, the cross-type attribute prediction field is used to indicate whether cross-type attribute prediction of the corresponding attribute data is allowed; the cross-type attribute prediction group identifier field is used to indicate the identifier of the cross-type attribute prediction group to which the corresponding attribute data belongs, the cross-type attribute prediction group being a group of attribute data that uses cross-type attribute prediction; the cross-type attribute prediction order field is used to indicate the prediction order of the corresponding attribute data in cross-type attribute prediction; the same-type attribute prediction field is used to indicate whether same-type attribute prediction of the corresponding attribute data is allowed; the same-type attribute prediction group identifier field is used to indicate the identifier of the same-type attribute prediction group to which the corresponding attribute data belongs, the same-type attribute prediction group being a group of attribute data that uses same-type attribute prediction; the same-type attribute prediction order field is used to indicate the prediction order of the corresponding attribute data in same-type attribute prediction.

[0215] In some embodiments, if the cross-type attribute prediction field is set to a first flag value, it indicates that cross-type attribute prediction of the corresponding attribute data is not allowed; if the cross-type attribute prediction field is set to a second flag value, it indicates that cross-type attribute prediction of the corresponding attribute data is allowed.

[0216] In some embodiments, if the value of the same type attribute prediction field is a first flag value, it indicates that same type attribute prediction of the corresponding attribute data is not allowed; if the value of the same type attribute prediction field is a second flag value, it indicates that same type attribute prediction of the corresponding attribute data is allowed.

[0217] In some embodiments, the point cloud stream includes a point cloud sequence, the point cloud sequence includes an attribute header and an attribute slice, the attribute header includes a set of parameters required for decoding the plurality of target attribute data, and the attribute slice includes the plurality of target attribute data; the cross-type attribute prediction field, the cross-type attribute prediction group identifier field, the same-type attribute prediction field, the same-type attribute prediction group identifier field, and the cross-type attribute prediction order field are located in the attribute header, and the same-type attribute prediction order field is located in the attribute slice.

[0218] In some embodiments, the point cloud sequence further includes a sequence header, which includes a set of parameters required for decoding the point cloud sequence; the plurality of target fields further include a first combination quantity identifier field and a second combination quantity identifier field, wherein the first combination quantity identifier field is used to indicate whether there are multiple sets of attribute data predicted using cross-type attribute in the point cloud sequence, and the second combination quantity identifier field is used to indicate whether there are multiple sets of attribute data predicted across reflectivity attribute in the point cloud sequence, wherein both the first combination quantity identifier field and the second combination quantity identifier field are located in the attribute header, or both are located in the sequence header.

[0219] In some embodiments, if the value of the first combination quantity identifier field is a second flag value, it indicates that there are multiple sets of attribute data predicted using cross-type attribute prediction; if the value of the first combination quantity identifier field is a first flag value, it indicates that there is at most one set of attribute data predicted using cross-type attribute prediction.

[0220] In some embodiments, if the value of the second combination quantity identifier field is a second flag value, it indicates that there are multiple sets of attribute data predicted using the transreflectivity attribute; if the value of the second combination quantity identifier field is a first flag value, it indicates that there is at most one set of attribute data predicted using the transreflectivity attribute.

[0221] In some embodiments, the plurality of target fields further include a first identifier indicator field and a second identifier indicator field located in the attribute header. The first identifier indicator field is used to indicate whether to set the identifier of the cross-type attribute prediction group to which the corresponding attribute data belongs; the second identifier indicator field is used to indicate whether to set the identifier of the same-type attribute prediction group to which the corresponding attribute data belongs.

[0222] In some embodiments, if the value of the first identifier indicating field is a first flag value, it indicates that the identifier of the cross-type attribute prediction group in which the corresponding attribute data is located is not set; if the value of the first identifier indicating field is a second flag value, it indicates that the identifier of the cross-type attribute prediction group in which the corresponding attribute data is located is set.

[0223] In some embodiments, if the value of the second identifier indicating field is a first flag value, it indicates that the identifier of the same type of attribute prediction group where the corresponding attribute data is located is not set; if the value of the second identifier indicating field is a second flag value, it indicates that the identifier of the same type of attribute prediction group where the corresponding attribute data is located is set.

[0224] In some embodiments, the plurality of target fields further include an attribute header information index field located in the attribute slice, the attribute header information index field being used to indicate the attribute header information index used by the attribute type of the attribute data in the corresponding attribute slice, the attribute header information index being used to obtain the corresponding parameter in the attribute header.

[0225] In some embodiments, the plurality of target fields further include a slice data type field and an attribute header information index field located in the attribute slice;

[0226] The slice data type field is used to indicate the attribute prediction type of the attribute data in the corresponding attribute slice, the attribute header information index field is used to indicate the attribute header information index used by the attribute type of the attribute data in the corresponding attribute slice, and the attribute header information index is used to obtain the corresponding parameters in the attribute header.

[0227] In some embodiments, if the piece data type field is a first flag value, the attribute prediction type indicates that the corresponding attribute piece includes color data that does not participate in cross-type attribute prediction; if the piece data type field is a second flag value, the attribute prediction type indicates that the corresponding attribute piece includes reflectance data that does not participate in cross-type attribute prediction or same-type attribute prediction; if the piece data type field is a third flag value, the attribute prediction type indicates that the corresponding attribute piece includes a set of color data and a set of reflectance data that have cross-type attribute prediction; if the piece data type field is a fourth flag value, the attribute prediction type indicates that the corresponding attribute piece includes at least one set of reflectance data that has same-type attribute prediction.

[0228] In some embodiments, if the piece data type field is a first flag value, the attribute prediction type indicates that the corresponding attribute piece includes a set of color data, and the set of color data is not allowed to be predicted across different attribute types; if the piece data type field is a second flag value, the attribute prediction type indicates that the corresponding attribute piece includes a set of reflectance data, and the set of reflectance data is not allowed to be predicted across different attribute types or within the same attribute type; if the piece data type field is a third flag value, the attribute prediction type indicates that the corresponding attribute piece includes a set of color data, and the set of color data is allowed to be predicted across different attribute types; if the piece data type field is a fourth flag value, it indicates that the corresponding attribute piece includes a set of reflectance data, and the set of reflectance data is allowed to be predicted across different attribute types or within the same attribute type.

[0229] In some embodiments, the point cloud stream includes a point cloud sequence, the point cloud sequence includes multiple point cloud data unit structures, the multiple target attribute data and the attribute dependency combination indication information are located in the multiple point cloud data unit structures, and each point cloud data unit structure includes a data unit type field, a data unit load length field and a data unit load field;

[0230] The data unit type field indicates the data type of the corresponding point cloud data unit structure, the data unit load length field indicates the load length of the corresponding point cloud data unit structure, and the data unit load field indicates the load of the corresponding point cloud data unit structure.

[0231] In some embodiments, when the data type is a sequence header type, the data unit payload field is used to indicate the bit stream corresponding to the sequence header; when the data type is an attribute header type, the data unit payload field is used to indicate the bit stream corresponding to the attribute header; when the data type is an attribute slice type, the data unit payload field is used to indicate the bit stream corresponding to the attribute slice; when the data type is an attribute slice block type, the data unit payload field is used to indicate the bit stream corresponding to the attribute slice block.

[0232] The attribute slice includes at least one attribute slice block.

[0233] In some embodiments, when the data type is the attribute piece block type, the corresponding point cloud data unit structure further includes a block combination order field, which is used to indicate the combination order of the corresponding attribute piece blocks when they are combined into the corresponding attribute piece.

[0234] In some embodiments, decoding the point cloud code stream according to the attribute dependency combination indication information includes:

[0235] Based on the attribute dependency combination indication information, at least one target same type attribute prediction group and a first prediction order of the target attribute data in the target same type attribute prediction group are determined from the plurality of target attribute data;

[0236] The target attribute prediction group is decoded according to the first prediction order.

[0237] In some embodiments, decoding the point cloud code stream according to the attribute dependency combination indication information further includes:

[0238] Based on the attribute dependency combination indication information, at least one target cross-type attribute prediction group and a second prediction order of the target attribute data in the target cross-type attribute prediction group are determined from the plurality of target attribute data;

[0239] The target cross-type attribute prediction group is decoded according to the second prediction order.

[0240] It should be noted that the encoding and decoding processes of point cloud media are reversible. Therefore, the syntax definitions involved in the data processing method for point cloud media applied to content production equipment provided in this application embodiment can be found in the above-described embodiment of the data processing method for point cloud media applied to content presentation equipment, and will not be repeated here.

[0241] Based on the methods described in the above embodiments, this application also provides a point cloud media data processing apparatus for performing the steps in the above-described point cloud media data processing method applied to a content presentation device. Please refer to... Figure 5 , Figure 5 This is a schematic diagram of the data processing device for point cloud media provided in an embodiment of this application. The data processing device 300 for point cloud media is applied to a content production device and includes a first acquisition module 301 and a decoding module 302, wherein:

[0242] The first acquisition module 301 is used to acquire the point cloud code stream of the point cloud media. The point cloud code stream includes multiple target attribute data and attribute dependency combination indication information. The attribute dependency combination indication information is used to indicate the encoding and decoding dependency combination relationship between the multiple target attribute data.

[0243] The decoding module 302 is used to decode the point cloud bitstream according to the attribute dependency combination indication information to present the point cloud media.

[0244] It should be noted that the specific details of each module unit in the above-mentioned point cloud media data processing device 300 have been described in detail in the embodiments of the point cloud media data processing method applied to the content presentation device, and will not be repeated here.

[0245] Based on the methods described in the above embodiments, this application also provides a point cloud media data processing apparatus for performing the steps in the point cloud media data processing method applied to a content production device. Please refer to... Figure 6 , Figure 6 This is a schematic diagram of another point cloud media data processing device provided in an embodiment of this application. The point cloud media data processing device 400 is applied to a content production device and includes a second acquisition module 401 and an encoding and compression module 302, wherein:

[0246] The second acquisition module 401 is used to acquire point cloud media;

[0247] Compression module 402 is used to encode and compress the point cloud media to obtain a point cloud code stream. The point cloud code stream includes multiple target attribute data and attribute dependency combination indication information. The attribute dependency combination indication information is used to indicate the encoding and decoding dependency combination relationship between the multiple target attribute data.

[0248] It should be noted that the specific details of each module unit in the above-mentioned point cloud media data processing device 400 have been described in detail in the embodiments of the point cloud media data processing method applied to the content production equipment, and will not be repeated here.

[0249] In some embodiments, the data processing device for point cloud media in this application can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application does not specifically limit the scope of the device.

[0250] In some embodiments, such as Figure 7As shown, this application embodiment also provides an electronic device 500, including a processor 501, a memory 502, and a computer program stored in the memory 502 and executable on the processor 501. When the program is executed by the processor 501, it implements the various processes of the above-described point cloud media data processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0251] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0252] Figure 8 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0253] The electronic device 600 includes, but is not limited to, components such as: radio frequency unit 601, network module 602, audio output unit 603, input unit 604, sensor 605, display unit 606, user input unit 607, interface unit 608, memory 609, and processor 610.

[0254] Those skilled in the art will understand that the electronic device 600 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 610 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 8 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0255] It should be understood that, in this embodiment, the input unit 604 may include a graphics processing unit (GPU) 6041 and a microphone 6042. The GPU 6041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 606 may include a display panel 6061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 607 includes at least one of a touch panel 6071 and other input devices 6072. The touch panel 6071 is also called a touch screen. The touch panel 6071 may include two parts: a touch detection device and a touch controller. Other input devices 6072 may include, but are not limited to, a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick, which will not be described in detail here.

[0256] The memory 609 can be used to store software programs and various data. The memory 609 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 609 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 609 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0257] Processor 610 may include one or more processing units; processor 610 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 610.

[0258] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described point cloud media data processing method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0259] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0260] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described point cloud media data processing method.

[0261] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0262] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0263] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0264] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0265] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0266] In the description of this application, "multiple" means two or more.

[0267] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0268] It should also be understood that the terms “comprising” and / or “including” as used herein specify the presence of the stated features, integers, steps, operations, units and / or components, without excluding the presence or addition of one or more other features, integers, steps, operations, units, components and / or combinations thereof.

[0269] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A data processing method for point cloud media, characterized in that, include: Obtain the point cloud bitstream of point cloud media, wherein the point cloud bitstream includes multiple target attribute data and attribute dependency combination indication information, and the attribute dependency combination indication information is used to indicate the encoding and decoding dependency combination relationship between the multiple target attribute data; The point cloud stream is decoded according to the attribute dependency combination indication information to present the point cloud media; The attribute dependency combination indication information is represented by multiple target fields, including a cross-type attribute prediction field, a cross-type attribute prediction group identifier field, a cross-type attribute prediction order field, a same-type attribute prediction field, a same-type attribute prediction group identifier field, and a same-type attribute prediction order field. Wherein, the cross-type attribute prediction field is used to indicate whether cross-type attribute prediction of the corresponding attribute data is allowed; the cross-type attribute prediction group identifier field is used to indicate the identifier of the cross-type attribute prediction group to which the corresponding attribute data belongs, the cross-type attribute prediction group being a group of attribute data that uses cross-type attribute prediction; the cross-type attribute prediction order field is used to indicate the prediction order of the corresponding attribute data in cross-type attribute prediction; the same-type attribute prediction field is used to indicate whether same-type attribute prediction of the corresponding attribute data is allowed; the same-type attribute prediction group identifier field is used to indicate the identifier of the same-type attribute prediction group to which the corresponding attribute data belongs, the same-type attribute prediction group being a group of attribute data that uses same-type attribute prediction; the same-type attribute prediction order field is used to indicate the prediction order of the corresponding attribute data in same-type attribute prediction.

2. The method according to claim 1, characterized in that, The plurality of target attribute data includes a plurality of first target attribute data of the same attribute type, and the attribute dependency combination indication information is used to indicate the encoding and decoding dependency combination relationship among the plurality of first target attribute data.

3. The method according to claim 2, characterized in that, The plurality of target attribute data also includes a plurality of second target attribute data of the same attribute type. The first target attribute data and the second target attribute data belong to different attribute types. The attribute dependency combination indication information is also used to indicate the encoding and decoding dependency combination relationship between the first target attribute data and the second target attribute data.

4. The method according to claim 1, characterized in that, If the value of the cross-type attribute prediction field is the first flag value, it indicates that cross-type attribute prediction of the corresponding attribute data is not allowed. If the value of the cross-type attribute prediction field is the second flag value, it indicates that cross-type attribute prediction of the corresponding attribute data is allowed; or If the same type attribute prediction field takes the value of the first flag value, it indicates that same type attribute prediction is not allowed for the corresponding attribute data; if the same type attribute prediction field takes the value of the second flag value, it indicates that same type attribute prediction is allowed for the corresponding attribute data.

5. The method according to claim 1, characterized in that, The point cloud code stream includes a point cloud sequence, the point cloud sequence includes an attribute header and an attribute slice, the attribute header includes a set of parameters required for decoding the plurality of target attribute data, and the attribute slice includes the plurality of target attribute data; The cross-type attribute prediction field, the cross-type attribute prediction group identifier field, the same-type attribute prediction field, the same-type attribute prediction group identifier field, and the cross-type attribute prediction order field are located in the attribute header, and the same-type attribute prediction order field is located in the attribute slice.

6. The method according to claim 5, characterized in that, The point cloud sequence further includes a sequence header, which includes a set of parameters required for decoding the point cloud sequence. The plurality of target fields further include a first combination quantity identifier field and a second combination quantity identifier field. The first combination quantity identifier field is used to indicate whether there are multiple sets of attribute data predicted using cross-type attribute in the point cloud sequence, and the second combination quantity identifier field is used to indicate whether there are multiple sets of attribute data predicted using cross-reflectivity attribute in the point cloud sequence. Both the first combination quantity identifier field and the second combination quantity identifier field are located in the attribute header or both are located in the sequence header.

7. The method according to claim 6, characterized in that, If the first combination quantity identifier field is set to the second flag value, it indicates that there are multiple sets of attribute data predicted using cross-type attribute prediction; if the first combination quantity identifier field is set to the first flag value, it indicates that there is at most one set of attribute data predicted using cross-type attribute prediction; or If the value of the second combination quantity identifier field is the second flag value, it indicates that there are multiple sets of attribute data predicted using the transreflectivity attribute; if the value of the second combination quantity identifier field is the first flag value, it indicates that there is at most one set of attribute data predicted using the transreflectivity attribute.

8. The method according to claim 7, characterized in that, The plurality of target fields also include a first identifier indicator field and a second identifier indicator field located in the attribute header. The first identifier indicator field is used to indicate whether to set the identifier of the cross-type attribute prediction group to which the corresponding attribute data belongs; the second identifier indicator field is used to indicate whether to set the identifier of the same-type attribute prediction group to which the corresponding attribute data belongs.

9. The method according to claim 8, characterized in that, If the value of the first identifier indicating field is the first flag value, then the identifier of the cross-type attribute prediction group where the corresponding attribute data is located is not set; if the value of the first identifier indicating field is the second flag value, then the identifier of the cross-type attribute prediction group where the corresponding attribute data is located is set. or If the value of the second identifier indicating field is the first flag value, then the identifier of the same type of attribute prediction group where the corresponding attribute data is located is not set; if the value of the second identifier indicating field is the second flag value, then the identifier of the same type of attribute prediction group where the corresponding attribute data is located is set.

10. The method according to claim 5, characterized in that, The plurality of target fields also include an attribute header information index field located in the attribute slice. The attribute header information index field is used to indicate the attribute header information index used by the attribute type of the attribute data in the corresponding attribute slice. The attribute header information index is used to obtain the corresponding parameter in the attribute header.

11. The method according to claim 5, characterized in that, The plurality of target fields also include slice data type fields and attribute header information index fields located in the attribute slice; The slice data type field is used to indicate the attribute prediction type of the attribute data in the corresponding attribute slice, the attribute header information index field is used to indicate the attribute header information index used by the attribute type of the attribute data in the corresponding attribute slice, and the attribute header information index is used to obtain the corresponding parameters in the attribute header.

12. The method according to claim 11, characterized in that, If the value of the slice data type field is the first flag value, then the attribute prediction type indicates that the corresponding attribute slice includes color data that does not participate in cross-type attribute prediction; If the value of the slice data type field is the second flag value, then the attribute prediction type indicates that the corresponding attribute slice includes reflectance data that does not participate in cross-type attribute prediction or same-type attribute prediction. If the piece data type field is set to a third flag value, the attribute prediction type indicates that the corresponding attribute piece includes a set of color data and a set of reflectance data that have cross-type attribute predictions; if the piece data type field is set to a fourth flag value, the attribute prediction type indicates that the corresponding attribute piece includes at least a set of reflectance data that have same-type attribute predictions.

13. The method according to claim 11, characterized in that, If the piece data type field is set to a first flag value, the attribute prediction type indicates that the corresponding attribute piece includes a set of color data, and the set of color data is not allowed to be predicted across different attribute types. If the piece data type field is set to a second flag value, the attribute prediction type indicates that the corresponding attribute piece includes a set of reflectance data, and the set of reflectance data is not allowed to be predicted across different attribute types or within the same attribute type. If the piece data type field is set to a third flag value, the attribute prediction type indicates that the corresponding attribute piece includes a set of color data, and the set of color data is allowed to be predicted across different attribute types. If the piece data type field is set to a fourth flag value, it indicates that the corresponding attribute piece includes a set of reflectance data, and the set of reflectance data is allowed to be predicted across different attribute types or within the same attribute type.

14. The method according to any one of claims 1-13, characterized in that, The point cloud code stream includes a point cloud sequence, the point cloud sequence includes multiple point cloud data unit structures, the multiple target attribute data and the attribute dependency combination indication information are located in the multiple point cloud data unit structures, and each point cloud data unit structure includes a data unit type field, a data unit load length field and a data unit load field. The data unit type field indicates the data type of the corresponding point cloud data unit structure, the data unit load length field indicates the load length of the corresponding point cloud data unit structure, and the data unit load field indicates the load of the corresponding point cloud data unit structure.

15. The method according to claim 14, characterized in that, When the data type is a sequence header type, the data unit payload field is used to indicate the bit stream corresponding to the sequence header; when the data type is an attribute header type, the data unit payload field is used to indicate the bit stream corresponding to the attribute header; when the data type is an attribute slice type, the data unit payload field is used to indicate the bit stream corresponding to the attribute slice; when the data type is an attribute slice block type, the data unit payload field is used to indicate the bit stream corresponding to the attribute slice block. The attribute slice includes at least one attribute slice block.

16. The method according to claim 15, characterized in that, When the data type is the attribute piece block type, the corresponding point cloud data unit structure also includes a block combination order field, which is used to indicate the combination order of the corresponding attribute piece blocks when they are combined into the corresponding attribute piece.

17. The method according to any one of claims 1-13, characterized in that, Decoding the point cloud code stream based on the attribute dependency combination indication information includes: Based on the attribute dependency combination indication information, at least one target same type attribute prediction group and a first prediction order of the target attribute data in the target same type attribute prediction group are determined from the plurality of target attribute data; The target attribute prediction group is decoded according to the first prediction order.

18. The method according to claim 17, characterized in that, Decoding the point cloud code stream based on the attribute dependency combination indication information further includes: Based on the attribute dependency combination indication information, at least one target cross-type attribute prediction group and a second prediction order of the target attribute data in the target cross-type attribute prediction group are determined from the plurality of target attribute data; The target cross-type attribute prediction group is decoded according to the second prediction order.

19. A data processing method for point cloud media, characterized in that, include: Acquire point cloud media; The point cloud media is encoded and compressed to obtain a point cloud bitstream. The point cloud bitstream includes multiple target attribute data and attribute dependency combination indication information. The attribute dependency combination indication information is used to indicate the encoding and decoding dependency combination relationship between the multiple target attribute data.

20. A data processing device for point cloud media, characterized in that, include: The first acquisition module is used to acquire the point cloud code stream of the point cloud media. The point cloud code stream includes multiple target attribute data and attribute dependency combination indication information. The attribute dependency combination indication information is used to indicate the encoding and decoding dependency combination relationship between the multiple target attribute data. A decoding module is used to decode the point cloud bitstream according to the attribute dependency combination indication information to present the point cloud media; The attribute dependency combination indication information is represented by multiple target fields, including a cross-type attribute prediction field, a cross-type attribute prediction group identifier field, a cross-type attribute prediction order field, a same-type attribute prediction field, a same-type attribute prediction group identifier field, and a same-type attribute prediction order field. Wherein, the cross-type attribute prediction field is used to indicate whether cross-type attribute prediction of the corresponding attribute data is allowed; the cross-type attribute prediction group identifier field is used to indicate the identifier of the cross-type attribute prediction group to which the corresponding attribute data belongs, the cross-type attribute prediction group being a group of attribute data that uses cross-type attribute prediction; the cross-type attribute prediction order field is used to indicate the prediction order of the corresponding attribute data in cross-type attribute prediction; the same-type attribute prediction field is used to indicate whether same-type attribute prediction of the corresponding attribute data is allowed; the same-type attribute prediction group identifier field is used to indicate the identifier of the same-type attribute prediction group to which the corresponding attribute data belongs, the same-type attribute prediction group being a group of attribute data that uses same-type attribute prediction; the same-type attribute prediction order field is used to indicate the prediction order of the corresponding attribute data in same-type attribute prediction.

21. A data processing device for point cloud media, characterized in that, include: The second acquisition module is used to acquire point cloud media; The encoding and compression module is used to encode and compress the point cloud media to obtain a point cloud code stream. The point cloud code stream includes multiple target attribute data and attribute dependency combination indication information. The attribute dependency combination indication information is used to indicate the encoding and decoding dependency combination relationship between the multiple target attribute data.

22. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the data processing method for point cloud media as described in any one of claims 1-18 or performs the data processing method for point cloud media as described in claim 19.

23. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the data processing method for point cloud media as described in any one of claims 1-18 or performs the data processing method for point cloud media as described in claim 19.

Images