Point cloud processing method and apparatus, storage medium, and electronic device

Abstract

The present application discloses a point cloud processing method and apparatus, a storage medium, and an electronic device. The method comprises: acquiring decoding indication information of a point cloud bitstream, wherein the decoding indication information comprises at least one of a first indication parameter and second indication parameters, the first indication parameter is set in sequence-level parameter information of a point cloud sequence, the second indication parameters are set in frame-level or slice-level parameter information of the point cloud sequence, the first indication parameter is used for performing sequence-level decoding indication, and the second indication parameters are used for performing frame-level or slice-level decoding indication; and decoding the point cloud bitstream on the basis of the first indication parameter or the second indication parameters. By means of the sequence-level first indication parameter, decoding indication is uniformly performed on each point cloud frame in the entire point cloud sequence, and when decoding is performed on the basis of the frame-level or slice-level second indication parameters, the second indication parameters can be quickly acquired when parsing point cloud frames or point cloud slices, and the decoding efficiency is high.

Description

Point cloud processing method, device, storage medium and electronic equipment

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on September 21, 2023, with application number 2023112347996 and invention name “Point cloud processing method, device, storage medium and electronic device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application belongs to the field of image coding and decoding technology, and in particular relates to a point cloud processing method, device, storage medium and electronic device. Background Art

[0003] A point cloud is a collection of points obtained by collecting data on an object's surface using measuring instruments. Point clouds are widely used in autonomous driving, high-precision mapping, and virtual reality. However, because point clouds often consist of tens of thousands to hundreds of millions of three-dimensional points, they pose significant challenges to data storage, processing, transmission, and display. Current storage resources and transmission bandwidth are limited, making it essential to encode and compress point clouds into a point cloud stream and then use that stream for storage or transmission.

[0004] Currently, the more common point cloud coding and compression methods include Geometry-based Point Cloud Compression (GPCC). In point cloud compression, a sequence header is generally used to decode and indicate parameters that change frame by frame. The sequence header must define the corresponding indication information for each point cloud frame. When decoding each point cloud frame, the sequence header must be parsed to obtain the corresponding indication information, resulting in low decoding efficiency.

[0005] Summary of the Invention

[0006] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a point cloud processing method, device, storage medium and electronic device that can improve the efficiency of point cloud decoding.

[0007] In the first aspect, the present application provides a point cloud processing method, which is applied to a decoding device, including obtaining decoding indication information of a point cloud code stream, the decoding indication information including at least one of a first indication parameter and a second indication parameter, the first indication parameter being set in the sequence-level parameter information of the point cloud sequence, and the second indication parameter being set in the frame-level or slice-level parameter information of the point cloud sequence, the first indication parameter being used for sequence-level decoding indication, and the second indication parameter being used for frame-level or slice-level decoding indication; the point cloud code stream is decoded according to the first indication parameter or the second indication parameter.

[0008] In the second aspect, the present application provides a point cloud processing method, which is applied to an encoding device, including acquiring point cloud data; encoding and compressing the point cloud data to obtain a point cloud code stream, wherein the decoding indication information of the point cloud code stream includes at least one of a first indication parameter and a second indication parameter, the first indication parameter is set in the sequence-level parameter information of the point cloud sequence, and the second indication parameter is set in the frame-level or slice-level parameter information of the point cloud sequence, the first indication parameter is used for sequence-level decoding indication, and the second indication parameter is used for frame-level or slice-level decoding indication, and the point cloud code stream is decoded according to the first indication parameter or the second indication parameter.

[0009] In a third aspect, the present application provides a point cloud processing device, which is applied to a decoding device, and includes a first acquisition module and a decoding module. The first acquisition module is used to obtain decoding indication information of a point cloud code stream, and the decoding indication information includes at least one of a first indication parameter and a second indication parameter. The first indication parameter is set in the sequence-level parameter information of the point cloud sequence, and the second indication parameter is set in the frame-level or slice-level parameter information of the point cloud sequence. The first indication parameter is used to perform sequence-level decoding indication, and the second indication parameter is used to perform frame-level or slice-level decoding indication; the decoding module is used to decode the point cloud code stream according to the first indication parameter or the second indication parameter.

[0010] In a fourth aspect, the present application provides a point cloud processing device, which is applied to an encoding device and includes a second acquisition module and an encoding compression module. The second acquisition module is used to acquire point cloud data; the encoding compression module is used to encode and compress the point cloud data to obtain a point cloud code stream, and the decoding indication information of the point cloud code stream includes at least one of a first indication parameter and a second indication parameter, the first indication parameter is set in the sequence-level parameter information of the point cloud sequence, and the second indication parameter is set in the frame-level or slice-level parameter information of the point cloud sequence, the first indication parameter is used to perform sequence-level decoding indication, and the second indication parameter is used to perform frame-level or slice-level decoding indication, and the point cloud code stream is decoded according to the first indication parameter or the second indication parameter.

[0011] In a fifth aspect, the present application provides a non-transitory computer-readable storage medium on which a computer program is stored, characterized in that when the computer program is executed by a processor, it implements any of the above-mentioned point cloud processing methods.

[0012] In a sixth aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any one of the above-described point cloud processing methods when executing the program.

[0013] In a seventh aspect, the present application provides a decoding device comprising a processor and a memory. The memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the above-mentioned point cloud processing method on the decoding side.

[0014] In an eighth aspect, the present application provides an encoding device comprising a processor and a memory. The memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the above-mentioned point cloud processing method on the encoding side.

[0015] In a ninth aspect, the present application provides a chip for implementing the method in the above embodiments. Specifically, the chip includes: a processor for calling and running a computer program from a memory, so that a device equipped with the chip executes any of the above point cloud processing methods.

[0016] In a tenth aspect, the present application provides a computer program product, comprising computer program instructions, wherein the computer program instructions enable a computer to execute any one of the above-described point cloud processing methods.

[0017] In an eleventh aspect, the present application provides a computer program which, when executed on a computer, enables the computer to execute any of the above-described point cloud processing methods.

[0018] The point cloud processing method, apparatus, storage medium, and electronic device provided in the embodiments of the present application obtain decoding indication information of a point cloud code stream and define a first indication parameter at the sequence level and a second indication parameter at the frame level or slice level in the decoding indication information, wherein the first indication parameter is set in the sequence-level parameter information of the point cloud sequence, and the second indication parameter is set in the frame-level or slice-level parameter information of the point cloud sequence. Decoding is then performed based on the first indication parameter or the second indication parameter. When decoding based on the first indication parameter at the sequence level, each point cloud frame in the point cloud sequence is decoded using a unified first indication parameter decoding, eliminating the need to parse the corresponding indication parameter for each point cloud frame for decoding, resulting in higher decoding efficiency. When decoding according to the second indication parameter at the frame level or the slice level, frame level or slice level decoding can be achieved according to the second indication parameter, so that each point cloud frame or point cloud slice has an adapted second indication parameter, realizing frame-by-frame or slice-by-slice change of the second indication parameter. Compared with parsing the point cloud frame each time, it is also necessary to parse the sequence header to obtain the indication parameter corresponding to the point cloud frame for decoding, and the decoding efficiency is low. There is no need to go to the sequence header to parse the corresponding indication parameter for each point cloud frame, and the corresponding second indication parameter can be quickly obtained, and the decoding efficiency is relatively high. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0020] FIG1a is a schematic diagram showing a 3DoF degree of freedom provided by an embodiment of the present application;

[0021] FIG1b is a schematic diagram showing the degrees of freedom of 3DoF+ provided in an embodiment of the present application;

[0022] FIG1c is a schematic diagram showing a 6DoF degree of freedom according to an embodiment of the present application;

[0023] FIG2 is a schematic diagram of a framework of a point cloud processing system for point cloud media provided in an embodiment of the present application;

[0024] FIG3 is a schematic diagram of a coding framework for point cloud coding provided in an embodiment of the present application;

[0025] FIG4 is a schematic diagram of a flow chart of a point cloud processing method provided in an embodiment of the present application;

[0026] FIG5 is a flow chart of a point cloud processing method according to an embodiment of the present application;

[0027] FIG6 is a flow chart of a point cloud processing method according to an embodiment of the present application;

[0028] FIG7 is a flow chart of another point cloud processing method provided in an embodiment of the present application;

[0029] FIG8 is a schematic structural diagram of a point cloud processing device provided in an embodiment of the present application;

[0030] FIG9 is a schematic structural diagram of another point cloud processing device provided in an embodiment of the present application;

[0031] FIG10 is a schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0032] FIG11 is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] The following describes in detail embodiments of the present application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and are not to be construed as limiting the present application.

[0034] The following is an introduction to some of the technical terms involved in this application:

[0035] 1. Point Cloud

[0036] A point cloud is a set of randomly distributed discrete points in space that represent the spatial structure and surface properties of a three-dimensional object or scene. Each point in a point cloud includes at least geometric data, which represents the point's three-dimensional position. Depending on the application scenario, a point in a point cloud may also include one or more sets of attribute data. Each set of attribute data reflects a specific attribute of the point, such as color, material, or other information. Typically, each point in a point cloud has the same number of sets of attribute data.

[0037] Point clouds can flexibly and conveniently express the spatial structure and surface properties of three-dimensional objects or scenes, and are therefore widely used in scenarios such as virtual reality (VR) games, computer-aided design (CAD), geographic information systems (GIS), autonomous navigation systems (ANS), digital cultural heritage, free viewpoint broadcasting, three-dimensional immersive telepresence, and three-dimensional reconstruction of biological tissues and organs.

[0038] Point clouds are primarily acquired through computer generation, three-dimensional (3D) laser scanning, and 3D photogrammetry. Specifically, point clouds can be obtained by capturing real-world visual scenes using a collection device (a set of cameras or a camera device with multiple lenses and sensors). 3D laser scanning can generate point clouds of static real-world 3D objects or scenes, with millions of point clouds acquired per second. 3D photography can generate point clouds of dynamic real-world 3D objects or scenes, with tens of millions of point clouds acquired per second. Furthermore, in the medical field, point clouds of biological tissues and organs can be obtained through magnetic resonance imaging (MRI), computed tomography (CT), and electromagnetic positioning information. For example, point clouds can also be generated directly by computers based on virtual 3D objects and scenes. With the continuous accumulation of large-scale point cloud data, efficient storage, transmission, publication, sharing, and standardization of point cloud data have become key to point cloud applications.

[0039] 2. Point Cloud Media

[0040] Point cloud media is a typical 6DoF immersive media. Immersive media refers to a media file that can provide immersive media content, so that viewers immersed in the media content can obtain visual, auditory and other sensory experiences in the real world. According to the degree of freedom of the viewer when consuming media content, immersive media can be divided into: 6DoF (Degree of Freedom) immersive media, 3DoF immersive media, and 3DoF+ immersive media. Among them, please refer to Figure 1a, Figure 1a is a 3DoF degree of freedom display diagram provided in an embodiment of the present application. 3DoF refers to three degrees of freedom, that is, the viewer of the immersive media is fixed at the center point of a three-dimensional space, and the head of the viewer of the immersive media rotates along the X-axis, Y-axis and Z-axis to view the picture provided by the media content. Please refer to Figure 1b, Figure 1b is a 3DoF+ degree of freedom display diagram provided in an embodiment of the present application. 3DoF+ means that on the basis of the three degrees of freedom, the head of the viewer of the immersive media also has limited freedom of movement along the X-axis, Y-axis and Z-axis. Please refer to Figure 1c, which is a schematic diagram showing 6DoF provided in an embodiment of the present application. 6DoF means that on the basis of the three degrees of freedom, the head of the immersive media viewer also has the freedom to move freely along the X-axis, Y-axis and Z-axis.

[0041] It should be noted that the point cloud processing method provided in the embodiment of the present application can be applied to the point cloud processing system of point cloud media, which may include a content production device (corresponding to an encoding device) and a content presentation device (corresponding to a decoding device). The content production device may refer to a computer device used by a provider of point cloud media (e.g., a content producer of point cloud media), which may be a terminal (such as a PC (Personal Computer), a smart mobile device (such as a smart phone), etc.) or a server. The server may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The content presentation device may refer to a computer device used by a user of point cloud media (e.g., a viewer of point cloud media, i.e., a business object), and the computer device may be a terminal (e.g., a PC (Personal Computer), a smart mobile device (e.g., a smart phone), a VR device (e.g., a VR helmet, VR glasses, etc.), a smart home appliance, a vehicle-mounted terminal, an aircraft, etc.), and the computer device is integrated with a client. The client here may be a client having the function of displaying data information such as text, images, audio, and video, including but not limited to multimedia clients (e.g., video clients), social clients (e.g., instant messaging clients), information applications (e.g., news clients), entertainment clients (e.g., game clients), shopping clients, vehicle-mounted clients, browsers, etc. The client may be an independent client or an embedded sub-client integrated in a client (e.g., a social client), which is not limited here.

[0042] The content production device and the content presentation device can be the same device or different devices, each of which includes multiple modules, different modules are used to implement different functions, and these modules can be integrated into the same electronic device or located in different electronic devices. The content production device can be used to implement functions such as the acquisition, encoding, and packaging of point cloud media. Correspondingly, the content presentation device can be used to implement functions such as decapsulation, decoding, rendering, and display of encapsulated files. Please refer to Figure 2, which is a schematic diagram of the data processing framework of the point cloud processing system for point cloud media provided in an embodiment of the present application.

[0043] In Figure 2, on the content production device side, the real-world visual scene A is captured by a group of cameras or a camera device with multiple lenses and sensors. The capture result is point cloud media B, which is a frame sequence consisting of a large number of point cloud frames. One or more point cloud frames are encoded into a point cloud code stream E, including an encoded geometry bit stream and an attribute bit stream. Afterwards, the point cloud code stream E is encapsulated according to a specific media container file format (such as ISOBMFF) to obtain a sequence of initialization segments and media segments (Fs) for streaming or a media file (F) for file playback. During the encapsulation process, metadata is also included in the file F or the media segment Fs. Afterwards, the segment Fs can be transmitted to the content presentation device using a transmission mechanism. This transmission mechanism may include but is not limited to: DASH protocol, HLS protocol, SMTP protocol, TCP protocol, etc.

[0044] On the content presentation device side, the sequence (Fs) or media file (F) is decapsulated to obtain the point cloud stream E' and parse the metadata. The point cloud stream E' is then decoded to generate point cloud media D'. After that, the point cloud media D' is rendered and the corresponding visual scene A' is presented. During the data processing process on the content presentation device side, the decoded point cloud media D' can be rendered and displayed on the screen of a head-mounted display or any other display device based on the current viewer's viewing position, viewing direction, or a window determined by various types of sensors (such as head, position, or eye movement sensors). Partially accessing the decoded point cloud media through the current viewing position or viewing direction can be used to optimize the processing of the point cloud media.

[0045] It is understood that the point cloud processing technology involved in this application (primarily including encoding and decoding technologies) can be implemented based on cloud technology; for example, using a cloud server as a content production device. Cloud technology refers to a hosting technology that unifies hardware, software, network and other resources within a wide area network or local area network to achieve data computing, storage, processing and sharing.

[0046] 3. Point Cloud Data. The geometric information and attribute information of each point in the point cloud together constitute the point cloud data. Among them, geometric information can also be called three-dimensional position information. The geometric information of a point in the point cloud refers to the spatial coordinates (x, y, z) of the point, which can include the coordinate values ​​of the point in the directions of each coordinate axis of the three-dimensional coordinate system, for example, the coordinate value x in the X-axis direction, the coordinate value y in the Y-axis direction, and the coordinate value z in the Z-axis direction. The attribute information of a point in the point cloud may include at least one of the following: color information, material information, laser reflection intensity information (also called reflectivity); usually, each point in the point cloud has the same amount of attribute information. For example, each point in the point cloud can have two kinds of attribute information: color information and laser reflection intensity. For another example, each point in the point cloud can have three kinds of attribute information: color information, material information, and laser reflection intensity information.

[0047] 4. Point Cloud Compression (PCC). Point cloud coding refers to the process of encoding the geometric information and attribute information of each point in the point cloud to obtain a compressed code stream (also called a point cloud code stream). Point cloud coding can include two main processes: geometric information encoding and attribute information encoding. The current mainstream point cloud coding technology can be divided into geometric structure-based point cloud coding and projection-based point cloud coding according to different types of point clouds. Here, G-PCC (Geometry-Based Point Cloud Compression) in MPEG (Moving Picture Expert Group, international audio and video coding standard) and AVS-PCC, the point cloud coding standard in AVS (Audio Video Coding Standard, China's national video coding standard) are introduced as examples.

[0048] The coding frameworks of G-PCC and AVS-PCC are roughly the same. Figure 3 shows the G-PCC coding framework, taking G-PCC as an example. It can be divided into the geometry information encoding process and the attribute information encoding process. In the geometry information encoding process, the geometry information of each point in the point cloud is encoded to obtain a geometry bitstream. In the attribute information encoding process, the attribute information of each point in the point cloud is encoded to obtain an attribute bitstream. Together, the geometry bitstream and the attribute bitstream constitute the compressed bitstream of the point cloud.

[0049] For the geometric information encoding process, the main operations and processing can be seen in the following description:

[0050] ① Pre-processing: This can include coordinate transformation and voxelization. Through scaling and translation operations, the point cloud data in 3D space is converted into integer form and its minimum geometric position is moved to the coordinate origin.

[0051] ②Geometric coding: Geometric coding can include two modes, namely octree-based geometric coding (Octree) and triangle representation-based geometric coding (Trisoup). These two coding modes can be used under different conditions.

[0052] in:

[0053] Octree-based geometric coding: An octree is a tree-like data structure that evenly divides a predefined bounding box in three-dimensional space, with each node having eight children. By indicating whether each child node in the octree is occupied using "1" or "0," an occupancy code is generated as the code stream of the point cloud geometry information.

[0054] Geometric encoding based on triangulation: Divide the point cloud into blocks of a certain size, locate the intersection points of the point cloud surface at the edge of the block and construct triangles. By encoding the intersection points, geometric information is compressed.

[0055] ③ Geometry Quantization: The degree of quantization fineness is usually determined by the quantization parameter (QP). A larger QP value means that coefficients with a larger value range will be quantized into the same output, which usually results in greater distortion and a lower bit rate. Conversely, a smaller QP value means that coefficients with a smaller value range will be quantized into the same output, which usually results in less distortion and a corresponding higher bit rate.

[0056] ④ Geometry Entropy Encoding: Statistical compression encoding is performed on the octtree's occupancy code information, ultimately outputting a binary (0 or 1) compressed bitstream. Statistical coding is a lossless encoding method that can effectively reduce the bitrate required to express the same signal. A commonly used statistical coding method is context-adaptive binary arithmetic coding (CABAC).

[0057] For the attribute information encoding process, the main operations and processing can be found in the following description:

[0058] 1. Attribute Recoloring: In lossy encoding, after encoding the geometric coordinates, the encoder needs to decode and reconstruct the geometry, i.e., restore the geometry of each point in the point cloud. The attribute information corresponding to one or more neighboring points in the original point cloud is searched and used as the attribute information for the reconstructed point.

[0059] ② Attribute information processing: Attribute information processing can include three attribute encoding modes, namely attribute prediction (Prediction), attribute transformation (Transform), and attribute prediction transformation (Prediction & Transform). These three attribute encoding modes can be used under different conditions.

[0060] in:

[0061] Attribute prediction: Based on information such as distance or spatial relationships, neighbor prediction points of the point to be coded are determined from among the already coded points. Based on the set criteria, the attribute prediction information of the point to be coded is calculated based on the attribute information of the neighbor prediction points. The difference between the actual attribute information of the point to be coded and the predicted attribute information is calculated as the attribute residual information, which is then quantized and entropy coded.

[0062] Attribute transformation: Using transformation methods such as DCT (Discrete Cosine Transform) and Haar (Haar Transform), attribute information is grouped and transformed to obtain transformation coefficients, which are then quantized and entropy-coded.

[0063] Attribute prediction transformation: The preceding steps are the same as attribute prediction. After obtaining the attribute residual information of the point to be encoded, the attribute residual information of the point to be encoded is transformed using a transformation algorithm to obtain transformation coefficients, which are then quantized and entropy coded.

[0064] Before attribute prediction transformation, the geometric decoding points need to be grouped. The grouping process is explained below using color attributes and reflectance attributes respectively.

[0065] For color attributes: group the reordered point cloud geometric points in sequence and calculate the initial right shift number L base , according to L base Get the current right shift number, group by the current right shift number, and set the maximum transformation order to colorMaxTransNum. The specific grouping rules are as follows:

[0066] a. Initial right shift number L base The calculation formula is as follows:

[0067] b. Divide the points that are identical after the Hilbert code is right-shifted by L bits into the same group. If the number of points in the current block is less than or equal to the maximum transformation order colorMaxTransNum, they are grouped together. If the number of points in the current block is greater than colorMaxTransNum, the points in the block are further subdivided into groups. The rules for subdividing groups are as follows:

[0068] When subdividing the group, get the current right shift bit L, the right shift bit L1 of the current subdivided group = L-1, then group the same points after the Hilbert code is right shifted by L1 bits into a new group, and continue to determine whether the number of points in the new group is greater than colorMaxTransNum. If it is greater than colorMaxTransNum, continue to repeat this operation until the number of points in each group is less than or equal to colorMaxTransNum.

[0069] c. Dynamically adjust L. The specific rule is to calculate the average value B of the number of geometric points in 8 consecutive groups. If B is less than 2, then L = L + 1; if B is greater than 8, then L = L - 1; otherwise, L remains unchanged. L is an integer greater than or equal to 3, and the value of L is adjusted every 32 groups.

[0070] d. Check if the points in the previous group are duplicates. If there are duplicates, use the duplicates to complete the group to colorMaxTransNum points. Subsequent duplicate points are grouped by colorMaxTransNum, and the remaining points are automatically grouped.

[0071] For the reflectivity attribute: group the reordered geometric points in sequence, calculate the initial right shift number, obtain the current right shift number, group them according to the current right shift number, and set the maximum transformation order to refMaxTransNum. The specific grouping rules are as follows:

[0072] a. When refMaxTransNum is not equal to 4, the initial right shift number L base =3; when refMaxTransNum is equal to 4, the initial right shift number is L base The calculation formula is as follows:

[0073] The maximum number of bits maxBits is the log2 logarithm of the volume of the point cloud bounding box: maxBits = log2(xyz)

[0074] The minimum number of bits minBits is the log2 logarithm of the number of point cloud points voxelCount: minBits = log2(voxelCount)

[0075] The initial offset shift is obtained by judging the size of attrQuantParam in sps. If attrQuantParam is greater than or equal to 32, shift is equal to 12, otherwise shift is equal to -6: shift = (sps.attrQuantParam>=32)? 12: -6

[0076] b. Shift the Hilbert code right by L base Points with the same position are grouped into the same macroblock. Points in each macroblock are grouped. If the current point is a duplicate point, the duplicate point is used to fill the group to refMaxTransNum points.

[0077] If the number of points in the jth block is K j If it is greater than refMaxTransNum, the j-th block is further grouped by the following method: take refMaxTransNum points in sequence until the number of points in each group within the block is less than or equal to refMaxTransNum.

[0078] c. When refMaxTransNum>2, the right shift number L is dynamically adjusted.

[0079] Set the initial value of L to L base Count the total number of geometric points in the first N groups (N = 8) and calculate the average number of points in each group. If the average number of points is less than 2, then L = L + 1; if the average number of points is greater than refMaxTransNum, then L = L - 1; otherwise, L remains unchanged. L is an integer greater than or equal to 1 and is adjusted every 8 groups.

[0080] ③ Attribute Quantization: The degree of quantization is usually determined by the quantization parameter. The transform coefficients and / or attribute residual information obtained from attribute information processing are quantized and entropy coded. For example, in attribute prediction, entropy coding is performed on the quantized attribute residual information; in attribute transformation and attribute prediction transformation, entropy coding is performed on the quantized transform coefficients.

[0081] 4. Attribute Entropy Coding: Quantized attribute residual information and / or transform coefficients are typically compressed using entropy coding methods such as run-length coding and arithmetic coding. The corresponding attribute coding mode, quantization parameters, and other information are also encoded using an entropy encoder.

[0082] 5. Point cloud decoding. Point cloud decoding refers to the process of decoding the compressed code stream obtained by point cloud encoding to reconstruct the point cloud. Specifically, decoding the point cloud stream code mainly involves two parts: decoding the geometric bit stream of the point cloud media (i.e., target geometry data), and decoding the attribute bit stream of the point cloud media (i.e., target attribute data). The decoding process of the point cloud code stream is the inverse of the encoding and compression process. After decoding the point cloud code stream, the content presentation device can further render and display the decoded data to present the point cloud media to the user (see Figure 2 above).

[0083] Here is a detailed description of the point cloud decoding process in attribute prediction mode and attribute prediction transformation mode:

[0084] In the attribute prediction mode, the quantized attribute residual information of the point to be decoded can be obtained from the attribute bit stream through entropy decoding, and the quantized attribute residual information is dequantized to obtain the attribute residual information of the point to be decoded; according to information such as distance or spatial relationship, the neighbor prediction points of the point to be decoded are determined among the decoded points, and based on the set criteria, the attribute prediction information of the point to be decoded is calculated according to the attribute reconstruction information of the neighbor prediction points; then, the attribute information of the point to be decoded (which can be called attribute reconstruction information) can be reconstructed according to the attribute residual information of the point to be decoded and the attribute prediction information of the point to be decoded.

[0085] In the attribute prediction transformation mode, the quantized transformation coefficients of the point to be decoded can be obtained from the attribute bit stream through entropy decoding, the quantized transformation coefficients are inversely quantized to obtain the transformation coefficients, and the transformation coefficients are inversely transformed to obtain the attribute residual information of the point to be decoded; according to information such as distance or spatial relationship, the neighbor prediction points of the point to be decoded are determined among the decoded points, and based on the set criteria, the attribute prediction information of the point to be decoded is calculated according to the attribute reconstruction information of the neighbor prediction points; then, the attribute information of the point to be decoded (which can be called attribute reconstruction information) can be reconstructed according to the attribute residual information of the point to be decoded and the attribute prediction information of the point to be decoded.

[0086] Based on the introduction of the above basic concepts, the embodiments of the present application provide a point cloud processing method, device, storage medium and electronic device.

[0087] To support the point cloud processing method in this embodiment, this embodiment adds several descriptive fields to the system layer, building upon existing techniques and defining corresponding high-level grammatical data structures. The point cloud processing method will be described in detail in conjunction with various grammatical tables in the following sections.

[0088] In the embodiments of this application, a start code exists in the point cloud sequence. The start code is a set of specific bit strings that can be used to extract the corresponding content of each part of the point cloud sequence. In the point cloud code stream that conforms to the embodiments of this application, these bit strings should not appear in any context other than the start code.

[0089] The start code consists of a start code prefix and a start code value. The start code prefix is ​​the bit string '0000 0000 0000 0000 00000001'. All start codes should be byte aligned.

[0090] The start code value is an 8-bit integer used to indicate the type of the start code. Please refer to Table 1 below. Table 1 is a type table of the start code provided in an embodiment of the present application.

[0091] Table 1

[0092] In this embodiment of the present application, a point cloud sequence (PCC_sequence) is the highest-level syntax structure of a point cloud codestream. A point cloud sequence begins with a sequence header (sequence_header), followed by one or more point cloud frames. Each point cloud frame should be preceded by a geometry header (geometry_header), an attribute header (attribute_header), and one or more point cloud slices. The point cloud slice data includes a slice header and a data bitstream. When the sequence header is successfully parsed, the rest of the data in the point cloud codestream is parsed.

[0093] Alternatively, a point cloud sequence begins with a sequence header, geometry header, and attribute header, followed by one or more point cloud frames. Each point cloud frame should be preceded by a frame header and one or more point cloud slices. Once the sequence header, geometry header, and attribute header are successfully parsed, the rest of the data in the codestream can be parsed.

[0094] The sequence header includes the parameter set required for decoding the point cloud sequence. The geometry header includes the parameter set required for decoding the point cloud's geometric data. The attribute header includes the parameter set required for decoding the point cloud's attribute data (such as the target attribute data in this point cloud code stream).

[0095] The point cloud slice here may include geometry slice information (referred to as geometry slice, geometry_slice), and may also include attribute slice information (referred to as attribute slice, attribute_slice). Each point cloud slice consists of a slice header and a data bitstream (geometry_data_bitstream).

[0096] A geometry slice consists of a geometry slice header and a geometry data bitstream. The geometry data bitstream contains the geometry data for the corresponding geometry slice. An attribute slice consists of an attribute slice header and an attribute data bitstream (general_attribute_data_bitstream). The attribute data bitstream contains the attribute data for the corresponding attribute slice. In other words, the slice header includes the geometry slice header and the attribute slice header, and the data bitstream includes the geometry data bitstream and the attribute data bitstream.

[0097] It should be pointed out that each part of the above-mentioned point cloud sequence can include decoding indication information. The embodiment of the present application mainly adds several descriptive fields to the sequence header, frame header, attribute header and attribute slice header as decoding indication information to support the real-time steps of the point cloud processing method of the present application.

[0098] Please refer to FIG4 , which is a flow chart of a point cloud processing method provided in an embodiment of the present application. The point cloud processing method can be applied to a decoding device. Optionally, the decoding device can include a content presentation device, specifically including:

[0099] Step 011: The decoding end obtains decoding indication information of the point cloud code stream. The decoding indication information includes at least one of a first indication parameter and a second indication parameter. The first indication parameter is set in the sequence-level parameter information of the point cloud sequence, and the second indication parameter is set in the frame-level or slice-level parameter information of the point cloud sequence. The first indication parameter is used for sequence-level decoding indication, and the second indication parameter is used for frame-level or slice-level decoding indication.

[0100] Step 012: The decoding end decodes the point cloud code stream according to the first indication parameter or the second indication parameter.

[0101] The point cloud codestream is the bitstream structure of the compressed point cloud media. It can be extracted by a decoding device (such as a content presentation device) after decapsulating the encapsulated file. Point cloud media includes attribute data and geometric data. Attribute data can include various types, such as color and reflectivity. Coding and compression methods can include geometry-based point cloud compression (GPCC).

[0102] The point cloud sequence is formed by multiple continuous point cloud frames, and the frame number of the point cloud frame is the sequence number of the point cloud frame in the point cloud sequence.

[0103] The decoding instruction information is used to indicate the decoding of different data in the point cloud code stream. The decoding instruction information can be set in the sequence header, frame header, attribute header, attribute slice header, etc.

[0104] A point cloud sequence can use the same attribute header and geometry header. That is, the sequence-level parameter information may include the sequence header, attribute header, geometry header, point cloud code stream, etc., and the frame-level parameter information may include the frame header. Alternatively, each point cloud frame has a corresponding attribute header and geometry header. In this case, the sequence-level parameter information may include the sequence header, point cloud code stream, etc., and the frame-level parameter information may include the attribute header, geometry header, frame header, etc. The slice-level parameter information may include the geometry slice, attribute slice, geometry data bit stream, attribute data bit stream, etc.

[0105] The sequence-level decoding indication information can be used to provide decoding instructions for each point cloud frame in the point cloud sequence; the frame-level decoding indication information can be used to provide decoding instructions for the corresponding point cloud frame (such as the point cloud frame where the decoding indication information is located); the slice-level decoding indication information can be used to provide decoding instructions for the corresponding point cloud slice (such as the point cloud slice where the decoding indication information is located).

[0106] For example, the decoding indication information set in the sequence header can be used to provide decoding instructions for each point cloud frame in the point cloud sequence; the decoding indication information set in the frame header can be used to provide decoding instructions for the corresponding point cloud frame; if the attribute header is sequence-level parameter information, the decoding indication information set in the attribute header can be used to provide attribute decoding instructions for each point cloud frame in the point cloud sequence; the decoding indication information set in the attribute slice header can be used to provide decoding instructions for the corresponding point cloud slice.

[0107] The following describes the decoding instructions set in the sequence header, attribute header, frame header, and attribute slice header:

[0108] (1) Decoding instruction information set in the sequence header

[0109] The point cloud code stream may include multiple point cloud frames, and multiple consecutive point cloud frames form a point cloud sequence. The decoding indication information may be set in the sequence header of the point cloud sequence. The decoding indication information set in the sequence header of the point cloud sequence may be used to provide decoding instructions for each point cloud frame.

[0110] Before introducing the decoding indication information set in the sequence header, the syntax elements of the general sequence header are introduced here. The syntax elements of the general sequence header are as shown in Table 2:

[0111] Table 2

[0112] The syntax elements in the general sequence header shown in Table 2 above are explained below:

[0113] Profile ID (profile_id): 8-bit unsigned integer. Indicates the profile to which the bitstream conforms.

[0114] Level ID (level_id): 8-bit unsigned integer. Indicates the level to which the bitstream conforms.

[0115] FrameRate: Unsigned integer. Indicates the frame rate of the point cloud sequence (frames per second).

[0116] Bounding box origin x coordinate upper 16 bits (bounding_box_offset_x_upper): 16-bit number. Represents the upper 16 bits of a 32-bit signed integer representing the bounding box x coordinate.

[0117] Bounding box origin x coordinate lower 16 bits (bounding_box_offset_x_lower): 16-bit number. Represents the lower 16 bits of the 32-bit signed integer of the bounding box origin x coordinate.

[0118] The x coordinate of the bounding box origin is: Bounding_box_offset_x = (bounding_box_offset_x_upper) << 16 + bounding_box_offset_x_lower.

[0119] Upper 16 bits of the bounding box origin y coordinate (bounding_box_offset_y_upper): 16-bit number. Represents the upper 16 bits of a 32-bit signed integer representing the bounding box y coordinate.

[0120] Bounding box origin y coordinate lower 16 bits (bounding_box_offset_y_lower): 16-bit number. Represents the lower 16 bits of the 32-bit signed integer of the bounding box origin y coordinate.

[0121] The y coordinate of the bounding box origin is: Bounding_box_offset_y = (bounding_box_offset_y_upper) << 16 + bounding_box_offset_y_lower.

[0122] Bounding box origin z coordinate upper 16 bits (bounding_box_offset_z_upper): 16-bit number. Represents the upper 16 bits of a 32-bit signed integer representing the bounding box z coordinate.

[0123] Bounding box origin z coordinate lower 16 bits (bounding_box_offset_z_lower): 16-bit number. Represents the lower 16 bits of the 32-bit signed integer of the bounding box origin z coordinate.

[0124] The z coordinate of the bounding box origin is: Bounding_box_offset_z = (bounding_box_offset_z_upper)<<16 + bounding_box_offset_z_lower.

[0125] Bounding box width upper 16 bits (bounding_box_size_width_upper): 16-bit number. Represents the upper 16 bits of a 32-bit unsigned integer representing the bounding box width.

[0126] Bounding box width lower 16 bits (bounding_box_size_width_lower): 16-bit number. Represents the lower 16 bits of a 32-bit unsigned integer representing the bounding box width.

[0127] The bounding box width is: bounding_box_size_width = (bounding_box_size_width_upper) << 16 + bounding_box_size_width_lower.

[0128] Bounding box height upper 16 bits (bounding_box_size_height_upper): 16-bit number. Represents the upper 16 bits of a 32-bit unsigned integer representing the bounding box height.

[0129] Bounding box height lower 16 bits (bounding_box_size_height_lower): 16-bit number. Represents the lower 16 bits of a 32-bit unsigned integer representing the bounding box height.

[0130] The bounding box height is: bounding_box_size_height = (bounding_box_size_height_upper)<<16+bounding_box_size_height_lower.

[0131] Bounding box depth upper 16 bits (bounding_box_size_depth_upper): 16-bit number. Represents the upper 16 bits of a 32-bit unsigned integer representing the bounding box depth.

[0132] Bounding box depth lower 16 bits (bounding_box_size_depth_lower): 16-bit number. Represents the lower 16 bits of a 32-bit unsigned integer representing the bounding box depth.

[0133] The bounding box depth is: bounding_box_size_depth = (bounding_box_size_depth_upper) << 16 + bounding_box_size_depth_lower.

[0134] Geometry quantization step upper 16 bits (geometry_quant_step_upper): 16-bit value. Indicates the upper 16 bits of the geometry quantization step 32-bit floating point number.

[0135] Geometry quantization step lower 16 bits (geometry_quant_step_lower): 16-bit number representing the lower 16 bits of the geometry quantization step 32-bit floating point number.

[0136] The quantization step size is: geometry_quant_step = (float)((geometry_quant_step_upper<<16)+geometry_quant_step_lower).

[0137] GeomRemoveDuplicateFlag: A binary variable. A value of '1' indicates that duplicate points (i.e., points with identical geometric positions) will be removed before geometry encoding; a value of '0' indicates that duplicate points will not be removed.

[0138] Attribute present flag (attribute_present_flag): A binary variable. A value of '1' indicates that the codestream contains attribute coding; a value of '0' indicates that the codestream does not contain attribute coding, that is, does not contain attribute headers, attribute slice headers, and attribute information.

[0139] Multiple attribute information enable flag (sps_multi_set_flag): A binary variable. A value of '1' indicates that support for multiple sets of attribute information for the same attribute data is enabled; a value of '0' indicates that support for multiple sets of attribute information for the same attribute data is disabled, and only one set of attribute information can be used for the same attribute data. When sps_multi_set_flag is not present in the codestream, its default value is zero.

[0140] MaxNumAttributesMinus1: Unsigned integer. Adding 1 indicates the maximum number of attribute types supported by this standard bitstream. The value range is [0, 127]. When maxNumAttributesMinus1 is not present in the bitstream, maxNumAttributesMinus1 defaults to -1.

[0141] Color quantization parameter (colorQuantParam): Unsigned integer. Indicates the color attribute quantization parameter.

[0142] Reflectivity quantization parameter (reflQuantParam): Unsigned integer. Represents the reflectivity attribute quantization parameter.

[0143] (2) Decoding instruction information set in the frame header

[0144] The point cloud code stream may include multiple point cloud frames, and the decoding indication information may be set in the frame header of the point cloud frame. The decoding indication information set in the frame header of the point cloud frame may be used to provide decoding indication for the point cloud frame corresponding to the frame header.

[0145] Before introducing the decoding indication information set in the frame header, the syntax elements of the general frame header are introduced here. The syntax elements of the general frame header are as shown in Table 3 below:

[0146] Table 3

[0147] The following explains the syntax elements in the general frame header shown in Table 3 above:

[0148] Frame number (frame_idx): This field indicates the sequence number of the point cloud frame in the point cloud sequence. The value range of the point cloud frame number is [0, 65535]. The sequence number of each point cloud frame is 1 plus the sequence number of the previous point cloud frame. The first point cloud frame number is 0. When the sequence number of a point cloud frame reaches 65535, the sequence number of the next point cloud frame is reset to 0.

[0149] The number of point cloud slices in the frame minus one (frame_num_slice_minus_one): The value of this field is the number of point cloud slices in the current point cloud frame minus one.

[0150] The upper 16 bits of the total number of points in the point cloud (geom_num_points_upper): a 16-bit value. It represents the upper 16 bits of a 32-bit unsigned integer representing the number of points in the point cloud after possible geometric quantization and deduplication.

[0151] Geom_num_points_lower: A 16-bit value representing the lower 16 bits of a 32-bit unsigned integer representing the number of points in the point cloud after possible geometric quantization and deduplication.

[0152] The number of points is: geom_num_points = (geom_num_points_upper) << 16 + geom_num_points_lower.

[0153] (3) Decoding instructions set in the attribute header

[0154] The point cloud code stream may include multiple point cloud frames, and the decoding indication information may be set in the attribute header of the point cloud frame. The decoding indication information set in the attribute header of the point cloud frame may be used to indicate attribute decoding of the point cloud frame.

[0155] Before introducing the decoding instruction information set in the attribute header, the syntax elements of the general attribute header are introduced here. The syntax elements of the general attribute header are as shown in Table 4 below:

[0156] Table 4

[0157] The following explains the syntax elements in the general attribute header shown in Table 4 above:

[0158] Attribute Present Flag field (attributePresentFlag[attrIdx]): A binary variable. A value of '1' indicates that the point cloud codestream contains the attribute code for attrIdx (i.e., the attribute code for the attribute type with the attribute index attrIdx); a value of '0' indicates that the point cloud codestream does not contain the attribute code for attrIdx. attrIdx represents the attribute index and is an integer between 0 and 15.

[0159] The meaning is explained in Table 5 below:

[0160] Table 5

[0161] Attribute data number minus one field (attribute_data_num_set_minus1[attrIdx]): An unsigned integer. Adding 1 indicates the number of attribute multi-datasets supported by the attribute identified by the attribute index attrIdx in the point cloud stream. This is a number between 0 and 127. When attribute_data_num_set_minus1[attrIdx] is not present in the point cloud, its default value is -1.

[0162] Attribute Information Enable Flag field (multi_set_flag[attrIdx]): A binary variable. A value of '1' indicates that the attribute identified by the attribute index attrIdx can support the use of multiple sets of attribute information; a value of '0' indicates that the attribute identified by the attribute index attrIdx can support the use of multiple sets of attribute information, and only one set of attribute information can be used for the attribute data. When multi_set_flag[attrIdx] is not present in the codestream, its default value is zero.

[0163] Attribute Information Number Minus One field (attribute_info_num_set_minus1[attrIdx]): An unsigned integer. 1 plus one indicates the number of attribute information supported by the attribute identified by the attribute index attrIdx in the point cloud codestream. This is a number between 0 and 127. When attribute_info_num_set_minus1[attrIdx] is not present in the codestream, its default value is -1.

[0164] Attribute Transformation Algorithm Flag field (transform): An unsigned integer between 0 and 2. This field controls the attribute encoding mode. A value of '0' specifies the attribute encoding mode to be attribute prediction mode; a value of '1' specifies the attribute encoding mode to be multi-layer transform mode; and a value of '2' specifies the attribute encoding mode to be attribute prediction transform mode.

[0165] Attribute transformation precision value (kFracBits): Unsigned integer. Used to indicate the shift precision of fixed-point operations during attribute transformation.

[0166] Attribute transform coefficient quantization parameter difference field (attrTransformQpDelta): Unsigned integer. Used to represent the difference from the attribute residual quantization parameter. Attribute transform coefficient quantization parameter attrTransformQp = attrQuantParam + attrTransformQpDelta.

[0167] Attribute Transformation Points (attrTransformNumPoints): An unsigned integer representing the number of attribute transformation points, i.e., the wavelet transform using attrTransformNumPoints points. AttrTransformNumPoints = 0 means that all points in the slice are used for the wavelet transform.

[0168] The maximum number of neighbors to search for, minus seven (maxNumOfNeighbour_log2_minus7) is an unsigned integer. This is used to derive the variable maxNumOfNeighbour, which represents the maximum number of encoded neighbors that can be searched. This controls the search range for candidate neighbor points and the number of points cached by the hardware during attribute prediction. maxNumOfNeighbour is calculated using the following formula: maxNumOfNeighbour = 2(maxNumOfNeighbor_log2_minus7 + 7), where maxNumOfNeighbour_log2_minus7 is an integer between 0 and 3.

[0169] Attribute residual secondary prediction field (cross_component_pred): A binary variable. A value of '1' indicates that attribute residual secondary prediction is allowed; a value of '0' indicates that attribute residual secondary prediction is not allowed.

[0170] Residual coding order switch field (orderSwitch): A binary variable. A value of '1' indicates that the residual coding order is UYV / GRB order; a value of '0' indicates that the residual coding order is YUV / RGB order.

[0171] Chroma channel Cb quantization parameter offset field (chromaQpOffsetCb): A signed integer. Used to control the Cb channel quantization parameter, with a value range of -16 to 16. If chromaQpOffsetCb does not exist in the attribute header, the value of chromaQpOffsetCb is 0. That is, chromaQpCb = Clip3(minQP, maxQP, attribute_qp + chromaQpOffsetCb). Note that the quantization parameter of the luma channel is lumaQp = attribute_qp. The minimum supported quantization parameter is minQP = 0, and the maximum supported quantization parameter is maxQP = 63.

[0172] Chroma channel chroma quantization parameter offset field (chromaQpOffsetCr): A signed integer. This field controls the chroma channel quantization parameter, with a value range of -16 to 16. If chromaQpOffsetCr does not exist in the attribute header, the value of chromaQpOffsetCr is 0. That is, choramQpCr = Clip3(minQP, maxQP, attribute_qp + chromaQpOffsetCr). Note that the quantization parameter for the luma channel is lumaQp = attribute_qp, with the minimum supported quantization parameter being minQP = 0 and the maximum supported quantization parameter being maxQP = 63.

[0173] Nearest neighbor prediction parameter 1 field (nearestPredParam1): Unsigned integer. Used to control the threshold of the nearest neighbor prediction.

[0174] Nearest Neighbor Prediction Parameter 2 (nearestPredParam2): An unsigned integer. This field controls the threshold for nearest neighbor prediction. The nearest neighbor prediction threshold is expressed as attrQuantParam*nearestPredParam1+nearestPredParam2.

[0175] Spatial bias coefficient field (axisBias): unsigned integer. Used to control the offset in the Z direction in the calculation of attribute prediction information.

[0176] Output Bit Depth Minus 1: An unsigned integer that controls the output bit depth. This value must be between 0 and 15. Output Bit Depth = outputBitDepthMinus1 + 1. If this syntax element is not present in the codestream, the default value is zero.

[0177] Color reordering mode field (colorReorderMode): Unsigned integer. Used to indicate the reordering mode currently selected for color information. '0' indicates the original point cloud input order, '1' indicates Hilbert reordering, and '2' indicates Morton reordering.

[0178] Reflectivity reordering mode field (reflReorderMode): Unsigned integer. Used to indicate the reordering mode currently selected for reflectivity information. '0' indicates the original point cloud input order, '1' indicates Hilbert reordering, and '2' indicates Morton reordering.

[0179] Maximum cache limit parameter field (log2maxNumofCoeffMinus8): unsigned integer between 0 and 16.

[0180] The maximum buffer limit parameter maxNumofCoeff for transform coefficients is calculated by the following formula: maxNumofCoeff = 1 << (log2maxNumofCoeffMinus8 + 8). When log2maxNumofCoeffMinus8 does not appear in the bitstream, maxNumofCoeff defaults to 1.

[0181] Maximum Latency Limit Parameter field (log2coeffLengthControlMinus8): An unsigned integer. Used to limit the maximum number of delays for transform parameters in attribute transform coding. The maximum delay point count is calculated as maxLatency = maxNumofCoeff * (1 << (log2coeffLengthControlMinus8 + 8)). The maximum value of maxLatency is 131072.

[0182] Attribute encoding order field (attrEncodeOrder): A binary variable. This field controls the order in which attributes are encoded when the point cloud contains multiple attribute types. A value of '0' indicates that color is encoded first, followed by reflectance; a value of '1' indicates that reflectance is encoded first, followed by color.

[0183] Adaptive Exponential Golomb Coding Sliding Window Size field (log2_golomb_group_size): Unsigned integer. Indicates the logarithm of the adaptive exponential Golomb coding sliding window size for attribute prediction residuals or transform coefficients. The adaptive exponential Golomb coding sliding window size is expressed as: golomb_sliding_window_size = 2log2_golomb_group_size.

[0184] Cross-type attribute prediction field (crossAttrTypePred): A binary variable. A value of '1' indicates that cross-type attribute prediction is allowed; a value of '0' indicates that cross-type attribute prediction is not allowed.

[0185] Cross-type attribute prediction weight parameter 1 (crossAttrTypePredParam1): 15-bit unsigned integer. Used to control the weight parameter 1 used to calculate the geometric information distance and attribute information distance in cross-type attribute prediction.

[0186] Cross-type attribute prediction weight parameter 2 (crossAttrTypePredParam2): 21-bit unsigned integer. Used to control the weight parameter 2 for calculating geometric information distance and attribute information distance in cross-type attribute prediction.

[0187] Reflectance group prediction flag field (refGroupPred): A binary variable used to control whether to enable the reflectance group prediction mode of the prediction transform. A value of '1' enables group prediction, and a value of '0' disables group prediction.

[0188] Color Initial Prediction Transform Ratio (colorInitPredTransRatio): A signed integer. Used in the multi-layer transform algorithm for attribute compression (transform=1), this controls the initial distance threshold used for color when building the prediction transform tree.

[0189] refInitPredTransRatio: Signed integer. Used in the multi-layer transform algorithm for attribute compression (transform=1), controls the size of the initial distance threshold used for reflectivity when building the prediction transform tree.

[0190] Transform Residual Layer Flag (transResLayer): A binary variable. Used in attribute compression multi-layer transform algorithms (transform = 1), this field controls whether attribute residual compensation is used. When transResLayer = 1, attribute residual compensation is used; when transResLayer = 0, attribute residual compensation is disabled.

[0191] Color Golomb order (ColorGolombNum): unsigned integer between 0 and 8. It is used to indicate the K-order Golomb order K = ColorGolombNum used when decoding the current color prediction residual or transform coefficient.

[0192] Reflectivity Index Golomb order (RefGolombNum): unsigned integer between 0 and 8. It is used to indicate the K-order Index Golomb order K = RefGolombNum used when decoding the current reflectivity prediction residual or transform coefficient.

[0193] ColorQPAdjustFlag: A binary variable. A value of '0' turns off the colorQPAdjustFlag; a value of '1' turns on the colorQPAdjustFlag.

[0194] Ratio of the number of points before and after point cloud geometry quantization (colorQPAdjustScalar): Unsigned integer. Used to calculate the distance threshold for the point cloud adaptive quantization tool.

[0195] Reflectance property prediction precision value (predFixedPointFracBit): Unsigned integer. Used to indicate the shift precision of fixed-point operation during reflectance property prediction. predFixedPointFracBit=0 means that fixed-point operation is not used.

[0196] Attribute value change statistics window size (log2_pred_dist_weight_group_size): Unsigned integer. Used to control the size of the attribute value change statistics window. The window size is calculated as N = 2log2_pred_dist_weight_group_size.

[0197] ColorMaxTransNum: An unsigned integer in the range [0,8]. This value indicates the maximum transformation order in the current color prediction transformation.

[0198] Maximum reflectivity transformation order (refMaxTransNum): Unsigned integer, range [0,8]. It is used to indicate the maximum transformation order in the current reflectivity prediction transformation.

[0199] Multi-attribute grouping parameter (multi_attr_group_id): unsigned integer array. When attribute_data_num_set_minus1[1] is greater than 0, it defines the grouping of multiple reflectivity attributes. Attributes in the same group need to be decoded as a group. multi_attri_group_id[i] indicates that the i-th attribute data belongs to the multi_attri_group_id[i]-th attribute group, and its value ranges from 0 to 127. The value of multi_attr_group_id[0] is 0, and the value of the next element in the array is equal to or greater than the value of the previous element. multi_attri_group_num is defined to represent the number of attribute data in the multi_attri_group_id[i]-th attribute group, which can be obtained by accumulating the number of consecutive identical multi_attri_group_id[i].

[0200] The above content introduces the syntax elements of the common attribute header of the point cloud frame.

[0201] (4) Decoding instruction information set in the attribute slice header

[0202] The decoding indication information may be set in the attribute slice header. The decoding indication information set in the attribute slice header may be used to perform attribute decoding indication on the point cloud slice.

[0203] Before introducing the decoding indication information set in the attribute slice header, the syntax elements of the general attribute slice header are introduced here. The syntax elements of the general attribute slice header are as shown in Table 6:

[0204] Table 6

[0205] The following explains the syntax elements in the general attribute slice header shown in Table 6 above:

[0206] Slice ID (slice_id): Unsigned integer. Indicates the slice ID.

[0207] Attribute Information Index (attributeID): Unsigned integer. Specifies the attribute information of the attribute ID of the corresponding type in the attribute header used by the attribute slice. This is a number between 0 and attribute_info_num_set_minus1[attrIdx]+1 (when the attribute slice is a color attribute slice, attIdx corresponds to 0, when the attribute slice is a reflectivity attribute slice, attIdx corresponds to 1). When this syntax element is not present in the bitstream, its default value is 0.

[0208] ReflQPoffset: A signed integer that controls the quantization parameter of the reflectivity attribute. The value range is [-32, 32].

[0209] In an embodiment of the present application, the first indication parameter is set in the parameter information at the sequence level, such as in the sequence header, for indicating the decoding of each point cloud frame in the point cloud code stream. The second indication parameter is set in the parameter information at the frame level or the slice level, such as in the frame header, for indicating the decoding of the corresponding point cloud frame (i.e., the point cloud frame corresponding to the frame header where the second indication parameter is located); or, if the second indication information is set in the slice header (such as the geometric slice header or the attribute slice header), for indicating the decoding of the corresponding point cloud slice (i.e., the point cloud frame corresponding to the slice header where the second indication parameter is located), if the slice header is a geometric slice header, the corresponding geometric slice is indicated by geometric decoding, and if the slice header is an attribute slice header, the corresponding attribute slice is indicated by attribute decoding.

[0210] The second indicator parameter is generally a parameter whose geometry or attributes change frame by frame or piece by piece. For example, since each point cloud frame is different, the corresponding bounding box related parameters are generally different; or, there are differences between the quantization parameters of each point cloud frame; or, there are differences between the color initial prediction transformation ratio (colorInitPredTransRatio) of each point cloud piece.

[0211] The first indication parameter is used to uniformly indicate decoding for each point cloud frame (e.g., each point cloud frame in a point cloud sequence), i.e., each point cloud frame can be decoded using the first indication parameter. The second indication parameter is used only to indicate decoding for the point cloud frame where the frame header is located or the point cloud slice where the slice header is located, i.e., each point cloud frame can be decoded according to the second indication parameter in the frame header, and each point cloud slice can be decoded according to the second indication parameter in the slice header.

[0212] In the present application, after obtaining the decoding indication information of the point cloud code stream, the first indication parameter located in the sequence header and / or the second indication parameter located in the frame header or slice header in the decoding indication information can be obtained.

[0213] During decoding, if the decoding indication information only includes the first indication parameter but not the second indication parameter, the first indication parameter of the sequence header can be selected to decode each point cloud frame in the point cloud code stream, thereby eliminating the need to calculate the parameters corresponding to each point cloud frame frame by frame, such as the bounding box-related parameters and quantization parameters corresponding to each point cloud frame, thereby reducing the amount of calculation required for decoding and improving decoding efficiency.

[0214] Alternatively, during decoding, if the decoding indication information only includes the second indication parameter but not the first indication parameter, the second indication parameter in the frame header of each point cloud frame can be selected to decode the corresponding point cloud frame, or the second indication parameter in the slice header of each point cloud slice can be selected to decode the corresponding point cloud slice. In order to avoid indication conflicts, in the same point cloud frame, the second indication parameter is generally defined only in the frame header or slice header, and not in both the frame header and the slice header. If the second indication parameter of each frame header or slice header includes bounding box related parameters, quantization parameters, etc., the corresponding point cloud frame can be decoded frame by frame or slice by slice through the second indication parameter, which can improve the decoding accuracy of the point cloud.

[0215] The bounding box related parameters may include a bounding box origin coordinate field and a bounding box size field.

[0216] The bounding box origin coordinate field includes: the upper 16-bit part of the bounding box origin x coordinate (bounding_box_offset_x_upper), the lower 16-bit part of the bounding box origin x coordinate (bounding_box_offset_x_lower), the bounding box origin x coordinate, the upper 16-bit part of the bounding box origin y coordinate (bounding_box_offset_y_upper), the lower 16-bit part of the bounding box origin y coordinate (bounding_box_offset_y_lower), the bounding box origin y coordinate, the upper 16-bit part of the bounding box origin z coordinate (bounding_box_offset_z_upper), the lower 16-bit part of the bounding box origin z coordinate (bounding_box_offset_z_lower), and the bounding box origin z coordinate;

[0217] The bounding box size field includes the upper 16-bit part of the bounding box width (bounding_box_size_width_upper), the lower 16-bit part of the bounding box width (bounding_box_size_width_lower), the bounding box width, the upper 16-bit part of the bounding box height (bounding_box_size_height_upper), the lower 16-bit part of the bounding box height (bounding_box_size_height_lower), the bounding box height, the upper 16-bit part of the bounding box depth (bounding_box_size_depth_upper), the lower 16-bit part of the bounding box depth (bounding_box_size_depth_lower), and the bounding box depth.

[0218] Quantization parameters may include geometric quantization parameters and attribute quantization parameters, etc.

[0219] For example, see Tables 7, 8, and 9 below. Table 7 is a syntax definition table for a sequence header provided in an embodiment of the present application, including first indication parameters, such as parameters related to a bounding box. Table 8 is a syntax definition table for a frame header provided in an embodiment of the present application, including second indication parameters, such as parameters related to a bounding box and a color initial prediction transformation ratio. Table 9 is a syntax definition table for a slice header (specifically, an attribute slice header) provided in an embodiment of the present application, including second indication parameters, such as a color initial prediction transformation ratio. It is easy to understand that Tables 7, 8, and 9 are not limited to complete syntax definition tables.

[0220] Table 7

[0221] Table 8

[0222] Table 9

[0223] The explanations of the fields in Table 7, Table 8, and Table 9 can be found in Table 2, Table 3, and Table 6, and are not repeated here.

[0224] The point cloud processing method of the present application obtains decoding indication information of a point cloud code stream and defines a first indication parameter at the sequence level and a second indication parameter at the frame level or slice level in the decoding indication information, wherein the first indication parameter is set in the sequence-level parameter information of the point cloud sequence, and the second indication parameter is set in the frame-level or slice-level parameter information of the point cloud sequence. Decoding is then performed based on the first indication parameter or the second indication parameter. When decoding based on the first indication parameter at the sequence level, each point cloud frame in the point cloud sequence is decoded using a unified first indication parameter decoding, eliminating the need to parse the corresponding indication parameter for each point cloud frame for decoding, resulting in higher decoding efficiency. When decoding according to the second indication parameter at the frame level or the slice level, frame level or slice level decoding can be achieved according to the second indication parameter, so that each point cloud frame or point cloud slice has an adapted second indication parameter, realizing frame-by-frame or slice-by-slice change of the second indication parameter. Compared with parsing the point cloud frame each time, it is also necessary to parse the sequence header to obtain the indication parameter corresponding to the point cloud frame for decoding, and the decoding efficiency is low. There is no need to go to the sequence header to parse the corresponding indication parameter for each point cloud frame, and the corresponding second indication parameter can be quickly obtained, and the decoding efficiency is relatively high.

[0225] In some embodiments, when the decoding indication information includes both the first indication parameter and the second indication parameter, a third indication parameter may be set in the sequence header, and the third indication parameter is used to indicate whether decoding is performed according to the first indication parameter or the second indication parameter.

[0226] Please continue to refer to Table 7 and Table 8. Taking the bounding box related parameters as an example, the third indicator parameter is the bounding box flag (global_bounding_box_flag). When global_bounding_box_flag is a first preset value (such as true), each point cloud frame in the point cloud sequence can be decoded according to the first indicator parameter. When global_bounding_box_flag is a second preset value (such as false), each point cloud frame in the point cloud sequence can be decoded according to the second indicator parameter.

[0227] In some embodiments, the first indicator parameter is determined based on the second indicator parameter of each point cloud frame.

[0228] For example, the first indication parameter is determined according to the maximum value of the second indication parameters of each point cloud frame, and the first indication parameter includes the maximum x-coordinate, y-coordinate, and z-coordinate of the bounding box, as well as the maximum width, maximum height, and maximum depth.

[0229] In some embodiments, the first indication parameter is a preset value.

[0230] For example, the preset value may be an empirical value, or a value agreed upon in advance by the encoding end and the decoding end.

[0231] In some embodiments, referring to FIG. 5 , the decoding indication information further includes a fourth indication parameter, which is used to indicate a grouping method of each point cloud frame. The point cloud processing method further includes:

[0232] Step 013: The decoding end determines a decoding method for each point cloud frame according to the fourth indication parameter, where the decoding method includes at least one of a direct prediction method and a prediction transformation method.

[0233] The decoding method refers to the method used by the decoding end for decoding, and mainly includes at least one of a direct prediction method and a predictive transformation method. Taking the decoded attribute bitstream as an example, the direct prediction method mainly obtains the residual value of each point in the current group by parsing, obtains the predicted value of each point, and thus determines the reconstruction attribute information of the current point; the predictive transformation method mainly obtains the transform coefficient of the current group by parsing, obtains the residual value of the current group after inverse transformation, obtains the predicted value of each point, and thus determines the reconstruction attribute information of the current point.

[0234] In some embodiments, different grouping methods require corresponding decoding methods to ensure decoding efficiency. In order for the decoding end to determine which decoding method to use, a decoding instruction is required to enable the decoding end to determine the specific decoding method.

[0235] The decoding indication information also includes a fourth indication parameter. By indicating the fourth indication parameter for grouping the point cloud frames, the decoding method corresponding to the fourth indication parameter can be determined.

[0236] In some embodiments, the fourth indication parameter can be set in a suitable position such as a sequence header, a frame header, an attribute header, a data bit stream (such as a geometric bit stream or an attribute bit stream), etc. There is no restriction here. It only needs to ensure that the decoding end can parse the grouping method and thus determine the corresponding decoding method.

[0237] In some embodiments, the fourth indication parameter is not set in the decoding indication information, the decoding mode of the decoding end is the default mode agreed upon with the encoding end, and the default mode is used to implement decoding.

[0238] In some embodiments, the grouping method includes at least one of the following methods:

[0239] (1) According to a preset grouping threshold, the points in each group are divided into multiple subgroups. When grouping, when the number of points in the previous subgroup reaches the preset grouping threshold, the next subgroup is grouped; that is, the preset grouping threshold is used to indicate the number of points included in a subgroup;

[0240] (2) If the current point being grouped is a duplicate point, the current point is used to complete the current subgroup until the number of points in the current subgroup reaches a preset grouping threshold, and then the remaining duplicate points are each divided into a subgroup; where a duplicate point is a point that is the same as any point in the group or any point in the first N points, where N is a positive integer;

[0241] (3) If the current point being grouped is a duplicate point, the current point is used to complete the current subgroup until the number of points in the current subgroup reaches the preset grouping threshold, and then the grouping of the remaining duplicate points is stopped;

[0242] (4) If the current point being grouped is a duplicate point, determine that the current sub-group has been completed, and continue to group the remaining points according to the preset grouping threshold;

[0243] (5) If the current point being grouped is a duplicate point, determine that the current subgroup has been grouped, and divide the remaining duplicate points into a subgroup respectively;

[0244] (6) When the current point being grouped is a duplicate point, it is determined that the current sub-group has been completed, and the grouping of the remaining duplicate points is stopped.

[0245] Specifically, when performing grouping, the decoder will first group based on the initial right-shift number, and right-shift the code corresponding to each point (such as the Hilbert code) according to the initial right-shift number to obtain the right-shifted value. If the code of the current point is a 10-bit binary value and the initial right-shift number is 3, the value formed by the remaining 7 bits can be obtained (such as a 10-bit binary value, a 7-bit value from right to left). Then, based on the obtained right-shifted value, points with the same value are determined to be in the same group, thereby obtaining multiple groups.

[0246] The decoding end then subdivides each group to obtain subgroups corresponding to each group. The above-mentioned multiple grouping methods are all methods for subdividing each group.

[0247] For method (1): the decoding end does not distinguish whether the current point is a duplicate point during grouping, and groups multiple points within the current group according to the preset grouping threshold. When grouping, after the number of points in the previous subgroup reaches the preset grouping threshold, the next subgroup is grouped. In other words, for each group, except for the last subgroup in which the number of points is less than or equal to the preset grouping threshold, the number of points in the other subgroups is the preset grouping threshold.

[0248] For example, for the color attribute, the preset grouping threshold may be the maximum transformation order colorMaxTransNum. For the reflectivity attribute, the preset grouping threshold may be the maximum transformation order refMaxTransNum.

[0249] A duplicate point is a point that is identical to any point in the current group or to any of the previous N points (e.g., the previous point). If the current point is a duplicate point, it means that there is a point identical to the current point in the current group, or there is a point identical to the current point in the N points before the current point. If the codes (e.g., geometry code or attribute code) of the two points are the same, then the two points are considered identical.

[0250] For method (2): the decoding end will consider whether the current point is a duplicate point when grouping. For non-duplicate points, the grouping method remains unchanged and is basically similar to method (1). In the case that the current point is a duplicate point, the current point will be used to complete the current subgroup first. After the current subgroup is filled to the preset grouping threshold, each duplicate point will be divided into a separate subgroup when the duplicate points are grouped. For example, if the current group has 10 points and the preset grouping threshold is 5, after the first subgroup is completed, there are still 2 duplicate points in the remaining 5 points. In this case, when the duplicate points are grouped, the remaining 2 duplicate points will be divided into 2 subgroups.

[0251] For method (3): the decoder will consider whether the current point is a duplicate point when grouping. For non-duplicate points, the grouping method remains unchanged and is basically similar to method (1). If the current point is a duplicate point, the current point will be used to complete the current subgroup. After the current subgroup is filled to the preset grouping threshold, the duplicate points will not be grouped again. For example, if the current group has 10 points and the preset grouping threshold is 5, after the first subgroup is completed, there are still 2 duplicate points in the remaining 5 points. In this case, the 2 duplicate points will not be grouped again.

[0252] For method (4): the decoding end will consider whether the current point is a repeated point when grouping. For non-repeated points, the grouping method remains unchanged and is basically similar to method (1). In the case that the current point is a repeated point, the current point will not be used to complete the current sub-group. Instead, it will be determined that the current sub-group has been grouped. Regardless of whether the current sub-group reaches the preset grouping threshold, the grouping of the next sub-group will continue, and the current point will be placed in the next sub-group. If the next point is still a repeated point, the next sub-group will be determined to be grouped. Then, the grouping process of the next sub-group of the next sub-group will continue, and this cycle will be repeated until all the current groups are grouped.

[0253] For method (5): the decoding end will consider whether the current point is a repeated point when grouping. For non-repeated points, the grouping method remains unchanged and is basically similar to method (1). In the case that the current point is a repeated point, the current point will not be used to complete the current sub-group. Instead, it is determined that the current sub-group has been grouped. Regardless of whether the current sub-group reaches the preset grouping threshold, the grouping of the next sub-group will be continued. When grouping repeated points in the future, each repeated point will be divided into a separate sub-group.

[0254] For method (6): the decoding end will consider whether the current point is a repeated point when grouping. For non-repeated points, the grouping method remains unchanged and is basically similar to method (1). In the case that the current point is a repeated point, the current point will not be used to complete the current sub-group. Instead, it is determined that the current sub-group has been grouped. Regardless of whether the current sub-group reaches the preset grouping threshold, the grouping of the next sub-group will continue, and in subsequent groupings, the remaining repeated points will not be grouped.

[0255] For different grouping modes, the fourth indication parameter can be used to perform decoding indication by setting different values. For example, the fourth indication parameter is any integer from 1 to 6, and the fourth indication parameters from 1 to 6 correspond to grouping modes (1) to (6) respectively, that is, the fourth indication parameter is 1, indicating that the grouping mode is mode (1).

[0256] In some embodiments, when the current subpacket to be decoded contains repeated points, the decoding end determines a decoding method for the current subpacket according to a fourth indication parameter.

[0257] Because the grouping mode is set for repeated points, during decoding, the decoding end can also determine the decoding mode corresponding to the sub-group containing repeated points only. When it is determined that the current sub-group to be decoded contains repeated points, the decoding mode for decoding the current sub-group is determined based on the grouping mode obtained by parsing the fourth indication parameter.

[0258] In some embodiments, referring to FIG6 , grouping is required during both encoding and decoding. The point cloud processing method includes:

[0259] Step 014: The decoding end determines whether to perform dynamic grouping based on the relevant parameters of the quantization parameter of the point cloud frame or the maximum number of points in each subgroup;

[0260] Step 015: If yes, the decoder adjusts the current right shift bit according to the average number of points in the first M consecutive subgroups, where M is a positive integer;

[0261] Step 016: The decoding end determines the target number of bits of the code corresponding to the points to be grouped according to the adjusted current right shift bit number;

[0262] Step 017: The decoding end regroups the values ​​according to the target number of bits.

[0263] Specifically, when grouping, the decoding end of the present application will perform dynamic grouping judgment to prevent the number of points in each group after grouping from not meeting expectations and affecting the encoding and decoding performance.

[0264] Therefore, when grouping, the decoding end will determine whether to perform dynamic grouping based on the relevant parameters of the quantization parameters of the point cloud frame or the maximum number of points in each sub-group.

[0265] For example, the relevant parameters of the quantization parameters of the point cloud frame may include an initial offset shift. When the initial offset is greater than 0, it is determined that dynamic grouping is to be performed; when the initial offset is less than 0, it is determined that dynamic grouping is not to be performed; or, when the initial offset is greater than 0, it is determined that dynamic grouping is not to be performed; when the initial offset is greater than 0, it is determined that dynamic grouping is to be performed.

[0266] For another example, the decoding end counts the maximum number of points in each subgroup in the point cloud frame and determines whether the maximum number is greater than a preset number threshold. If it is greater than the preset threshold, dynamic grouping is determined to be performed; if it is less than or equal to the preset threshold, dynamic grouping is not determined to be performed. Alternatively, the decoding end determines whether the maximum number is less than a preset number threshold. If it is less than the preset threshold, dynamic grouping is determined to be performed; if it is greater than or equal to the preset threshold, dynamic grouping is not determined to be performed.

[0267] In this way, when the grouping does not meet expectations, the decoding end performs dynamic grouping to make the grouping meet expectations as much as possible.

[0268] When performing dynamic grouping, the decoding end will obtain the average value of the number of points of the first M consecutive sub-groups (such as the first 8, 10, 15, etc.), and use the average value to determine whether to adjust the current right-shift bit number for grouping. If the average value is less than 2, the current right-shift bit number will be increased by 1. If the average value is greater than 8, the current right-shift bit number will be reduced by 1. Then, based on the adjusted current right-shift bit number, the grouped points (such as the points in the point cloud frame) are grouped again. Specifically, the target number of bits after the right shift of each point can be determined first (for example, if the code is a 10-bit binary number and the current right-shift bit number is 3, the remaining seven bits after the right shift of 3 bits are obtained as the target number of bits), and the points with the same value formed by the target number of bits are divided into the same group to obtain multiple groups with a more reasonable number of points.

[0269] In some embodiments, the point cloud processing method further includes:

[0270] Step 018: When the current point to be decoded is a repeated point, the decoding end determines that the predicted values ​​of each attribute type of the current point to be decoded are respectively the reconstructed values ​​of each attribute type of the previous point of the current point to be decoded, and determines that the current residual value of the attribute type of the current point to be decoded is non-negative or non-positive.

[0271] Specifically, for the decoding of repeated points, the decoding end can determine the predicted values ​​of each attribute type of the current point to be decoded based on the reconstructed values ​​of each attribute type of the previous point of the current point to be decoded when determining that the current point to be decoded is a repeated point. For example, the reconstructed values ​​of each attribute type of the previous point of the current point to be decoded are used as the predicted values ​​of each corresponding attribute type of the current point to be decoded, thereby quickly decoding to obtain the predicted value of the current point to be decoded.

[0272] Among them, for the reflectivity attribute, when the current point to be decoded is a repeated point, the decoding end determines that the predicted value of the reflectivity attribute of the current point to be decoded is the reconstructed value of the reflectivity attribute of the previous point of the current point to be decoded, and determines that the sign bit of the current residual value of the reflectivity attribute of the current point to be decoded is non-negative or non-positive.

[0273] For color attributes, when the current point to be decoded is a repeated point, the decoding end determines that the predicted value of the Kth component of the color attribute of the current point to be decoded is the reconstructed value of the Kth component of the point before the current point to be decoded, and determines that the Kth component of the current residual value of the color attribute of the current point to be decoded is non-negative or non-positive, and K is any integer from 1 to 3; or when the current point to be decoded is a repeated point, the predicted values ​​of each component of the color attribute of the current point to be decoded are determined to be the reconstructed values ​​of each component of the point before the current point to be decoded, and determines that each component of the current residual value of the color attribute of the current point to be decoded is non-negative or non-positive.

[0274] During encoding, the residual values ​​of repeated points are processed accordingly so that the decoding process does not need to identify the sign bits of the repeated points. The processing of the residual values ​​of repeated points during encoding includes one of the following methods:

[0275] (1) When the residual value of the duplicate point is negative, the residual value of the duplicate point is set to 0;

[0276] (2) For the color attribute, when the Kth component of the residual value of the repeated point is negative, the Kth component is set to 0;

[0277] (3) For the color attribute, when any component of the residual value of the repeated point is negative, the component is set to 0.

[0278] At this time, for case (1), for the color attribute or reflectivity attribute, when the decoding end decodes the point to be decoded that is a repeated point, it is not necessary to identify the sign bit, and the residual value of the point to be decoded can be directly determined as a non-negative number for decoding. For case (2), for the color attribute, when the decoding end decodes the color attribute of the point to be decoded that is a repeated point, it is not necessary to identify the sign bit of the Kth component of the residual value, and the Kth component of the residual value of the point to be decoded can be directly determined as a non-negative number for decoding. For case (3), for the color attribute, when the decoding end decodes the color attribute of the point to be decoded that is a repeated point, it is not necessary to identify the sign bit of any component of the residual value, and each component of the residual value of the point to be decoded can be directly determined as a non-negative number for decoding.

[0279] The processing of the residual value of the repeated point during encoding may also include one of the following methods:

[0280] (1) When the residual value of the duplicate point is positive, the residual value of the duplicate point is set to 0;

[0281] (2) For the color attribute, when the Kth component of the residual value of the repeated point is positive, the Kth component is set to 0;

[0282] (3) For the color attribute, when any component of the residual value of the repeated point is positive, the component is set to 0.

[0283] At this time, for case (1), for the color attribute or reflectivity attribute, when the decoding end decodes the point to be decoded that is a repeated point, it is not necessary to identify the sign bit, and the residual value of the point to be decoded can be directly determined as a non-positive number for decoding. For case (2), for the color attribute, when the decoding end decodes the color attribute of the point to be decoded that is a repeated point, it is not necessary to identify the sign bit of the Kth component of the residual value, and the Kth component of the residual value of the point to be decoded can be directly determined as a non-positive number for decoding. For case (3), for the color attribute, when the decoding end decodes the color attribute of the point to be decoded that is a repeated point, it is not necessary to identify the sign bit of any component of the residual value, and each component of the residual value of the point to be decoded can be directly determined as a non-positive number for decoding.

[0284] As shown in the examples of Tables 10 and 11 below, the method for processing the residual values ​​of the color attributes of the repeated points can be defined in the color attribute data:

[0285] Table 10

[0286] Table 11

[0287] The following is a semantic description of the syntax elements in Table 10 and Table 11:

[0288] Color first component residual value zero flag (color_first_comp_zero): A binary variable. A value of '1' indicates that the color first component residual value is zero; a value of '0' indicates that the color first component residual value is not zero. When orderSwitch = 0, the color first component is Y (if the color space is YUV) or R (if the color space is RGB); otherwise, the color first component is U (if the color space is YUV) or G (if the color space is RGB).

[0289] Color second component residual value zero flag (color_second_comp_zero): A binary variable. A value of '1' indicates that the color second component residual value is 0; a value of '0' indicates that the color second component residual value is not 0. When orderSwitch = 0, the color second component is U (if the color space is YUV) or G (if the color space is RGB); otherwise, the color second component is Y (if the color space is YUV) or R (if the color space is RGB).

[0290] Color residual component (color_component[k]): A binary variable. A value of '1' indicates a positive residual of the first color component; a value of '0' indicates a negative residual of the first color component. k = 0, 1, 2.

[0291] Color residual component sign (color_component_sign[k]): A binary variable. A value of '1' indicates a positive residual value for the first color component; a value of '0' indicates a negative residual value for the first color component. k = 0, 1, 2.

[0292] Residual encoding order switch (orderSwitch): A binary variable. A value of '1' indicates that the residual encoding order is UYV / GRB order; a value of '0' indicates that the residual encoding order is YUV / RGB order.

[0293] Duplicate point flag field (isDuplicatePoint): indicates whether the current point is a duplicate point. A value of "true" indicates that the current point is a duplicate point, and a value of "false" indicates that the current point is not a duplicate point.

[0294] In some embodiments, the color attribute is represented by an RGB encoding format or a YUV encoding format, where the RGB encoding format includes three components R, G, and B, or the YUV encoding format includes three components Y, U, and V, and the Kth component is any one of the three components R, G, and B, or any one of the three components Y, U, and V.

[0295] Specifically, RGB and YUV are both color encoding formats. The R component in the RGB encoding format represents the red component, the G component represents the green component, and the B component represents the blue component. The Y component in the YUV encoding format represents the lightness component, and the U and V components represent the chrominance.

[0296] When the color attribute is represented by the RGB encoding format, the Kth component is any one of the three components R, G, and B; when the color attribute is represented by the YUV encoding format, the Kth component is any one of the three components Y, U, and V, thereby quickly determining the Kth component of the residual value of the point to be decoded.

[0297] In some embodiments, the decoding indication information also includes a fifth indication parameter, which is set in the sequence header. The value of the fifth indication parameter includes any one of multiple preset values. When the value of the fifth indication parameter is different preset values, the corresponding decoding frame rate is different.

[0298] Specifically, during actual decoding, the decoder generally only supports one or more decoding frame rates. Therefore, to provide decoding instructions, a fifth indication parameter can be set in the sequence header to indicate the decoding frame rate. Different preset values ​​of the fifth indication parameter correspond to one of the multiple decoding frame rates supported by the decoder. Table 12 below shows a sequence header with the fifth indication parameter set.

[0299] Table 12

[0300] The fifth indication parameter is frameRateCode, which is a 4-bit unsigned integer and is used to specify the frame rate.

[0301] The mapping relationship between the frame rate and the value of the fifth indicator parameter is shown in Table 13 below:

[0302] Table 13

[0303] It can be understood that the above Table 13 is only one example of the mapping relationship between the value of the fifth indication parameter and the frame rate, and is not intended to limit the value of the fifth indication parameter.

[0304] In some embodiments, the attribute header includes an attribute quantization parameter field, where the attribute quantization parameter field is used to control the attribute quantization parameter of the point cloud, and the attribute quantization parameter is used to perform attribute decoding instructions on the point cloud frame regarding attribute quantization.

[0305] Specifically, by setting the attribute quantization parameters in the attribute header, the attribute quantization parameters of each attribute type can be defined. The specific settings are shown in Table 14 below, which defines the attribute header of the attribute quantization parameters:

[0306] Table 14

[0307] The attribute quantization parameter field is attributeQuantParam[attrIdx]. For details of other syntax elements in the above table, please refer to the description of Table 4, which will not be repeated here.

[0308] In some embodiments, the attribute header includes multiple attribute types, each attribute type has an attribute quantization parameter field, the attribute quantization parameter field under the target attribute type is used to control the attribute quantization parameters of the point cloud under the target attribute type, and the attribute quantization parameters under the target attribute type are used to perform attribute decoding instructions on the point cloud frame regarding the attribute quantization under the target attribute type. The target attribute type is any one of the multiple attribute types.

[0309] Specifically, when there are multiple attribute types, the attribute quantization parameters corresponding to each attribute type can be defined separately. For example, Table 15 below shows the attribute header that defines the attribute quantization parameters corresponding to each attribute type:

[0310] Table 15

[0311] Among them, the attribute quantization parameter is defined in a loop statement based on the attribute information number minus one field (attribute_info_num_set_minus1[attrIdx]). Different attribute quantization parameters can be set for each attribute type, and the attribute quantization parameters corresponding to each attribute type are parsed through the loop statement.

[0312] In addition to the definition of attribute quantization parameters, the corresponding attribute quantization parameter offset can also be defined for each attribute type. The following Table 16 or 17 sets the attribute header for the attribute quantization parameter offset corresponding to each attribute type:

[0313] Table 16:

[0314] Table 16 defines the corresponding attribute quantization parameter offset (attributeQuantParamOffset[attrIdx][i]) for each attribute type.

[0315] Table 17:

[0316] Table 17 defines the corresponding color attribute quantization parameter offset (colorQuantParamOffset[i]) and reflectivity attribute quantization parameter offset (reflQuantParamOffset[i]) for each attribute type, respectively, for the color attribute and the reflectivity attribute.

[0317] It can be understood that the explanation of the syntax elements in Table 16 and Table 17 can be found in the description of Table 4 and will not be repeated here.

[0318] In some embodiments, when the target attribute type is a reflectance attribute, the attribute quantization parameter field under the target attribute type includes at least one of a reflectance attribute quantization parameter and a reflectance attribute quantization parameter offset; when the target attribute type is a color attribute, the attribute quantization parameter field under the target attribute type includes any one of the following: a color attribute quantization parameter, an attribute quantization parameter offset of each color component.

[0319] In some embodiments, the slice header includes an attribute slice header, and the decoding indication information further includes an attribute quantization parameter field set in the attribute slice header.

[0320] Specifically, when defining the attribute quantization parameter field, in addition to defining it in the attribute header, the attribute quantization parameter field can also be defined in the attribute slice header, so that each attribute slice has a corresponding attribute quantization parameter. During decoding, the attribute quantization parameter corresponding to each attribute slice is parsed to achieve slice-level attribute quantization control.

[0321] Table 18 below defines the attribute slice header that defines the attribute quantization parameter field:

[0322] Table 18

[0323] Wherein, reflQPoffset is the reflectivity attribute quantization parameter offset. It is understood that when the attribute type of the attribute quantization parameter field is the reflectivity attribute, the attribute quantization parameter field may include at least one of the reflectivity attribute quantization parameter and the reflectivity attribute quantization parameter offset; when the attribute type of the attribute quantization parameter field is the color attribute, the attribute quantization parameter field may include any of the following: the color attribute quantization parameter, and the attribute quantization parameter offsets of each color component.

[0324] In some embodiments, the decoding indication information includes a loop statement set in the point cloud sequence, the point cloud frame includes a point cloud slice, the point cloud slice includes an attribute slice, the attribute slice is located in the loop statement, the attribute slice includes an attribute slice header and an attribute data bit stream, and the loop statement traverses each attribute slice through the attribute slice header start code.

[0325] Specifically, in order to realize the traversal and parsing of multiple attribute slices, the decoding instruction information may also include a loop statement set in the point cloud sequence. By setting the attribute slice in the loop statement, the loop statement then traverses and parses each attribute slice in turn through the attribute slice header start code to ensure that all attribute slices can be parsed.

[0326] Table 19 below shows the point cloud sequence that defines the loop statement:

[0327] Table 19

[0328] Among them, the loop statement is such as the loop where the attribute slice (attribute_slice()) in Table 19 is located, and the attribute slice header start code (attribute_slice_header_start_code) is looped through to traverse each attribute slice.

[0329] In some embodiments, the point cloud frame includes a point cloud slice, the point cloud slice includes an attribute slice, the decoding instruction information includes a loop statement set on the attribute slice, and the attribute slice includes an attribute slice header and an attribute data bit stream;

[0330] The attribute slice header and the attribute data bit stream are both located in a loop statement. The loop statement traverses each attribute slice header and attribute data bit stream through the attribute slice header start code. The attribute slice header and the attribute data bit stream correspond one to one.

[0331] Specifically, it is also possible to define a loop statement within the attribute slice, define the attribute slice header and attribute data bit stream of the attribute slice into the loop statement, and make the loop statement traverse each attribute slice header and attribute data bit stream through the attribute slice header start code, thereby realizing the traversal and parsing of the attribute slice header and attribute data bit stream of each attribute slice.

[0332] Table 20 below shows the attribute slice that defines the loop statement:

[0333] Table 20

[0334] Among them, the loop statement where the attribute slice header (attribute_slice_header()) and the attribute data bit stream (general_attribute_data_bitstream()) are located is traversed and parsed according to the attribute slice header start code.

[0335] Alternatively, the attribute data bit stream is placed in the loop statement and the attribute slice header is placed outside the loop statement, so that the attribute slice header can correspond to one or more attribute data bit streams, and the loop statement traverses each attribute data bit stream through the attribute data start code.

[0336] Table 21 below defines the attribute slice of the loop statement:

[0337] Table 21

[0338] Among them, the attribute slice header (attribute_slice_header()) is located outside the loop statement, and the attribute data bit stream (general_attribute_data_bitstream()) is located inside the loop statement. It is traversed according to the attribute data start code to parse one or more attribute data bit streams corresponding to the attribute slice header.

[0339] According to the method described in the above embodiment, the embodiment of the present application further provides another point cloud processing method. Please refer to Figure 7, which is a flow chart of another point cloud processing method provided by the embodiment of the present application. The point cloud processing method can be applied to an encoding device. Optionally, the encoding device can be a content production device, specifically including:

[0340] Step 021: The encoder obtains point cloud data;

[0341] Step 022: The encoding end encodes and compresses the point cloud data to obtain a point cloud code stream. The decoding indication information of the point cloud code stream includes at least one of a first indication parameter and a second indication parameter. The first indication parameter is set in the sequence-level parameter information of the point cloud sequence, and the second indication parameter is set in the frame-level or slice-level parameter information of the point cloud sequence. The first indication parameter is used for sequence-level decoding indication, and the second indication parameter is used for frame-level or slice-level decoding indication. The point cloud code stream is decoded according to the first indication parameter or the second indication parameter.

[0342] In some embodiments, the sequence header further includes a third indicator parameter. When the value of the third indicator parameter is a first preset value, each point cloud frame in the point cloud sequence is encoded according to the first indicator parameter. When the value of the third indicator parameter is a second preset value, each point cloud frame is encoded according to the second indicator parameter of each point cloud frame, and the first preset value and the second preset value are different.

[0343] In some embodiments, the first indication parameter and the second indication parameter each include at least one of a bounding box origin coordinate field, a bounding box size field, and a quantization parameter field.

[0344] In some embodiments, the first indication parameter is determined according to the second indication parameter of each point cloud frame; or, the first indication parameter is a preset value.

[0345] In some embodiments, the decoding indication information further includes a fourth indication parameter, and the fourth indication parameter is used to indicate a grouping method of each point cloud frame.

[0346] In some embodiments, the grouping method includes at least one of the following methods:

[0347] According to the preset grouping threshold, the points in each group are divided into multiple sub-groups. When grouping, when the number of points in the previous sub-group reaches the preset grouping threshold, the next sub-group is grouped;

[0348] If the current point being grouped is a duplicate point, the current point is used to complete the current subgroup until the number of points in the current subgroup reaches the preset grouping threshold, and then the remaining duplicate points are each divided into a subgroup; where a duplicate point is a point that is the same as any point in the group or any point in the first N points, where N is a positive integer;

[0349] If the current point being grouped is a duplicate point, the current point is used to complete the current sub-group until the number of points in the current sub-group reaches the preset grouping threshold, and then the remaining duplicate points are stopped from being grouped.

[0350] When the current point being grouped is a duplicate point, it is determined that the current sub-group has been completed, and the remaining points are grouped according to the preset grouping threshold;

[0351] When the current point being grouped is a duplicate point, determining that the current subgroup has been grouped, and dividing the remaining duplicate points into a subgroup respectively;

[0352] When the current point being grouped is a duplicate point, it is determined that the current sub-group has been completed, and grouping of the remaining duplicate points is stopped.

[0353] In some embodiments, whether to perform dynamic grouping is determined based on relevant parameters of the quantization parameters of the point cloud frame or the maximum number of points in each subgroup; if so, the current right shift bit number is adjusted based on the average number of points in the first M consecutive subgroups, where M is a positive integer; based on the adjusted current right shift bit number, the target number of encoding bits corresponding to the grouped points is determined; and re-grouping is performed based on the value within the target number of bits.

[0354] In some embodiments, the sequence header further includes a fifth indication parameter, and the value of the fifth indication parameter includes any one of a plurality of preset values. When the value of the fifth indication parameter is different preset values, the corresponding encoding frame rates are different.

[0355] In some embodiments, the decoding indication information also includes an attribute header set in the point cloud frame, the attribute header includes an attribute quantization parameter field, the attribute quantization parameter field is used to control the attribute quantization parameters of the point cloud, and the attribute quantization parameters are used to perform attribute encoding instructions on attribute quantization of the point cloud frame.

[0356] In some embodiments, the attribute header includes multiple attribute types, each attribute type has an attribute quantization parameter field, the attribute quantization parameter field under the target attribute type is used to control the attribute quantization parameters of the point cloud under the target attribute type, and the attribute quantization parameters under the target attribute type are used to indicate attribute encoding of the point cloud frame regarding attribute quantization under the target attribute type. The target attribute type is any one of multiple attribute types.

[0357] In some embodiments, the slice header includes an attribute slice header, and the decoding indication information further includes an attribute quantization parameter field set in the attribute slice header.

[0358] In some embodiments, when the attribute type of the attribute quantization parameter field is a reflectance attribute, the attribute quantization parameter field includes at least one of a reflectance attribute quantization parameter and a reflectance attribute quantization parameter offset; when the attribute type of the attribute quantization parameter field is a color attribute, the attribute quantization parameter field includes any one of the following: a color attribute quantization parameter, an attribute quantization parameter offset of each color component.

[0359] In some embodiments, the decoding indication information includes a loop statement set in the point cloud sequence, the point cloud frame includes point cloud slices, the point cloud slices include attribute slices, the attribute slices are located in the loop statement, and the loop statement traverses each attribute slice through the attribute slice header start code.

[0360] In some embodiments, the point cloud slice includes an attribute slice, the decoding instruction information includes a loop statement set on the attribute slice, and the attribute slice includes an attribute slice header and an attribute data bit stream;

[0361] The attribute slice header and the attribute data bit stream are both located in a loop statement, and the loop statement traverses each attribute slice header and attribute data bit stream through the attribute slice header start code, and the attribute slice header and the attribute data bit stream correspond one to one; or,

[0362] The attribute data bit stream is located in a loop statement. The loop statement traverses each attribute data bit stream through the attribute data start code. The attribute slice header corresponds to one or more attribute data bit streams.

[0363] It should be pointed out that the encoding and decoding processes of the point cloud are reversible. Therefore, the syntax definitions involved in the point cloud processing method applied to the content production device provided in the embodiment of the present application can be referred to the above-mentioned embodiment of the point cloud processing method applied to the content presentation device, and will not be repeated here.

[0364] Based on the method described in the above embodiment, the present application also provides a point cloud processing device for executing the steps of the above-mentioned point cloud processing method applied to a content presentation device. Please refer to Figure 8, which is a schematic diagram of the structure of the point cloud processing device provided in the present application. The point cloud processing device 300 is applied to a decoding device, which can optionally be a content presentation device, and includes a first acquisition module 301 and a decoding module 302, wherein:

[0365] A first acquisition module 301 is configured to acquire decoding instruction information of a point cloud code stream, where the decoding instruction information includes at least one of a first instruction parameter and a second instruction parameter. The first instruction parameter is set in the sequence-level parameter information of the point cloud sequence, and the second instruction parameter is set in the frame-level or slice-level parameter information of the point cloud sequence. The first instruction parameter is used for sequence-level decoding instruction, and the second instruction parameter is used for frame-level or slice-level decoding instruction.

[0366] The decoding module 302 is configured to decode the point cloud code stream according to the first indication parameter or the second indication parameter.

[0367] In some embodiments, the sequence-level parameter information includes a sequence header, and the sequence header also includes a third indication parameter, and the third indication parameter is used to indicate decoding according to the first indication parameter or the second indication parameter; the decoding module 302 is specifically used to decode each point cloud frame of the point cloud sequence according to the first indication parameter when the value of the third indication parameter is a first preset value; when the value of the third indication parameter is a second preset value, decode each point cloud frame according to the second indication parameter of each point cloud frame, and the first preset value and the second preset value are different.

[0368] In some embodiments, the first indication parameter and the second indication parameter each include at least one of a bounding box origin coordinate field, the bounding box size field, and a quantization parameter field.

[0369] In some embodiments, the first indication parameter is determined according to the second indication parameter of each point cloud frame of the point cloud sequence; or, the first indication parameter is a preset value.

[0370] In some embodiments, the decoding indication information also includes a fourth indication parameter, which is used to indicate the grouping method of each point cloud frame in the point cloud sequence. The decoding module 302 is also used to determine the decoding method of each point cloud frame based on the fourth indication parameter, and the decoding method includes at least one of a direct prediction method and a prediction transformation method.

[0371] In some embodiments, the grouping method includes at least one of the following methods:

[0372] According to a preset grouping threshold, the points in each group are divided into multiple subgroups. When grouping, after the number of points in the previous subgroup reaches the preset grouping threshold, the next subgroup is grouped;

[0373] If the current point being grouped is a duplicate point, the current point is used to complete the current subgroup until the number of points in the current subgroup reaches the preset grouping threshold, and then the remaining duplicate points are each divided into one of the subgroups; wherein the duplicate point is a point that is the same as any point in the group or any point in the first N points, where N is a positive integer;

[0374] When the current point being grouped is a duplicate point, the current point is used to complete the current subgroup until the number of points in the current subgroup reaches the preset grouping threshold, and then the grouping of the remaining duplicate points is stopped;

[0375] When the current point being grouped is a duplicate point, determining that the current sub-group has been grouped, and continuing to group the remaining points according to the preset grouping threshold;

[0376] In the case that the current point being grouped is a repeated point, determining that the current subgroup has been grouped, and dividing the remaining repeated points into one of the subgroups;

[0377] When the current point being grouped is a duplicate point, it is determined that the current sub-group has been completed, and grouping of the remaining duplicate points is stopped.

[0378] In some embodiments, the decoding module 302 is specifically configured to determine a decoding method for the current subpacket according to the fourth indication parameter when the current subpacket to be decoded contains repeated points.

[0379] In some embodiments, the decoding module 302 is also used to determine whether to perform dynamic grouping based on relevant parameters of the quantization parameters of the point cloud frame or the maximum number of points in each of the sub-groups; if so, adjust the current right shift bit number based on the average number of points in the first M consecutive sub-groups, where M is a positive integer; determine the target number of encoding bits corresponding to the grouped points based on the adjusted current right shift bit number; and re-group based on the numerical value within the target number of bits.

[0380] In some embodiments, the decoding module 302 is further used to determine, when the current point to be decoded is a repeated point, that the predicted values ​​of each attribute type of the current point to be decoded are respectively the reconstructed values ​​of each attribute type of the previous point of the current point to be decoded, and to determine that the current residual value of the attribute type of the current point to be decoded is non-negative or non-positive.

[0381] In some embodiments, the attribute type includes a color attribute, and the decoding module 302 is further used to determine, when the current point to be decoded is a repeated point, that the predicted value of the Kth component of the color attribute of the current point to be decoded is the reconstructed value of the Kth component of the previous point of the current point to be decoded, and to determine that the Kth component of the current residual value of the color attribute of the current point to be decoded is a non-negative number or a non-positive number, where K is any integer from 1 to 3; or, when the current point to be decoded is a repeated point, to determine that the predicted values ​​of each component of the color attribute of the current point to be decoded are respectively the reconstructed values ​​of each component of the previous point of the current point to be decoded, and to determine that each component of the current residual value of the color attribute of the current point to be decoded is a non-negative number or a non-positive number.

[0382] In some embodiments, the color attribute is represented by an RGB encoding format or a YUV encoding format, the RGB encoding format includes three components R, G, and B, or the YUV encoding format includes three components Y, U, and V, and the Kth component is any one of the three components R, G, and B, or any one of the three components Y, U, and V.

[0383] In some embodiments, the decoding module 302 is also used to determine that the predicted value of the reflectivity attribute of the current point to be decoded is a reconstructed value of the reflectivity attribute of the previous point of the current point to be decoded when the current point to be decoded is a repeated point, and to determine that the sign bit of the current residual value of the reflectivity attribute of the current point to be decoded is a non-negative number or a non-positive number.

[0384] In some embodiments, the sequence-level parameter information includes a sequence header, and the decoding indication information also includes a fifth indication parameter, the fifth indication parameter is used to indicate the decoding frame rate, the fifth indication parameter is set in the sequence header, and the value of the fifth indication parameter includes any one of a plurality of preset values. When the value of the fifth indication parameter is different from the preset value, the corresponding decoding frame rate is different.

[0385] In some embodiments, the decoding indication information also includes an attribute header set in the point cloud frame, the attribute header includes an attribute quantization parameter field, the attribute quantization parameter field is used to control the attribute quantization parameters of the point cloud, and the attribute quantization parameters are used to perform attribute decoding instructions on the point cloud frame regarding attribute quantization.

[0386] In some embodiments, the attribute header includes multiple attribute types, each of the attribute types has an attribute quantization parameter field, the attribute quantization parameter field under the target attribute type is used to control the attribute quantization parameters of the point cloud under the target attribute type, the attribute quantization parameters under the target attribute type are used to perform attribute decoding instructions on the point cloud frame regarding the attribute quantization under the target attribute type, and the target attribute type is any one of the multiple attribute types.

[0387] In some embodiments, the point cloud sequence includes a point cloud frame, the point cloud frame includes a point cloud slice, the point cloud slice includes an attribute slice header, and the decoding indication information further includes an attribute quantization parameter field set in the attribute slice header.

[0388] In some embodiments, when the attribute type of the attribute quantization parameter field is a reflectivity attribute, the attribute quantization parameter field includes at least one of a reflectivity attribute quantization parameter and a reflectivity attribute quantization parameter offset; when the attribute type of the attribute quantization parameter field is a color attribute, the attribute quantization parameter field includes any one of the following: a color attribute quantization parameter, an attribute quantization parameter offset of each color component.

[0389] In some embodiments, the decoding indication information includes a loop statement set in the point cloud sequence, the point cloud sequence includes a point cloud frame, the point cloud frame includes a point cloud slice, the point cloud slice includes an attribute slice, the attribute slice is located in the loop statement, and the loop statement traverses each of the attribute slices through the attribute slice header start code.

[0390] In some embodiments, the point cloud sequence includes a point cloud frame, the point cloud frame includes a point cloud slice, the point cloud slice includes an attribute slice, the decoding indication information includes a loop statement set on the attribute slice, and the attribute slice includes an attribute slice header and an attribute data bit stream; the attribute slice header and the attribute data bit stream are both located within the loop statement, and the loop statement traverses each of the attribute slice headers and attribute data bit streams through the attribute slice header start code, and the attribute slice header and the attribute data bit stream correspond one to one; or, the attribute data bit stream is located within the loop statement, and the loop statement traverses each of the attribute data bit streams through the attribute data start code, and the attribute slice header corresponds to one or more attribute data bit streams.

[0391] It should be noted that the specific details of each module unit in the above-mentioned point cloud processing device 300 have been described in detail in the embodiment of the above-mentioned point cloud processing method applied to the decoding device, and will not be repeated here.

[0392] Based on the method described in the above embodiment, the present application also provides a point cloud processing device for executing the steps of the above-mentioned point cloud processing method applied to an encoding device. Please refer to Figure 9, which is a schematic diagram of the structure of another point cloud processing device provided by the present application. The point cloud processing device 400 is applied to an encoding device (e.g., a content production device) and includes a second acquisition module 401 and an encoding compression module 402, wherein:

[0393] The second acquisition module 401 is used to acquire point cloud data;

[0394] Coding and compression module 402 is configured to encode and compress point cloud data to generate a point cloud stream. The point cloud stream includes multiple point cloud frames. The decoding indication information for the point cloud stream includes at least one of a first indication parameter and a second indication parameter. The first indication parameter is set in the sequence-level parameter information of the point cloud sequence, and the second indication parameter is set in the frame-level or slice-level parameter information of the point cloud sequence. The first indication parameter is used for sequence-level decoding indication, and the second indication parameter is used for frame-level or slice-level decoding indication. The point cloud stream is decoded based on the first indication parameter or the second indication parameter.

[0395] It should be noted that the specific details of each module unit in the above-mentioned point cloud processing device 400 have been described in detail in the embodiment of the above-mentioned point cloud processing method applied to the encoding device, and will not be repeated here.

[0396] In some embodiments, the point cloud processing device in the embodiments of the present application can be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other device other than a terminal. For example, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a mobile Internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc. It can also be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), an ATM or a self-service machine, etc., and the embodiments of the present application do not specifically limit this.

[0397] In some embodiments, as shown in Figure 10, the embodiment of the present application also provides an electronic device 500, including a processor 501, a memory 502, and a computer program stored in the memory 502 and executable on the processor 501. When the program is executed by the processor 501, each process of the above-mentioned point cloud processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0398] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0399] FIG11 is a schematic diagram of the hardware structure of an electronic device implementing an embodiment of the present application.

[0400] The electronic device 600 includes but is not limited to: a radio frequency unit 601, a network module 602, an audio output unit 603, an input unit 604, a sensor 605, a display unit 606, a user input unit 607, an interface unit 608, a memory 609 and a processor 610.

[0401] Those skilled in the art will appreciate that the electronic device 600 may further include a power source (e.g., a battery) to power various components. The power source may be logically connected to the processor 610 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The electronic device structure shown in FIG11 does not limit the electronic device. The electronic device may include more or fewer components than shown, or may combine certain components, or have different component arrangements, which will not be described in detail here.

[0402] It should be understood that in an embodiment of the present application, the input unit 604 may include a graphics processing unit (GPU) 6041 and a microphone 6042, and the graphics processor 6041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 606 may include a display panel 6061, and the display panel 6061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 607 includes a touch panel 6071 and at least one of other input devices 6072. The touch panel 6071 is also called a touch screen. The touch panel 6071 may include two parts: a touch detection device and a touch controller. Other input devices 6072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0403] The memory 609 can be used to store software programs and various data. The memory 609 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 609 may include a volatile memory or a non-volatile memory, or the memory 609 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM), and a direct memory bus random access memory (DRRAM). The memory 609 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0404] Processor 610 may include one or more processing units. Processor 610 integrates an application processor and a modem processor. The application processor primarily handles operations related to the operating system, user interface, and application programs, while the modem processor primarily processes wireless communication signals, such as a baseband processor. It is understood that the modem processor may not be integrated into processor 610.

[0405] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned point cloud processing method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0406] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0407] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned point cloud processing method when executed by a processor.

[0408] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0409] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and intent of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A point cloud processing method, characterized in that: The method comprises: Obtain decoding indication information of a point cloud code stream, the decoding indication information including at least one of a first indication parameter and a second indication parameter, the first indication parameter being set in parameter information at a sequence level of a point cloud sequence, and the second indication parameter being set in parameter information at a frame level or a slice level of the point cloud sequence, the first indication parameter being used for performing sequence-level decoding indication, and the second indication parameter being used for performing frame-level or slice-level decoding indication; The point cloud code stream is decoded according to the first indication parameter or the second indication parameter.

2. The method according to claim 1, characterized in that The sequence-level parameter information includes a sequence header, the sequence header also includes a third indication parameter, and the third indication parameter is used to indicate decoding according to the first indication parameter or the second indication parameter; The decoding the point cloud code stream according to the first indication parameter or the second indication parameter includes: When the value of the third indication parameter is a first preset value, decoding each point cloud frame of the point cloud sequence according to the first indication parameter; In a case where the value of the third indication parameter is the second preset value, each of the point cloud frames is decoded according to the second indication parameter of each of the point cloud frames, and the first preset value is different from the second preset value.

3. The method according to claim 1 or 2, characterized in that: The first indication parameter and the second indication parameter both include at least one of a bounding box origin coordinate field, a bounding box size field, and a quantization parameter field.

4. The method according to any one of claims 1 to 3, characterized in that: The first indication parameter is determined according to the second indication parameter of each point cloud frame of the point cloud sequence; or, the first indication parameter is a preset value.

5. The method according to any one of claims 1 to 4, characterized in that: The decoding indication information further includes a fourth indication parameter, and the fourth indication parameter is used to indicate a grouping method of each point cloud frame of the point cloud sequence. The method further includes: A decoding method for each of the point cloud frames is determined according to the fourth indication parameter, and the decoding method includes at least one of a direct prediction method and a prediction transformation method.

6. The method according to claim 5, characterized in that The grouping method includes at least one of the following methods: According to a preset grouping threshold, points in each group are divided into a plurality of sub-groups. When grouping, after the number of points in the previous sub-group reaches the preset grouping threshold, the next sub-group is grouped; In the case that the current point to be grouped is a duplicate point, the current point is used to complete the current subgroup until the number of points in the current subgroup reaches the preset grouping threshold, and the remaining duplicate points are each divided into one of the subgroups; wherein the duplicate point is a point that is the same as any point in the group or any point in the first N points, where N is a positive integer; When the current point being grouped is a duplicate point, the current point is used to complete the current sub-group until the number of points in the current sub-group reaches the preset grouping threshold, and then the remaining duplicate points are stopped from being grouped; When the current point to be grouped is a duplicate point, determine that the current sub-group has been grouped, and continue to group the remaining points according to the preset grouping threshold; In the case that the current point to be grouped is a repeated point, determining that the current sub-group has been grouped, and each of the remaining repeated points is divided into one of the sub-groups; When the current point to be grouped is a duplicate point, it is determined that the current sub-grouping has been completed, and the grouping of the remaining duplicate points is stopped.

7. The method according to claim 5 or 6, characterized in that: The step of determining a decoding method for each of the point cloud frames according to the fourth indication parameter includes: In the case that the current subpacket to be decoded contains repeated points, a decoding method of the current subpacket is determined according to the fourth indication parameter.

8. The method according to claim 6 or 7, characterized in that: Also includes: Determining whether to perform dynamic grouping according to relevant parameters of the quantization parameters of the point cloud frame or the maximum number of points in each of the subgroups; If yes, then adjust the current right shift bit according to the average number of points in the first M consecutive subgroups, where M is a positive integer; Determine the target number of bits of the code corresponding to the points to be grouped according to the adjusted current right shift number; Regroup according to the values ​​within the target number of digits.

9. The method according to any one of claims 1 to 8, characterized in that: Also includes: When the current point to be decoded is a repeated point, it is determined that the predicted values ​​of each attribute type of the current point to be decoded are respectively the reconstructed values ​​of each attribute type of the previous point of the current point to be decoded, and it is determined that the current residual value of the attribute type of the current point to be decoded is non-negative or non-positive.

10. The method according to claim 9, characterized in that The attribute type includes a color attribute, and determining that the predicted values ​​of each attribute type of the current point to be decoded are respectively reconstructed values ​​of each attribute type of a point before the current point to be decoded includes: In the case where the current point to be decoded is a repeated point, determining that the predicted value of the Kth component of the color attribute of the current point to be decoded is the reconstructed value of the Kth component of the previous point of the current point to be decoded, and determining that the Kth component of the current residual value of the color attribute of the current point to be decoded is a non-negative number or a non-positive number, and K is any integer from 1 to 3; or When the current point to be decoded is a repeated point, the predicted values ​​of each component of the color attribute of the current point to be decoded are determined to be the reconstructed values ​​of each component of the previous point of the current point to be decoded, and the current residual value of the color attribute of the current point to be decoded is determined to be a non-negative number or a non-positive number.

11. The method according to any one of claims 1 to 10, characterized in that: The color attribute is represented by an RGB encoding format or a YUV encoding format, the RGB encoding format includes three components of R, G, and B, or the YUV encoding format includes three components of Y, U, and V, and the Kth component is any one of the three components of R, G, and B, or any one of the three components of Y, U, and V.

12. The method according to any one of claims 1 to 11, characterized in that: Also includes: When the current point to be decoded is a repeated point, the predicted value of the reflectivity attribute of the current point to be decoded is determined to be the reconstructed value of the reflectivity attribute of the previous point of the current point to be decoded, and the sign bit of the current residual value of the reflectivity attribute of the current point to be decoded is determined to be a non-negative number or a non-positive number.

13. The method according to any one of claims 1 to 12, characterized in that: The sequence-level parameter information includes a sequence header, and the decoding indication information also includes a fifth indication parameter, where the fifth indication parameter is used to indicate a decoding frame rate. The fifth indication parameter is set in the sequence header, and the value of the fifth indication parameter includes any one of a plurality of preset values. When the value of the fifth indication parameter is different from the preset value, the corresponding decoding frame rate is different.

14. The method according to any one of claims 1 to 13, characterized in that: The decoding indication information also includes an attribute header set in the point cloud frame, the attribute header includes an attribute quantization parameter field, the attribute quantization parameter field is used to control the attribute quantization parameter of the point cloud, and the attribute quantization parameter is used to perform attribute decoding indication on the attribute quantization of the point cloud frame.

15. The method according to any one of claims 1 to 14, characterized in that: The attribute header includes multiple attribute types, each of which has an attribute quantization parameter field. The attribute quantization parameter field under the target attribute type is used to control the attribute quantization parameters of the point cloud under the target attribute type. The attribute quantization parameters under the target attribute type are used to perform attribute decoding instructions on the point cloud frame regarding the attribute quantization under the target attribute type. The target attribute type is any one of the multiple attribute types.

16. The method according to any one of claims 1 to 15, characterized in that: The point cloud sequence includes a point cloud frame, the point cloud frame includes a point cloud slice, the point cloud slice includes an attribute slice header, and the decoding indication information also includes an attribute quantization parameter field set in the attribute slice header.

17. The method according to any one of claims 1 to 16, characterized in that: When the attribute type of the attribute quantization parameter field is a reflectance attribute, the attribute quantization parameter field includes at least one of a reflectance attribute quantization parameter and a reflectance attribute quantization parameter offset; when the attribute type of the attribute quantization parameter field is a color attribute, the attribute quantization parameter field includes any one of the following: a color attribute quantization parameter, an attribute quantization parameter offset of each color component.

18. The method according to any one of claims 1 to 17, characterized in that: The decoding indication information includes a loop statement set in the point cloud sequence, the point cloud sequence includes a point cloud frame, the point cloud frame includes a point cloud slice, the point cloud slice includes an attribute slice, the attribute slice is located in the loop statement, and the loop statement traverses each of the attribute slices through the attribute slice header start code.

19. The method according to any one of claims 1 to 18, characterized in that: The point cloud sequence includes a point cloud frame, the point cloud frame includes a point cloud slice, the point cloud slice includes an attribute slice, the decoding instruction information includes a loop statement set on the attribute slice, and the attribute slice includes an attribute slice header and an attribute data bit stream; The attribute slice header and the attribute data bit stream are both located in the loop statement, and the loop statement traverses each of the attribute slice headers and attribute data bit streams through the attribute slice header start code, and the attribute slice headers and attribute data bit streams correspond to each other one by one; or The attribute data bit stream is located in the loop statement, and the loop statement traverses the attribute data start code For each of the attribute data bit streams, the attribute slice header corresponds to one or more of the attribute data bit streams.

20. A point cloud processing method, characterized in that: Applied to an encoding device, the method comprises: Get point cloud data; The point cloud data is encoded and compressed to obtain a point cloud code stream, wherein decoding indication information of the point cloud code stream includes at least one of a first indication parameter and a second indication parameter, wherein the first indication parameter is set in parameter information at a sequence level of the point cloud sequence, and the second indication parameter is set in parameter information at a frame level or a slice level of the point cloud sequence, wherein the first indication parameter is used for performing sequence-level decoding indication, and the second indication parameter is used for performing frame-level or slice-level decoding indication, and the point cloud code stream is decoded according to the first indication parameter or the second indication parameter.

21. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the point cloud processing method according to any one of claims 1 to 19 or 20 is implemented.

22. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the point cloud processing method according to any one of claims 1 to 19 or 20 is implemented.

23. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the point cloud processing method according to any one of claims 1 to 19 or 20.

Images