Method and apparatus for point cloud coding

Abstract

In some examples, an apparatus for point cloud compression / decompression includes processing circuitry. The processing circuitry determines a flag indicating an enable / disable control for saving an encoding state in maximum coding unit (LCU) based encoding of a point cloud. In some examples, the processing circuitry stores encoding state information before encoding of a first LCU; and in response to the flag indicating the enable control, recovers the encoding state from the stored encoding state information before encoding of a second LCU. In some examples, in response to the flag indicating the enable control, the processing circuitry stores encoding state information before encoding of a first LCU. In some examples, in response to the flag indicating the disable control, the processing circuitry skips the storing / recovering of the encoding state information.

Description

[0001] This application is a divisional application. The original application has the application number 2021800303731 and the application date is September 8, 2021.

[0002] References merged

[0003] This application claims the benefit of priority to U.S. Patent Application No. 17 / 404,740, filed August 17, 2021, entitled “METHOD AND APPARATUS FOR POINT CLOUD CODING,” which in turn claims the benefit of priority to U.S. Provisional Application No. 63 / 139,161, filed January 19, 2021, entitled “Additional Information on Node-Based Geometry and Attribute Coding for a Point Cloud.” The entire disclosure of the earlier applications is incorporated herein by reference. Technical Field

[0004] This disclosure describes implementations of point cloud decoding in general, and more particularly relates to a method and apparatus for point cloud decoding. Background Technology

[0005] The purpose of the background description provided herein is to provide an overall context for this disclosure. The work of the currently named inventors, the work described in this background section, and aspects of the description that may not otherwise be considered prior art prior to the filing date are neither expressly nor implicitly acknowledged as prior art to this disclosure.

[0006] Various technologies have been developed to capture and represent the world in 3-dimensional (3D) space, such as objects and environments within the world. A 3D representation of the world enables more immersive forms of interaction and communication. Point clouds can be used as a 3D representation of the world. A point cloud is a set of points in 3D space, each with associated attributes such as color, material properties, texture information, intensity attributes, reflectivity attributes, motion-related attributes, morphological attributes, and various other attributes. Such point clouds can involve massive amounts of data, and storage and transmission can be both expensive and time-consuming. Summary of the Invention

[0007] According to an embodiment, a method for point cloud decoding is provided. The method includes: determining a first flag indicating an enable / disable control for saving encoding state during maximum coding unit (LCU)-based encoding of a point cloud; storing encoding state information prior to encoding a first LCU; and, in response to the first flag indicating the enable control for saving encoding state, restoring the encoding state based on the stored encoding state information before encoding a second LCU, the second LCU being different from the first LCU.

[0008] According to an embodiment, a computer device is provided. The computer device includes: one or more computer-readable non-transitory storage media configured to store computer program code; and one or more computer processors configured to access the computer program code and execute the above-described method for point cloud decoding as instructed by the computer program code.

[0009] According to an embodiment, an apparatus for point cloud decoding is provided. The apparatus includes processing circuitry configured to perform the above-described method for point cloud decoding.

[0010] According to an embodiment, a non-transitory computer-readable medium is provided for storing instructions. When executed by a computer, the instructions cause the computer to perform the above-described method for point cloud decoding.

[0011] The method and apparatus for point cloud decoding of the present invention provide node-based geometric encoding and attribute encoding for point clouds. Additional information regarding node-based geometric encoding is provided. Specifically, flags are introduced to control whether to save and restore the encoding state. This enables parallel encoding, and when parallel encoding is not required, the encoding state information is not saved and restored, thereby reducing memory and operations. Attached Figure Description

[0012] Other features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which:

[0013] Figure 1 This is a simplified block diagram of a communication system according to an implementation method;

[0014] Figure 2 This is a simplified block diagram of a streaming system according to an implementation method;

[0015] Figure 3 A block diagram of an encoder for encoding point cloud frames according to some embodiments is shown.

[0016] Figure 4A block diagram of a decoder for decoding a compressed bitstream corresponding to a point cloud frame, according to some embodiments, is shown.

[0017] Figure 5 This is a simplified block diagram of a video decoder according to one embodiment;

[0018] Figure 6 This is a simplified block diagram of a video encoder according to another embodiment;

[0019] Figure 7 A block diagram of an encoder for encoding point cloud frames according to some embodiments is shown.

[0020] Figure 8 A block diagram of a decoder for decoding a compressed bitstream corresponding to a point cloud frame, according to some embodiments, is shown.

[0021] Figure 9 The diagram illustrates a cube segmentation technique based on an octree segmentation method according to some embodiments of the present disclosure.

[0022] Figure 10 Examples of octree partitioning and corresponding octree structures according to some embodiments of this disclosure are shown.

[0023] Figure 11 Three quadtree partitioning examples are shown.

[0024] Figure 12 Three binary tree partitioning examples are shown.

[0025] Figure 13 A diagram illustrating the octree structure for breadth-first encoding is shown.

[0026] Figure 14 A diagram illustrating the octree structure for depth-first encoding is shown.

[0027] Figure 15 An example of a prediction tree is shown.

[0028] Figure 16 A diagram is shown illustrating the use of a direct / forward transform architecture in boost-based attribute encoding on the encoder side.

[0029] Figure 17 A diagram is shown illustrating the use of an inverse transform architecture in boost-based attribute encoding on the decoder side.

[0030] Figure 18The diagram shows the forward transformation architecture of Region Adaptive Hierarchical Transform (RAHT) and the inverse transformation architecture of RAHT.

[0031] Figure 19 The following are examples of tree structures corresponding to point clouds according to this disclosure.

[0032] Figure 20 A flowchart outlining the encoding process for encoding geometric and / or attribute information is shown in some examples.

[0033] Figure 21 A flowchart outlining another encoding process for encoding geometric and / or attribute information, as shown in some examples, is illustrated.

[0034] Figures 22A to 22B Examples of syntax representations of some embodiments according to this disclosure are shown.

[0035] Figure 23 A flowchart outlining some embodiments of the encoding process for encoding geometric and / or attribute information is shown.

[0036] Figure 24 A flowchart outlining another encoding process for encoding geometric and / or attribute information according to some embodiments of this disclosure is shown.

[0037] Figure 25 A flowchart outlining an example of a process according to some implementation methods is shown.

[0038] Figure 26 This is a schematic diagram of a computer system according to an implementation method. Detailed Implementation

[0039] This disclosure provides various aspects of Point Cloud Coding (PCC) technology. PCC can be performed using various schemes, such as geometry-based schemes called G-PCC (Geometry-based Point Cloud Coding) and video-based schemes called V-PCC (Video-based Point Cloud Coding). According to some aspects of this disclosure, G-PCC directly encodes 3D geometry and is a purely geometry-based method with little connection to video coding, while V-PCC is largely based on video coding. For example, V-PCC can map points of a 3D cloud to pixels of a 2D mesh (image). The V-PCC scheme can utilize general-purpose video codecs for point cloud compression. The Moving Picture Experts Group (MPEG) is researching G-PCC and V-PCC standards using the G-PCC and V-PCC schemes, respectively.

[0040] Point clouds have a wide range of applications. For example, they can be used for object detection and localization in autonomous vehicles; for map building in Geographic Information Systems (GIS); and for visualizing and archiving cultural heritage objects and collections, among others.

[0041] In the following text, a point cloud can generally refer to a set of points in 3D space, each with associated attributes such as color, material properties, texture information, intensity attributes, reflectivity attributes, motion-related attributes, morphological attributes, and various other attributes. Point clouds can be used to reconstruct objects or scenes as combinations of such points. These points can be captured using multiple camera devices, depth sensors, and / or LiDAR in various settings, and can consist of thousands to billions of points to realistically represent the reconstructed scene or object. A block can generally refer to a continuous subset of surfaces described by a point cloud. In the example, a block consists of points with surface normal vectors that are offset from each other by less than a threshold amount.

[0042] Compression techniques can reduce the amount of data required to represent point clouds for faster transmission or reduced storage. Therefore, there is a need for lossy compression of point clouds for real-time communication and six-degrees-of-freedom (6DoF) virtual reality. Additionally, there is a search for techniques for lossless point cloud compression in the context of dynamic mapping for applications such as autonomous driving and cultural heritage.

[0043] According to one aspect of this disclosure, the main principle behind V-PCC is to use existing video codecs to compress the geometry, occupancy, and texture of a dynamic point cloud into three separate video sequences. The additional metadata required to interpret these three video sequences is compressed separately. A small portion of the overall bitstream is metadata, which can be efficiently encoded / decoded using software implementations. The majority of the information is processed by the video codec.

[0044] Figure 1 A simplified block diagram of a communication system (100) according to an embodiment of the present disclosure is shown. The communication system (100) includes a plurality of terminal devices that can communicate with each other via, for example, a network (150). For example, the communication system (100) includes a pair of terminal devices (110) and (120) interconnected via the network (150). Figure 1 In the example, the first pair of terminal devices (110) and (120) can perform unidirectional transmission of point cloud data. For example, terminal device (110) can compress a point cloud (e.g., points representing structures) captured by a sensor (105) connected to terminal device (110). The compressed point cloud can be transmitted, for example, as a bitstream via a network (150) to another terminal device (120). Terminal device (120) can receive the compressed point cloud from the network (150), decompress the bitstream to reconstruct the point cloud, and display the reconstructed point cloud appropriately. Unidirectional data transmission is common in media service applications, etc.

[0045] exist Figure 1 In the examples, terminal devices (110) and (120) may be shown as servers and personal computers, but the principles of this disclosure are not limited thereto. Implementations of this disclosure are applicable to laptop computers, tablet computers, smartphones, gaming terminals, media players, and / or dedicated three-dimensional (3D) equipment. Network (150) refers to any number of networks that transmit compressed point clouds between terminal devices (110) and (120). Network (150) may include, for example, cable (wired) and / or wireless communication networks. Network (150) may exchange data in circuit-switched channels and / or packet-switched channels. Representative networks include telecommunications networks, local area networks (LANs), wide area networks (WANs), and / or the Internet. For the purposes of this discussion, the architecture and topology of network (150) may be of little importance to the operation of this disclosure unless explained below.

[0046] Figure 2 A simplified block diagram of a streaming system (200) according to an embodiment is shown. Figure 2The example provided is for point cloud applications. The disclosed topics are equally applicable to other point cloud-enabled applications, such as 3D telepresence applications, virtual reality applications, etc.

[0047] The streaming system (200) may include a capture subsystem (213). The capture subsystem (213) may include a point cloud source (201), such as a Light Detection and Ranging (LIDAR) system, a 3D camera device, a 3D scanner, a graphics generation component in software form for generating an uncompressed point cloud, and similar means for generating, for example, an uncompressed point cloud (202). In the example, the point cloud (202) includes points captured by the 3D camera device. The point cloud (202) is depicted as thick lines to emphasize the high data volume compared to a compressed point cloud (204) (a bitstream of the compressed point cloud). The compressed point cloud (204) may be generated by an electronic device (220) including an encoder (203) coupled to the point cloud source (201). The encoder (203) may include hardware, software, or a combination thereof to implement or carry out aspects of the disclosed subject matter as described in more detail below. A compressed point cloud (204) (or a bitstream of the compressed point cloud (204) depicted as thin lines to emphasize its low data volume compared to the stream of the point cloud (202) can be stored on a streaming server (205) for future use. One or more streaming client subsystems, for example Figure 2 The client subsystems (206) and (208) can access the streaming server (205) to retrieve copies (207) and (209) of the compressed point cloud (204). The client subsystem (206) may include a decoder (210) in the electronic device (230), for example. The decoder (210) decodes the input copy (207) of the compressed point cloud and creates an output stream of the reconstructed point cloud (211) that can be rendered on the rendering device (212).

[0048] Note that electronic devices (220) and (230) may include other components (not shown). For example, electronic device (220) may include a decoder (not shown), and electronic device (230) may also include an encoder (not shown).

[0049] In some streaming systems, compressed point clouds (204), (207), and (209) (e.g., a bitstream of a compressed point cloud) can be compressed according to certain standards. In some examples, video coding standards are used in the compression of point clouds. Examples of these standards include High Efficiency Video Coding (HEVC), Versatile Video Coding (VVC), etc.

[0050] Figure 3 A block diagram of a V-PCC encoder (300) for encoding point cloud frames is shown according to some embodiments. In some embodiments, the V-PCC encoder (300) can be used in a communication system (100) and a streaming system (200). For example, the encoder (203) can be configured and operated in a manner similar to that of the V-PCC encoder (300).

[0051] The V-PCC encoder (300) receives point cloud frames as uncompressed input and generates a bitstream corresponding to the compressed point cloud frames. In some implementations, the V-PCC encoder (300) can receive point cloud frames from a point cloud source such as a point cloud source (201).

[0052] exist Figure 3 In the example, the V-PCC encoder (300) includes a block generation module (306), a block encapsulation module (308), a geometric image generation module (310), a texture image generation module (312), a block information module (304), an occupancy map module (314), a smoothing module (336), image filling modules (316) and (318), a group extension module (320), video compression modules (322), (323) and (332), an auxiliary block information compression module (338), an entropy compression module (334), and a multiplexer (324).

[0053] According to one aspect of this disclosure, a V-PCC encoder (300) converts a 3D point cloud frame into an image-based representation and some metadata (e.g., occupancy map and block information) used to convert the compressed point cloud back into a decompressed point cloud. In some examples, the V-PCC encoder (300) can convert a 3D point cloud frame into a geometry image, a texture image, and an occupancy map, and then encode the geometry image, texture image, and occupancy map into a bitstream using video coding techniques. Typically, a geometry image is a 2D image having pixels filled with geometric values ​​associated with points projected onto the pixel, and pixels filled with geometric values ​​may be referred to as geometry samples. A texture image is a 2D image having pixels filled with texture values ​​associated with points projected onto the pixel, and pixels filled with texture values ​​may be referred to as texture samples. An occupancy map is a 2D image having pixels filled with values ​​indicating whether a block is occupied or not.

[0054] The block generation module (306) divides the point cloud into a set of blocks that may overlap or not overlap (e.g., blocks are defined as a continuous subset of surfaces described by the point cloud), such that each block can be described by a depth field relative to a plane in 2D space. In some implementations, the block generation module (306) aims to decompose the point cloud into a minimum number of blocks with smooth boundaries while also minimizing reconstruction errors.

[0055] The block information module (304) can collect block information indicating the size and shape of the blocks. In some examples, the block information can be encapsulated into an image frame and then encoded by the auxiliary block information compression module (338) to generate compressed auxiliary block information.

[0056] The block encapsulation module (308) is configured to map the extracted blocks to a two-dimensional (2D) grid while minimizing unused space and ensuring that each M×M (e.g., 16x16) block of the grid is associated with a unique block. Effective block encapsulation can directly impact compression efficiency by minimizing unused space or ensuring temporal consistency.

[0057] The geometry image generation module (310) generates a 2D geometry image associated with the geometry of the point cloud at a given block location. The texture image generation module (312) generates a 2D texture image associated with the texture of the point cloud at a given block location. The geometry image generation module (310) and the texture image generation module (312) utilize a 3D-to-2D mapping computed during the encapsulation process to store the geometry and texture of the point cloud as images. To better handle the case where multiple points are projected onto the same sample, each block is projected onto two images (referred to as layers). In the example, the geometry image is represented by a monochrome frame in WxH format of YUV420-8 bits. To generate the texture image, the texture generation process utilizes the reconstructed / smoothed geometry to compute the color to be associated with the resampled points.

[0058] The occupancy map module (314) can generate an occupancy map describing the filling information at each cell. For example, the occupancy map includes a binary map that indicates for each cell of the grid whether the cell belongs to empty space or to the point cloud. In the example, the occupancy map uses binary information describing whether a pixel is filled for each pixel. In another example, the occupancy map uses binary information describing whether a pixel block is filled for each pixel block.

[0059] The occupancy graph generated by the occupancy graph module (314) can be compressed using either lossless or lossy encoding. When lossless encoding is used, the occupancy graph is compressed using the entropy compression module (334). When lossy encoding is used, the occupancy graph is compressed using the video compression module (332).

[0060] Note that the block encapsulation module (308) can leave some blank space between the 2D blocks encapsulated in the image frame. The image padding module (316) and the image padding module (318) can fill the blank space (referred to as padding) to generate an image frame that can be adapted to 2D video and image codecs. Image padding is also referred to as background padding, which fills unused space with redundant information. In some examples, good background padding increases the bit rate to a minimum without introducing significant coding distortion around block boundaries.

[0061] The video compression modules (322), (323), and (332) can encode 2D images, such as filled geometric images, filled texture images, and occupancy maps, based on suitable video coding standards such as HEVC, VVC, etc. In this example, the video compression modules (322), (323), and (332) are separate components that operate independently. Note that in another example, the video compression modules (322), (323), and (332) can be implemented as a single component.

[0062] In some examples, the smoothing module (336) is configured to generate a smoothed image of the reconstructed geometry. The smoothed image can be provided to the texture image generation module (312). The texture image generation module (312) can then adjust the generation of the texture image based on the reconstructed geometry. For example, if the block shape (e.g., geometry) is slightly distorted during encoding and decoding, the distortion can be taken into account when generating the texture image to correct for the distortion of the block shape.

[0063] In some implementations, the group extension module (320) is configured to fill pixels around the object boundary with redundant low-frequency content to improve the coding gain and visual quality of the reconstructed point cloud.

[0064] The multiplexer (324) can multiplex the compressed geometric image, the compressed texture image, the compressed occupancy map, and the compressed auxiliary block information into a compressed bit stream.

[0065] Figure 4 A block diagram of a V-PCC decoder (400) for decoding a compressed bitstream corresponding to a point cloud frame, according to some embodiments, is shown. In some embodiments, the V-PCC decoder (400) can be used in a communication system (100) and a streaming system (200). For example, a decoder (210) can be configured to operate in a manner similar to the V-PCC decoder (400). The V-PCC decoder (400) receives the compressed bitstream and generates a reconstructed point cloud based on the compressed bitstream.

[0066] exist Figure 4 In the example, the V-PCC decoder (400) includes a demultiplexer (432), a video decompression module (434) and (436), a occupancy graph decompression module (438), an auxiliary block information decompression module (442), a geometry reconstruction module (444), a smoothing module (446), a texture reconstruction module (448), and a color smoothing module (452).

[0067] The demultiplexer (432) can receive the compressed bit stream and separate it into a compressed texture image, a compressed geometric image, a compressed occupancy map, and compressed auxiliary block information.

[0068] Video decompression modules (434) and (436) can decode compressed images according to suitable standards (e.g., HEVC, VVC, etc.) and output decompressed images. For example, video decompression module (434) decodes compressed texture images and outputs decompressed texture images; and video decompression module (436) decodes compressed geometric images and outputs decompressed geometric images.

[0069] The occupancy map decompression module (438) can decode the compressed occupancy map according to a suitable standard (e.g., HEVC, VVC, etc.) and output the decompressed occupancy map.

[0070] The auxiliary block information decompression module (442) can decode the compressed auxiliary block information according to a suitable standard (e.g., HEVC, VVC, etc.) and output the decompressed auxiliary block information.

[0071] The geometry reconstruction module (444) can receive a decompressed geometry image and generate a reconstructed point cloud geometry based on the decompressed occupancy map and decompressed auxiliary block information.

[0072] The smoothing module (446) can smooth out inconsistencies at the edges of blocks. The smoothing process is designed to mitigate potential discontinuities that may occur at block boundaries due to compression artifacts. In some implementations, a smoothing filter can be applied to pixels located at block boundaries to mitigate distortion that may be caused by compression / decompression.

[0073] The texture reconstruction module (448) can determine the texture information of points in the point cloud based on the decompressed texture image and smooth geometry.

[0074] The color smoothing module (452) can smooth out inconsistencies in shading. Non-adjacent blocks in 3D space are typically encapsulated adjacent to each other in 2D video. In some examples, pixel values ​​from non-adjacent blocks may be mixed by a block-based video codec. The purpose of color smoothing is to reduce visible artifacts that appear at block boundaries.

[0075] Figure 5 A block diagram of a video decoder (510) according to an embodiment of the present disclosure is shown. The video decoder (510) can be used in a V-PCC decoder (400). For example, video decompression modules (434) and (436), and occupancy graph decompression module (438) can be similarly configured as video decoders (510).

[0076] The video decoder (510) may include a parser (520) to reconstruct symbols (521) from a compressed image, such as an encoded video sequence. The categories of these symbols include information for managing the operation of the video decoder (510). The parser (520) may perform parsing / entropy decoding on the received encoded video sequence. The encoding of the encoded video sequence may be performed according to video coding techniques or standards and may follow various principles, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (520) may extract a set of subgroup parameters from the encoded video sequence based on at least one parameter corresponding to a group, for use in the video decoder. Subgroups may include Group of Pictures (GOP), pictures, tiles, slices, macroblocks, coding units (CU), blocks, transform units (TU), prediction units (PU), etc. The parser (520) may also extract information such as transform coefficients, quantizer parameter values, motion vectors, etc., from the encoded video sequence.

[0077] The parser (520) can perform entropy decoding / parsing operations on the video sequence received from the buffer memory to create symbols (521).

[0078] The reconstruction of the symbol (521) may involve multiple different units, depending on the type of the encoded video picture or a portion thereof (e.g., inter-frame and intra-frame pictures, inter-frame blocks and intra-frame blocks) and other factors. Which units are involved and how they are involved can be controlled by subgroup control information parsed from the encoded video sequence by the parser (520). For simplicity, the flow of this subgroup control information between the parser (520) and the following multiple units is not depicted.

[0079] In addition to the functional blocks already mentioned, the video decoder (510) can be conceptually subdivided into multiple functional units as described below. In a practical implementation operating under commercial constraints, many of these units interact closely with each other and can be at least partially integrated with one another. However, for the purposes of describing the disclosed subject matter, it is appropriate to conceptually subdivide it into the functional units described below.

[0080] The first unit is the scaler / inverse transform unit (551). The scaler / inverse transform unit (551) receives quantized transform coefficients as symbols (521) from the parser (520) and control information, including which transform to use, block size, quantization factor, quantization scaling matrix, etc. The scaler / inverse transform unit (551) can output blocks containing sample values, which can be input into the aggregator (555).

[0081] In some cases, the output samples of the scaler / inverse transform unit (551) may belong to intra-coded blocks; that is, blocks that do not use predictive information from previously reconstructed images but can use predictive information from previously reconstructed portions of the current image. Such predictive information may be provided by the intra-picture prediction unit (552). In some cases, the intra-picture prediction unit (552) uses surrounding reconstructed information extracted from the current picture buffer (558) to generate blocks with the same size and shape as the blocks being reconstructed. For example, the current picture buffer (558) buffers, for example, a partially reconstructed current picture and / or a fully reconstructed current picture. In some cases, the aggregator (555) adds the predictive information already generated by the intra-picture prediction unit (552) to the output sample information provided by the scaler / inverse transform unit (551) based on each sample.

[0082] In other cases, the output samples of the scaler / inverse transform unit (551) may belong to blocks that are inter-frame coded and potentially motion-compensated. In this case, the motion compensation prediction unit (553) can access the reference image memory (557) to extract samples for prediction. After motion compensation of the extracted samples according to the symbols (521) belonging to the block, these samples can be added to the output of the scaler / inverse transform unit (551) (referred to as residual samples or residual signals in this case) by the aggregator (555) to generate output sample information. The address in the reference image memory (557) from which the motion compensation prediction unit (553) extracts its prediction samples can be controlled by motion vectors and can be used by the motion compensation prediction unit (553) in the form of symbols (521), which may have, for example, X components, Y components, and reference image components. Motion compensation may also include, for example, interpolation of sample values ​​extracted from the reference image memory (557) when using subsample precise motion vectors, motion vector prediction mechanisms, etc.

[0083] The output samples of the aggregator (555) can undergo various loop filtering techniques in the loop filter unit (556). The video compression technique may include an in-loop filtering technique controlled by parameters included in the encoded video sequence (also known as the encoded video bitstream), and these parameters can be used as symbols (521) from the parser (520) in the loop filter unit (556). However, the video compression technique may also respond to metadata obtained during the decoding of a previous (in the order of decoding) portion of the encoded picture or encoded video sequence, and to previously reconstructed and loop-filtered sample values.

[0084] The output of the loop filter unit (556) can be a sample stream, which can be output to the presentation device and stored in the reference image memory (557) for future inter-frame image prediction.

[0085] Once fully reconstructed, certain encoded images can be used as reference images for future predictions. For example, once the encoded image corresponding to the current image has been fully reconstructed and that encoded image (by, for example, the parser (520)) has been identified as the reference image, the current image buffer (558) can become part of the reference image memory (557), and a new current image buffer can be reallocated before the reconstruction of subsequent encoded images begins.

[0086] The video decoder (510) can perform decoding operations according to a predetermined video compression technique as specified in a standard such as ITU-T (International Telecommunication Union-Telecommunication Standardization Sector) Recommendation H.265. The encoded video sequence may conform to the syntax specified by the video compression technique or standard used, in the sense that the encoded video sequence follows the syntax of the video compression technique or standard and the configuration file recorded in the video compression technique or standard. Specifically, the configuration file may select certain tools from all available tools in the video compression technique or standard as the only tools available under said configuration file. For compliance, the complexity of the encoded video sequence is also required to be within the range defined by the hierarchy of the video compression technique or standard. In some cases, the hierarchy limits the maximum picture size, maximum frame rate, maximum reconstruction sampling rate (measured in megasamples per second, for example), maximum reference picture size, etc. In some cases, the limitations set by the hierarchy can be further limited by the Hypothetical Reference Decoder (HRD) specification and metadata sent in the encoded video sequence for HRD buffer management.

[0087] Figure 6 A block diagram of a video encoder (603) according to an embodiment of the present disclosure is shown. The video encoder (603) can be used in a V-PCC encoder (300) for compressing point clouds. In the example, video compression modules (322) and (323) and video compression module (332) are configured similarly to the encoder (603).

[0088] The video encoder (603) can receive images such as filled geometric images, filled texture images, etc., and can generate compressed images.

[0089] According to the implementation, the video encoder (603) can encode and compress the images (pictures) of the source video sequence into an encoded video sequence (compressed images) in real time or under any other time constraints required by the application. Implementing an appropriate encoding rate is a function of the controller (650). In some implementations, the controller (650) controls and is functionally coupled to other functional units as described below. For simplicity, such coupling is not depicted. Parameters set by the controller (650) may include rate control related parameters (picture skipping, quantizer, λ value of rate distortion optimization techniques, etc.), picture size, Group of Picture (GOP) layout, maximum motion vector search range, etc. The controller (650) can be configured to have other suitable functions related to the video encoder (603) optimized for a particular system design.

[0090] In some implementations, the video encoder (603) is configured to operate within an encoding loop. As an oversimplification, in this example, the encoding loop may include a source encoder (630) (e.g., responsible for creating symbols, such as a symbol stream, based on the input picture to be encoded and a reference picture) and a (local) decoder (633) embedded within the video encoder (603). The decoder (633) reconstructs the symbols to create sample data in a manner similar to how a (remote) decoder would also create sample data (since any compression between the symbols and the encoded video bitstream is lossless in the video compression or techniques considered in the disclosed subject matter). The reconstructed sample stream (sample data) is input to a reference picture memory (634). Since decoding of the symbol stream produces bit-precise results independent of the decoder's location (local or remote), the contents of the reference picture memory (634) are also bit-precise between the local and remote encoders. That is, the reference picture samples "seen" by the encoder's prediction section are exactly the same sample values ​​that the decoder will "see" when using the prediction during decoding. This fundamental principle of reference image synchronization (and the drift that occurs when synchronization cannot be maintained, for example, due to channel errors) is also used in some related technologies.

[0091] The operation of the “local” decoder (633) can be combined with, for example, the above-mentioned... Figure 5 The video decoder (510) described in detail is the same as the "remote" decoder. However, a brief reference is also provided. Figure 5 When symbols are available and the entropy encoder (645) and parser (520) are able to encode / decode the symbols into an encoded video sequence without loss, the entropy decoding portion of the video decoder (510), including the parser (520), may not be fully implemented in the local decoder (633).

[0092] It can be observed that any decoder technique other than parsing / entropy decoding, which exists in the decoder, must also exist in the corresponding encoder in essentially the same functional form. For this reason, the subject matter disclosed focuses on decoder operation. The description of encoder techniques can be simplified, as encoder techniques are inverses of the fully described decoder techniques. More detailed descriptions are only required in certain areas, and are provided below.

[0093] During operation, in some examples, the source encoder (630) may perform motion-compensated predictive coding, which predictively codes the input image with reference to one or more previously encoded images from the video sequence designated as "reference images". In this way, the encoding engine (632) encodes the difference between pixel blocks of the input image and pixel blocks of the reference image that can be selected as the predictive reference for the input image.

[0094] The local video decoder (633) can decode encoded video data of a picture that can be designated as a reference picture based on symbols created by the source encoder (630). The operation of the encoding engine (632) can be advantageously for lossy processing. When encoded video data can be decoded by the video decoder (633), Figure 6 When decoded at (not shown), the reconstructed video sequence can typically be a copy of the source video sequence with some errors. The local video decoder (633) repeats the decoding processing that can be performed on the reference image by the video decoder, and can store the reconstructed reference image in a reference image cache (634). In this way, the video encoder (603) can locally store a copy of the reconstructed reference image that shares common content with the reconstructed reference image to be obtained by the remote video decoder (no transmission errors).

[0095] The predictor (635) can perform a prediction search against the encoding engine (632). That is, for a new image to be encoded, the predictor (635) can search in the reference image memory (634) for sample data (as candidate reference pixel blocks) or certain metadata such as reference image motion vectors, block shapes, etc. that can serve as appropriate prediction references for the new image. The predictor (635) can operate pixel-by-pixel based on the sample blocks to find suitable prediction references. In some cases, as determined by the search results obtained by the predictor (635), the input image may have prediction references obtained from multiple reference images stored in the reference image memory (634).

[0096] The controller (650) can manage the encoding operations of the source encoder (630), including, for example, setting parameters and subgroup parameters for encoding video data.

[0097] The outputs of all the aforementioned functional units can be entropy encoded in the entropy encoder (645). The entropy encoder (645) converts the symbols generated by the various functional units into an encoded video sequence by lossless compression of the symbols using techniques such as Huffman coding, variable-length coding, and arithmetic coding.

[0098] The controller (650) can manage the operation of the video encoder (603). During encoding, the controller (650) can specify a specific encoded image type for each encoded image, which may affect the encoding techniques that can be applied to the corresponding image. For example, an image can typically be specified as one of the following image types:

[0099] An intra-frame picture (I-picture) can be a picture that can be encoded and decoded without using any other pictures in the sequence as a prediction source. Some video codecs allow different types of intra-frame pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art will understand these variations of I-pictures and their corresponding applications and characteristics.

[0100] A predictive image (P-image) can be an image that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses at most one motion vector and reference index to predict sample values ​​for each block.

[0101] A bidirectional predictive image (B-image) can be an image that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses at most two motion vectors and a reference index to predict sample values ​​for each block. Similarly, multiple predictive images can use more than two reference images and associated metadata to reconstruct a single block.

[0102] Typically, the source image can be spatially subdivided into multiple sample blocks (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples), and encoding can be performed on a block-by-block basis. Blocks can be predictively coded with reference to other (already coded) blocks, as determined by the coding assignment of the corresponding image applied to the block. For example, blocks of image I can be non-predictively coded, or blocks of image I can be predictively coded with reference to already coded blocks of the same image (spatial prediction or intra-frame prediction). Pixel blocks of image P can be predictively coded with reference to a previously coded reference image via spatial prediction or temporal prediction. Blocks of image B can be predictively coded with reference to one or two previously coded reference images via spatial prediction or temporal prediction.

[0103] The video encoder (603) can perform encoding operations according to a predetermined video coding technique or standard, such as ITU-T H.265 Recommendation. In the operation of the video encoder (603), various compression operations can be performed, including predictive coding operations utilizing temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard used.

[0104] Video can be presented as a series of multiple source images over time. Intra-frame image prediction (often abbreviated as intra-prediction) utilizes spatial correlations within a given image, while inter-frame image prediction utilizes (temporal or other) correlations between images. In the example, the specific image being encoded / decoded—called the current image—is segmented into blocks. If a block in the current image resembles a reference block in a previously encoded and still buffered reference image in the video, the block in the current image can be encoded using a vector called a motion vector. The motion vector points to the reference block in the reference image, and when using multiple reference images, the motion vector can have a third dimension that identifies the reference images.

[0105] In some implementations, bidirectional prediction techniques can be used in inter-frame image prediction. According to bidirectional prediction, two reference images are used, such as a first reference image and a second reference image that precede the current image in the video in decoding order (but may be past and future in display order). A block in the current image can be encoded using a first motion vector pointing to a first reference block in the first reference image and a second motion vector pointing to a second reference block in the second reference image. The block can be predicted using a combination of the first and second reference blocks.

[0106] In addition, merging mode techniques can be used in inter-frame image prediction to improve coding efficiency.

[0107] According to some embodiments of this disclosure, predictions such as inter-frame picture prediction and intra-frame picture prediction are performed on a block-by-block basis. For example, according to the HEVC standard, images in a video picture sequence are segmented into Coding Tree Units (CTUs) for compression, with CTUs in the images having the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Generally, a CTU comprises three Coding Tree Blocks (CTBs), one luma CTB and two chroma CTBs. Each CTU can also be recursively subdivided into one or more Coding Units (CUs) using a quadtree. For example, a 64×64 pixel CTU can be subdivided into one 64×64 pixel CU, or four 32×32 pixel CUs, or sixteen 16×16 pixel CUs. In the example, each CU is analyzed to determine the prediction type used for that CU, such as inter-frame prediction or intra-frame prediction. Based on temporal and / or spatial predictability, the CU is divided into one or more Prediction Units (PUs). Typically, each PU includes a luma prediction block (PB) and two chroma PBs. In implementations, prediction operations during decoding (encoding / decoding) are performed on a per-prediction-block basis. Using a luma prediction block as an example, a prediction block comprises a matrix of pixel values ​​(e.g., luma values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0108] Figure 7 A block diagram of a G-PPC encoder (700) according to an embodiment is shown. The encoder (700) can be configured to receive point cloud data and compress the point cloud data to generate a bitstream carrying the compressed point cloud data. In an embodiment, the encoder (700) may include a position quantization module (710), a duplicate point removal module (712), an octree encoding module (730), an attribute transformation module (720), a Level of Detail (LOD) generation module (740), an attribute prediction module (750), a residual quantization module (760), an arithmetic encoding module (770), an inverse residual quantization module (780), an addition module (781), and a memory (790) for storing the reconstructed attribute values.

[0109] As shown, an input point cloud (701) can be received at the encoder (700). The positions (e.g., 3D coordinates) of the point cloud (701) are provided to the quantization module (710). The quantization module (710) is configured to quantize the coordinates to generate quantized positions. A duplicate point removal module (712) is configured to receive the quantized positions and perform filtering to identify and remove duplicate points. An octree encoding module (730) is configured to receive the filtered positions from the duplicate point removal module (712) and perform octree-based encoding to generate a sequence of occupancy codes describing a 3D mesh of voxels. The occupancy codes are provided to the arithmetic encoding module (770).

[0110] The attribute transformation module (720) is configured to receive attributes of the input point cloud and perform attribute transformation processing to determine the attribute value of each voxel when multiple attribute values ​​are associated with corresponding voxels. Attribute transformation processing can be performed on the reordered points output from the octree encoding module (730). The attributes after the transformation operation are provided to the attribute prediction module (750). The LOD generation module (740) is configured to operate on the reordered points output from the octree encoding module (730) and reorganize these points into different LODs. LOD information is provided to the attribute prediction module (750).

[0111] The attribute prediction module (750) processes points according to the LOD-based order indicated by LOD information from the LOD generation module (740). The attribute prediction module (750) generates an attribute prediction for the current point based on the reconstructed attributes of a set of neighboring points stored in memory (790). The prediction residual can then be obtained based on the original attribute values ​​received from the attribute transformation module (720) and the locally generated attribute prediction. When a candidate index is used in the corresponding attribute prediction process, the index corresponding to the selected prediction candidate can be provided to the arithmetic encoding module (770).

[0112] The residual quantization module (760) is configured to receive the predicted residuals from the attribute prediction module (750) and perform quantization to generate quantized residuals. The quantized residuals are then provided to the arithmetic encoding module (770).

[0113] The inverse residual quantization module (780) is configured to receive the quantized residual from the residual quantization module (760) and generate the reconstructed prediction residual by performing the inverse operation of the quantization operation performed at the residual quantization module (760). The addition module (781) is configured to receive the reconstructed prediction residual from the inverse residual quantization module (780) and the corresponding attribute prediction from the attribute prediction module (750). By combining the reconstructed prediction residual and the attribute prediction, a reconstructed attribute value is generated and stored in the memory (790).

[0114] The arithmetic coding module (770) is configured to receive occupancy codes, candidate indices (if used), quantized prediction residuals (if generated), and other information, and to perform entropy coding to further compress the received values ​​or information. As a result, a compressed bitstream (702) carrying the compressed information can be generated. The bitstream (702) can be transmitted or otherwise provided to a decoder that decodes the compressed bitstream, or it can be stored in a storage device.

[0115] Figure 8 A block diagram of a G-PCC decoder (800) according to an embodiment is shown. The decoder (800) can be configured to receive a compressed bitstream and perform point cloud data decompression to decompress the bitstream to generate decoded point cloud data. In an embodiment, the decoder (800) may include an arithmetic decoding module (810), an inverse residual quantization module (820), an octree decoding module (830), an LOD generation module (840), an attribute prediction module (850), and a memory (860) for storing the reconstructed attribute values.

[0116] As shown, a compressed bitstream (801) can be received at the arithmetic decoding module (810). The arithmetic decoding module (810) is configured to decode the compressed bitstream (801) to obtain the quantized residual (if generated) and occupancy code of the point cloud. The octree decoding module (830) is configured to determine the reconstructed position of the points in the point cloud based on the occupancy code. The LOD generation module (840) is configured to reorganize the points into different LODs based on the reconstructed positions and determine the LOD-based order. The inverse residual quantization module (820) is configured to generate the reconstructed residual based on the quantized residual received from the arithmetic decoding module (810).

[0117] The attribute prediction module (850) is configured to perform attribute prediction processing to determine the attribute predictions of points based on LOD-based order. For example, the attribute predictions of the current point can be determined based on the reconstructed attribute values ​​of the current point's neighboring points stored in memory (860). In some examples, the attribute predictions can be combined with the corresponding reconstruction residuals to generate reconstructed attributes for the current point.

[0118] In one example, the sequence of reconstructed attributes generated from the attribute prediction module (850), along with the reconstructed locations generated from the octree decoding module (830), corresponds to the decoded point cloud (802) output from the decoder (800). Additionally, the reconstructed attributes are stored in memory (860) and can be subsequently used to derive attribute predictions for subsequent points.

[0119] In various implementations, the encoder (300), decoder (400), encoder (700), and / or decoder (800) can be implemented in hardware, software, or a combination thereof. For example, the encoder (300), decoder (400), encoder (700), and / or decoder (800) can be implemented using processing circuitry such as one or more integrated circuits (ICs) operating with or without software, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), etc. In another example, the encoder (300), decoder (400), encoder (700), and / or decoder (800) can be implemented as software or firmware comprising instructions stored in a non-volatile (or non-transitory) computer-readable storage medium. These instructions, when executed by processing circuitry such as one or more processors, cause the processing circuitry to perform the functions of the encoder (300), decoder (400), encoder (700), and / or decoder (800).

[0120] Note that the attribute prediction modules (750) and (850) configured to implement the attribute prediction techniques disclosed herein may include components that can have the same characteristics as those described herein. Figure 7 and Figure 8 The decoder or encoder shown may be in other decoders or encoders with similar or different structures. Additionally, in various examples, the encoder (700) and decoder (800) may be included in the same device or in separate devices.

[0121] This disclosure provides various aspects of techniques for use in G-PCC. Specifically, node-based geometric coding techniques and / or attribute coding techniques for G-PCC are described in the following description.

[0122] According to some aspects of this disclosure, the geometric information and associated attributes of a point cloud, such as color and reflectivity, can be compressed separately (e.g., in the MPEG G-PCC Test Model 13 (TMC13) model). The geometric information of the point cloud—including the 3D coordinates of points in the point cloud—can be encoded by segmentation and the occupancy information of said segmentation, for example by octree segmentation, quadtree segmentation, and binary tree segmentation and the occupancy information of these segmentations. After the geometric information is encoded, attributes can be compressed based on the reconstructed geometry using techniques such as prediction, lifting, and region-adaptive hierarchical transformation.

[0123] Several techniques can be used for geometric coding. These techniques may include tree-based geometric coding, prediction tree-based geometric coding, and trisoup-based geometric coding. Tree-based geometric coding, prediction tree-based geometric coding, and trisoup-based geometric coding will be described separately.

[0124] According to one aspect of this disclosure, tree segmentation, such as octree segmentation, quadtree segmentation, binary tree segmentation, etc., can be used to segment three-dimensional space. Using octree segmentation as an example, an octree is a three-dimensional simulation of a quadtree in two-dimensional space. Octtree segmentation refers to a segmentation technique that recursively subdivides three-dimensional space into eight octets, and an octree structure refers to a tree structure representing the segmentation. In the example, each node in the octree structure corresponds to three-dimensional space, and this node can be a terminal node (no longer segmented, also referred to as a leaf node in some examples) or a node with further segmentation. An octree segmentation at a node can divide the three-dimensional space represented by that node into eight octets. In some examples, the node corresponding to the segmentation of a particular node can be referred to as a child node of that particular node.

[0125] Figure 9 A diagram illustrating the segmentation of a 3D cube (900) (corresponding to nodes) using an octree-based segmentation technique according to some embodiments of the present disclosure is shown. This segmentation divides the 3D cube (900) into eight smaller cubes of equal size, 0 to 7, such as... Figure 9 As shown. In Figure 9 In the 3D cube (900), the x, y and z dimensions are divided into two parts, and the division can produce 8 sub-cubes of the same size.

[0126] Octree partitioning techniques (e.g., in TMC13) can recursively divide the original 3D space into smaller units and encode the occupancy information of each subspace to represent its geometric location.

[0127] In some implementations (e.g., in TMC13), an octree geometry codec is used. The octree geometry codec can perform geometric encoding. In some examples, geometric encoding is performed on a cube box. For example, the cube box could be composed of two points (0, 0, 0) and (2... M-1 , 2 M-1 , 2 M-1 The axis-aligned bounding box B is defined by 2. M-1 The size of the bounding box B is defined, and M can be specified in the bitstream.

[0128] Then, the octree structure is constructed by recursively subdividing the cube boxes. For example, from two points (0, 0, 0) and (2... M-1, 2 M-1 , 2 M-1 The defined cube is divided into 8 sub-cubes, and then an 8-bit code is generated, called the occupancy code. Each bit in the occupancy code is associated with a sub-cube, and the value of that bit indicates whether the associated sub-cube contains any points in the point cloud. For example, a bit value of 1 indicates that the sub-cube associated with that bit contains one or more points in the point cloud; while a bit value of 0 indicates that the sub-cube associated with that bit does not contain any points in the point cloud.

[0129] Furthermore, for empty sub-cube boxes (e.g., the bit associated with the sub-cube box has a value of 0), no further subdivision is applied. When a sub-cube box has one or more points in the point cloud (e.g., the bit associated with the sub-cube box has a value of 1), the sub-cube box is further subdivided into 8 smaller sub-cube boxes, and an occupancy code is generated for the sub-cube box to indicate the occupancy of the smaller sub-cube boxes. In some examples, the subdivision operation can be repeatedly performed on non-empty sub-cube boxes until the size of the sub-cube box is equal to a predetermined threshold, such as a size of 1. In some examples, sub-cube boxes with a size of 1 are referred to as voxels, and sub-cube boxes with a size larger than a voxel can be referred to as non-voxels.

[0130] Figure 10 Examples of an octree segmentation (1010) and an octree structure (1020) corresponding to an octree segmentation (1010) according to some embodiments of the present disclosure are shown. Figure 10 A two-level partitioning in an octree partition (1010) is shown. The octree structure (1020) includes nodes (N0) corresponding to the cube boxes used for the octree partition (1010). Node N0 is at depth 0 in the octree structure (1020). At the first level of the partitioning, the cube boxes are partitioned according to... Figure 9 The diagram shows eight sub-cube boxes numbered 0 through 7. The occupancy code for the segmentation of node N0 is in binary form "10000001", indicating that the first sub-cube box represented by node N0-0 and the eighth sub-cube box represented by node N0-7 contain points in the point cloud, while the other sub-cube boxes are empty. Nodes N0-0 to N0-7 are at depth 1 in the octree structure (1020).

[0131] Then, at the second level of segmentation, the first sub-cube box (represented by node N0-0) and the eighth sub-cube box (represented by node N0-7) are further subdivided into eight octets. For example, the first sub-cube box (represented by node N0-0) is segmented according to... Figure 9The diagram shows eight smaller sub-cube boxes numbered 0 through 7. The occupancy code for the segmentation of node N0-0 is in binary form "00011000", indicating that the fourth smaller sub-cube box (represented by node N0-0-3) and the fifth smaller sub-cube box (represented by node N0-0-4) contain points in the point cloud, while the other smaller sub-cube boxes are empty. Nodes N0-0-0 to N0-0-7 are at depth 2 in the octree structure (1020). Similarly, at the second level, the eighth sub-cube box (represented by node N0-7) is segmented into eight smaller sub-cube boxes, as shown... Figure 10 As shown. Nodes N0-7-0 to N0-7-7 are at depth 2 in the octree structure (1020).

[0132] exist Figure 10 In the example, nodes corresponding to non-empty cube spaces (e.g., cube boxes, sub-cube boxes, smaller sub-cube boxes, etc.) are colored gray and are called shadow nodes. Nodes corresponding to empty cube spaces (e.g., cube boxes, sub-cube boxes, smaller sub-cube boxes, etc.) are colored white and can be called blank nodes.

[0133] Although octree segmentation was described above, tree-based geometric coding techniques can typically use other segmentations, such as quadtree segmentation or binary tree segmentation, to segment point clouds.

[0134] More typically, for point clouds, the bounding box B is not limited to having the same size in all directions; rather, the bounding box B can be a rectangular prism of arbitrary size to better fit the shape of the 3D scene or object. In some examples, the size of the bounding box B can be expressed as a power of 2, for example... In the example, d x d y d z They are not equal.

[0135] In order to segment the bounding box B of the point cloud, in addition to the octree segmentation described above, quadtree segmentation and binary tree segmentation described below can also be used.

[0136] Figure 11 Three quadtree partitions are shown in some examples. For a quadtree partition, two of the three dimensions of the bounding box B (e.g., x, y, and z dimensions) can be divided into two parts, and this quadtree partition can produce four sub-boxes of the same size.

[0137] exist Figure 11In the example, the bounding box (1110) is partitioned in the x and y dimensions by quadtree segmentation, and the result of the quadtree segmentation is shown as four sub-boxes of the same size labeled 0, 2, 4 and 6.

[0138] In addition, Figure 11 In the example, the bounding box (1120) is partitioned in the x and z dimensions by quadtree segmentation, and the result of the quadtree segmentation is shown as four sub-boxes of the same size labeled 0, 1, 4 and 5.

[0139] In addition, Figure 11 In the example, the bounding box (1130) is partitioned in the y and z dimensions by quadtree segmentation, and the result of the quadtree segmentation is shown as four sub-boxes of the same size labeled 0, 1, 2 and 3.

[0140] Figure 12 Three binary tree partitions are shown in some examples. For a binary tree partition, one of the three dimensions of the bounding box B (e.g., x, y, and z dimensions) can be divided into two parts, and the binary tree partition can produce two sub-boxes of the same size.

[0141] exist Figure 12 In the example, the bounding box (1210) is partitioned in the x dimension by a binary tree partition, the result of which is shown as two sub-boxes of the same size labeled 0 and 4.

[0142] In addition, Figure 12 In the example, the bounding box (1220) is partitioned in the y-dimensional space by a binary tree segmentation, the result of which is shown as two sub-boxes of the same size labeled 0 and 2.

[0143] In addition, Figure 12 In the example, the bounding box (1230) is partitioned in the z-dimensional by binary tree partitioning, and the result of the binary tree partitioning is shown as two sub-boxes of the same size labeled 0 and 1.

[0144] Therefore, point clouds can be represented by a general tree structure with a suitable mixture of octree, quadtree, and binary tree partitions. To traverse the tree structure, a breadth-first search approach is used in some examples (e.g., a version of the TMC13 model). In some other examples, a depth-first search approach can also be used.

[0145] In some relevant examples (e.g., a version of TMC13), the tree structure is traversed in breadth-first order to encode occupancy codes. According to breadth-first order, a tree node in a level (e.g., a node in the tree structure) can only be accessed after all tree nodes in the previous level have been visited. In implementation examples, a First-In-First-Out (FIFO) data structure can be used.

[0146] Figure 13 A diagram illustrating the breadth-first coding order is shown. In the octree structure (1300), a shaded node corresponds to a non-empty cube space occupied by one or more points in the point cloud. The occupancy code of a shaded node can be... Figure 13 The encoding is performed in a breadth-first order from 0 to 8. In the breadth-first encoding order, octree nodes are visited level by level. According to one aspect of this disclosure, the breadth-first encoding order itself is not suitable for parallel processing because the current level must wait for the previous level to be encoded.

[0147] In some examples, the hybrid encoding order includes at least one level using a depth-first encoding order instead of a breadth-first encoding order. Therefore, in some implementations, nodes at levels using a depth-first encoding order and their descendant nodes can form subtree structures within a tree structure. When a level using a depth-first encoding order comprises multiple nodes, each corresponding to a non-empty cubic space, these multiple nodes and their respective descendant nodes can form multiple subtree structures. In some implementations, multiple subtree structures can be encoded in parallel.

[0148] Figure 14 A diagram illustrating the depth-first coding order of the octree structure (1400) is shown. The shaded nodes in the octree structure (1400) are nodes corresponding to non-empty cubic spaces. The octree structure (1400) can correspond to the same point cloud occupancy geometry as the octree structure (1300). The occupancy code of the shaded nodes can be... Figure 14 The encoding is performed in the depth-first encoding order from 0 to 8 as shown.

[0149] exist Figure 14 In the example, node "0" can be at any suitable segment depth, such as PD0, the child nodes of node "0" are at segment depth PD0+1, and the grandchild nodes of node "0" are at segment depth PD0+2. Figure 14In the example, the node at segment depth PD0+1 can be encoded in depth-first encoding order. The node at segment depth PD0+1 includes two nodes corresponding to the non-empty space. These two nodes and their respective descendant nodes can form a first sub-octree structure (1410) and a second sub-octree structure (1420), and these two nodes can be referred to as the root nodes of the two sub-octree structures, respectively.

[0150] Figure 14 The depth-first encoding order in the code is called the preorder version of the depth-first encoding order. In the preorder version of the depth-first encoding order, for each sub-octree structure, the root node of the sub-octree is visited first before visiting the child nodes of the sub-octree structure. Furthermore, the deepest node is visited first, and then backtracking is performed to the sibling nodes of the parent node.

[0151] exist Figure 14 In the example, the first sub-octree structure (1410) and the second sub-octree structure (1420) can be encoded in a parallel processing manner in some implementations. For example, nodes 1 and 5 can be accessed simultaneously. In some examples, a depth-first encoding order can be implemented using recursive programming or a stack data structure.

[0152] In some implementations, the hybrid encoding sequence begins with breadth-first traversal (encoding) and, after several levels of breadth-first traversal, depth-first traversal (encoding) can be enabled.

[0153] In some examples, prediction tree-based geometric coding can be used. Prediction tree-based geometric coding can be used when a prediction tree, such as a spanning tree, can be built for all points in the point cloud. In the example, for the prediction of a given point, all ancestors can be used.

[0154] Figure 15 A portion of the prediction tree (1510) for the point cloud (1500) across the rabbit is shown. In some examples, the position of a point in the prediction tree (1510) can be predicted based on the position of its parent point or based on the position of its parent point and the position of its grandparent point.

[0155] In some examples, trisoup-based geometric encoding can be used. Trisoup-based geometric encoding methods can represent the surface of an object as a series of triangular meshes. In the example, a trisoup-based geometric encoding method is applied to a dense surface point cloud. A decoder using trisoup-based geometric encoding can generate a point cloud from the mesh surface at a specified voxel granularity to ensure the density of the reconstructed point cloud. According to one aspect of this disclosure, trisoup-based geometric encoding may introduce distortion to the original point cloud, but it can provide the benefit of a reduced bitstream size.

[0156] Several techniques can be used for attribute coding. These techniques include prediction-based attribute coding, boosting-based attribute coding, and Region Adaptive Hierarchical Transformation (RAHT)-based attribute coding. The following descriptions will describe prediction-based attribute coding, boosting-based attribute coding, and RAHT-based attribute coding, respectively.

[0157] For prediction-based attribute encoding, let This represents a set of locations associated with points in a point cloud. For each location represented by multidimensional data, a one-dimensional Morton code can be determined that is associated with that location. Let... Indicates the difference between the two and the position. The associated Morton codes. Prediction-based attribute encoding includes a sorting step, followed by an encoding step. In the sorting step, the points in the point cloud are sorted, for example, in ascending order, according to the associated Morton codes. For example, let I denote the array of indices of the points sorted according to the sorting step.

[0158] In the encoding step, the encoder / decoder can compress / decompress points individually in an order defined by I during iterations. In each iteration i, points P are selected according to the order defined by I. i For P i The distances to multiple previous points in this sequence are analyzed. For example, s (e.g., s=64) represents the number of previous points that have been analyzed. Based on the analysis, P is selected. i The k (e.g., k=3) nearest neighbors are used for attribute prediction. Specifically, in some examples, this can be achieved by using the k nearest neighbors... Linear interpolation of the attributes is used to predict the attributes of point i (a). i The attribute is weighted based on the distance to the nearest neighbor point i. In the example, on the decoder side, let Let k be the k nearest neighbors of the current point i. Represents the decoded / reconstructed attribute values ​​of the nearest neighbors, and lets This represents the distance of the nearest neighbor from the current point i. Then, the predicted attribute value of the current point i can be calculated based on the attribute values ​​of the nearest neighbors and their distances from the current point, according to equation (1). :

[0159] Equation (1)

[0160] For boosting-based attribute encoding, additional steps are applied during prediction-based encoding. In the example, two additional steps are used: a first step called the update operator and a second step called adaptive quantization.

[0161] Figure 16A diagram is shown illustrating the use of a direct / forward transform architecture (1600) in lift-based attribute coding on the encoder side of G-PCC. In some examples, to facilitate prediction and update, the direct / forward transform architecture (1600) includes multiple prediction / update (P / U) decomposition stages to decompose the attribute signal. At each of the multiple P / U decomposition stages, the signal (approximate signal) from the previous stage is partitioned into two highly correlated groups. In the example, in the lift-based attribute coding scheme of G-PCC, partitioning is performed by utilizing a hierarchy of details (LoD) structure, in which such high correlation between levels is desired, and each level is constructed by nearest neighbor search to organize the non-uniform point cloud into structured data. The P / U decomposition stage at stage (N) produces a detail signal D(N) and an approximate signal L'(N), which are further decomposed into D(N-1) and L'(N-1) in the next stage (e.g., stage (N-1)). The decomposition is repeated until the base approximate signal L'(0) is obtained in the example. Therefore, instead of encoding the input attribute signal itself, which consists of various levels of detail, the detail signals D(N), D(N-1), ..., D(0) and the basic approximation signal L'(0) can be encoded in a lifting-based attribute encoding scheme.

[0162] Note that the application of the P / U decomposition stage can generate sparse subbands in the coefficients of D(N), ..., D(0), thereby providing transform coding gain advantages.

[0163] Figure 17 A diagram is shown illustrating the use of an inverse transform architecture (1700) in boost-based attribute coding on the decoder side of G-PCC. The inverse transform architecture (1700) includes multiple prediction / update (P / U) merging stages to reconstruct the attribute signal. At each P / U merging stage, the detail signal is decoded and merged with the approximate signal from the previous stage to generate a new approximate signal for the next P / U merging stage.

[0164] For RAHT-based attribute encoding, an adaptive hierarchical transformation can be used.

[0165] Figure 18 A diagram of the RAHT forward transform architecture (1810) that can be used on the encoder side of the G-PCC and a diagram of the RAHT inverse transform architecture (1820) that can be used on the decoder side of the G-PCC are shown. Figure 18 In the example, and And w0 is the input coefficient F l+1,2n The sign of the weight, and w1 is F l+1,2n+1 The sign of the weight.

[0166] Based on some aspects of this disclosure, point clouds can be geometrically and attribute-encoded using node-based geometric coding and attribute coding methods. In tree-based geometric coding, point clouds are represented as general tree structures, for example, using octree segmentation, quadtree segmentation, and binary tree segmentation. The root node of the tree structure corresponds to the entire volume of the point cloud, while the intermediate nodes of the tree structure correspond to sub-volumes of the point cloud (or subtrees of the tree structure).

[0167] Return to reference Figure 10 An octree structure (1020) is used as an example of a tree structure. For convenience, node N0 is the root node and is at depth 0 in the tree structure. Splitting at a node can produce nodes with greater depths (e.g., increasing depth by one). For example, splitting the box corresponding to node N0 can produce a node at depth 1 in the tree structure. Splitting the subbox at depth (k-1) can produce a node at depth k in the tree structure. Splitting can be performed until all nodes are unit nodes; for example, in some examples, all three dimensions are of size 1.

[0168] In some implementations, instead of encoding attributes after the entire point cloud's geometry has been encoded, attributes can be encoded between the geometry encodings of the point cloud. In some examples, the geometry of the point cloud can be encoded up to a depth k, where k is a positive integer and can be specified by the encoder and transmitted in the encoded bitstream. In a first example, geometry encoding and attribute encoding can be performed at depth k in a sub-volume-based interleaving manner. Specifically, for example, for each occupied node at depth k, which corresponds to a sub-volume (or subtree) of the point cloud, the geometry of the points (or nodes in the subtree) within that sub-volume can be encoded first, and the attribute information of the points (or nodes in the subtree) within that sub-volume can be encoded after the geometry of the points (or nodes in the subtree). The encoding process can then move on to the next occupied node at depth k. Therefore, in the encoded bitstream, the attribute information of the first occupied node at depth k (e.g., the attribute information of the point in the first sub-volume corresponding to the first occupied node) is encoded before the geometric information of the second occupied node at depth k (e.g., the geometric information of the point in the second sub-volume corresponding to the second occupied node).

[0169] In the second example, geometry and attribute encoding can be performed at depth k in a node-based interleaving manner. For example, for each node at depth k and deeper than k, the node's geometry can be encoded first, followed by the node's attribute information. The encoding process can then proceed to the next node at depth k or deeper than k. Thus, in the encoded bitstream, the attribute information of the first node is encoded before the geometry information of the second node.

[0170] In the first example of subvolume-based interleaving at depth k and the second example of node-based interleaving, the node at depth k can be considered as the highest-level coding unit, similar to the Largest Coding Unit (LCU) used in the HEVC video coding standard. In some examples, in the field of point cloud coding, each node at depth k forms a separate subtree, and each subtree can be referred to as an LCU in point cloud coding. Point clouds can be encoded using LCU-based coding. In some examples, LCU-based coding can be referred to as node-based coding. LCU-based coding can include LCU-based geometric coding, as well as LCU-based geometric coding and attribute coding. LCU-based geometric coding can encode geometric information based on LCUs. LCU-based geometric coding and attribute coding can encode geometric and attribute information based on LCUs, for example, using subvolume-based interleaving, node-based interleaving, etc.

[0171] Figure 19 A tree structure (1900) corresponding to a point cloud is shown as an example of some examples according to this disclosure. Figure 19 In the code, nodes corresponding to non-empty spaces are colored gray and are called shadow nodes. Nodes corresponding to empty spaces are colored white and can be called blank nodes. Shadow nodes are numbered according to the encoding order.

[0172] exist Figure 19In the example, the node at depth 1 is an LCU. For example, the tree structure (1900) includes a first LCU (1910) and a second LCU (1920). Point clouds can be encoded using node-based geometric coding and attribute coding (also known as LCU-based geometric coding and attribute coding). In some examples of node-based geometric coding and attribute coding at depth 1 in a sub-volume-based interleaving manner, the geometric information of the nodes in the first LCU (1910) is encoded, and the attribute information of the nodes in the first LCU (1910) is encoded after the geometric information of the nodes in the first LCU (1910). Then, the geometric information of the nodes in the second LCU (1920) is encoded after the attribute information of the nodes in the first LCU (1910), and the attribute information of the nodes in the second LCU (1920) is encoded after the geometric information of the nodes in the second LCU (1920). For simplicity, in this disclosure, depth k (in Figure 19 The node at depth k (k=1) can be called the LCU. In this disclosure, the LCU and the node at depth k are interchangeable.

[0173] In some examples, node-based geometry encoding and attribute encoding are performed using a node-based interleaving approach. For instance, the geometry of node 1 is encoded, and then the attribute information of node 1 is encoded after the geometry of node 1. Then, the geometry of node 2 is encoded after the attribute information of node 1, and then the attribute information of node 2 is encoded after the geometry of node 2.

[0174] For simplicity, we will use a quadtree partition in the tree structure (1900) for illustration. Note that other partitions, such as octree partitions and binary tree partitions, can also be used in the tree structure.

[0175] According to one aspect of this disclosure, on the encoder side, a bitstream generated for the geometry and attributes of each node (e.g., LCU) can be transmitted without waiting for the geometry encoding of the entire point cloud to be completed. On the decoder side, the decoder can decode the points in the corresponding node (e.g., LCU) and display the decoded points without waiting for the geometry encoding of the entire point cloud to be completed. Therefore, low-latency encoding and decoding can be achieved by using node-based geometry encoding and attribute encoding.

[0176] Note that occupied nodes (e.g., LCUs) at depth k can be encoded in any suitable order. In the example, occupied nodes (e.g., LCUs) at depth k are encoded in Morton order. In another example, occupied nodes at depth k are encoded in a space-filling order other than Morton code.

[0177] In some examples, the geometric and attribute encoding of an LCU does not depend on information from its neighboring LCUs. Specifically, prediction / reference across LCU boundaries is disabled, and the context and history information for each LCU is reinitialized. History information can include geometric occupancy information of nodes with depths smaller than k. Therefore, LCUs can be encoded or decoded independently (referred to as LCU encoding independence). LCU encoding independence can achieve maximum parallelism at depth k, i.e., LCU-level parallel encoding and decoding.

[0178] In some other examples, the geometric and attribute encoding of an LCU can depend on information from its nearest neighbors (e.g., neighboring LCUs). For instance, the geometric and attribute encoding of an LCU may depend on information about its encoded neighboring nodes and their encoded child nodes (a phenomenon known as LCU encoding dependency). LCU encoding dependency can contribute to better compression efficiency.

[0179] In some implementations, parallel node-based (LCU-based) coding techniques can be used for point cloud coding. To achieve parallel node-based (LCU-based) coding, the geometric and attribute coding of the LCU cannot rely on information from neighboring LCUs. To enable parallel processing and avoid uncertainty, prediction / reference across LCU boundaries is disabled, and for each LCU, the coding state information, such as context and history information, needs to be reinitialized to a known coding state consistent on both the encoder and decoder sides.

[0180] In the implementation, at each LCU, the encoding state, such as the context for entropy encoding, geometric occupancy history information, and other necessary state information for LCU-based encoding, can be set to an initial state, which is the encoding state at the start of point cloud encoding.

[0181] In another implementation, instead of using an initial state, suitable intermediate encoding states can be stored, such as encoding states prior to reaching the first LCU (e.g., after encoding at tree depth (k-1) is complete). These encoding states include context and geometric occupancy history information for entropy encoding, as well as other necessary state information for LCU-based encoding. Nodes at tree depth k can be encoded according to LCU-based encoding. When encoding each LCU, the encoding state can be set based on the stored intermediate encoding states. Then, encoding based on parallel nodes (LCUs) can be implemented. Furthermore, the stored intermediate encoding states can help improve encoding performance compared to the initial encoding state at the start of encoding processing.

[0182] Figure 20A flowchart is shown outlining an encoding process (2000) for encoding geometric information and / or attribute information according to some embodiments of this disclosure.

[0183] At (S2010), the encoding of the tree structure at depth (k-1) is completed before the encoding of any occupied nodes at depth k. The occupied nodes at depth k of the tree structure are called LCU nodes, and each LCU node corresponds to a subtree in the tree structure and can be encoded using LCU-based encoding (e.g., LCU-based geometric encoding, LCU-based geometric encoding, and attribute encoding). In some examples, LCU nodes can be encoded in parallel.

[0184] At (S2020), the variable N is set to the number of LCUs (e.g., occupied nodes at depth k) in the tree structure, and the variable I is initialized, for example, to 0. The variable I is used to follow the index of the LCUs.

[0185] At (S2030), variable I is compared with variable N. If variable I is less than variable N, the process proceeds to (S2040); otherwise, the LCU encoding is complete, and the process can proceed to another encoding process.

[0186] At (S2040), when variable I equals 0, the corresponding LCU with the value of variable I is the first LCU, and then the process proceeds to (S2050); otherwise, the process proceeds to (S2060).

[0187] At (S2050), the encoding state is stored (including context and geometric occupancy history information for entropy encoding, as well as other necessary state information for LCU-based encoding).

[0188] At (S2060), the encoding state is restored according to the stored encoding state.

[0189] At (S2070), LCUs with values ​​indexed as variable I are encoded using LCU-based encoding.

[0190] At (S2080), variable I is incremented by 1, and the process returns to (S2030).

[0191] Note that the processing can be modified appropriately (2000).

[0192] Figure 21 A flowchart is shown outlining another encoding process (2100) for encoding geometric information and / or attribute information according to some embodiments of the present disclosure.

[0193] At (S2110), the encoding of the tree structure at depth (k-1) is completed before the encoding of the occupied nodes at depth k. The occupied nodes at depth k of the tree structure are called LCU nodes, and each LCU node corresponds to a subtree in the tree structure and can be encoded using LCU-based encoding (e.g., LCU-based geometric encoding, LCU-based geometric encoding, and attribute encoding). In some examples, LCU nodes can be encoded in parallel.

[0194] At (S2120), the variable N is set to the number of LCUs at depth k of the tree structure (e.g., the number of occupied nodes at depth k), and the variable I is initialized, for example, to 0. The variable I is used to follow the index of the LCU.

[0195] At (S2130), the encoding state (including context and geometric occupancy history information for entropy encoding, as well as other necessary state information for LCU-based encoding) is stored.

[0196] At (S2140), variable I is compared with variable N. If variable I is less than variable N, the process proceeds to (S2150); otherwise, the LCU encoding is complete, and the process can proceed to another encoding process.

[0197] At (S2150), the encoding state is restored according to the stored encoding state.

[0198] At (S2160), LCUs with values ​​indexed as variable I are encoded using LCU-based geometric encoding and attribute encoding.

[0199] At (S2170), variable I is incremented by 1, and the process returns to (S2140).

[0200] According to some aspects of this disclosure, flags are used to provide additional information to control the storage and retrieval of encoding status information used for LCU-based encoding. The techniques disclosed in this disclosure can be applied to any suitable software, standard, or system used for PCC.

[0201] In parallel node-based (LCU-based) encoding implementations, the encoding state immediately preceding the first LCU can be saved. The saved encoding state can be restored when encoding each LCU. Therefore, the encoding state of the LCUs can be known and consistent on both the encoder and decoder sides. In this way, parallel node-based (LCU-based) encoding can be implemented. In some examples, parallel encoding is not required, and for the purpose of reducing memory and operations, it is not necessary to save and restore encoding state information.

[0202] According to one aspect of this disclosure, flags can be used to control whether to save and restore the encoded state. In some examples, flags can be added in high-level syntax—such as in sequence parameter sets, geometry parameter sets, or slice headers—to control whether to save and restore the encoded state. This flag may be referred to as a state storage flag.

[0203] Figure 22A A syntax table in a geometry header (2200A) according to some embodiments of this disclosure is shown. The geometry header (2200A) includes a set of geometry parameters that can be transmitted in an encoded bitstream that transfers the point cloud from the encoder side to the decoder side. The geometry parameter set includes flags (2210), such as "gps_save_state_flag," to control whether the encoded state is saved and restored. Specifically, in some examples, in response to the flag "gps_save_state_flag" having a binary value of "1," encoded state information prior to the first LCU is stored, and the encoded state can be restored based on the stored encoded state information before encoding each LCU (or each subsequent LCU). On the other hand, in response to the flag "gps_save_state_flag" having a binary value of "0," no encoded state storage and restoration operation is performed.

[0204] In some examples, LCU encoding only includes geometry encoding, and the state storage flag is included in the geometry header, such as the geometry header (2200A).

[0205] In some examples, LCU encoding includes both geometric encoding and attribute encoding, and state storage flags are included in the sequence parameter set to control both geometric and attribute encoding in LCU encoding.

[0206] Figure 22B A syntax table in a sequence header (2200B) according to some embodiments of this disclosure is shown. The sequence header (2200B) includes a sequence parameter set that can be transmitted in the encoded bitstream that transmits the point cloud from the encoder side to the decoder side. The sequence parameter set includes flags (2220), such as "save_state_flag", to control whether the encoding state is saved and restored. Specifically, in some examples, in response to the flag "save_state_flag" having a binary value of "1", encoding state information prior to the first LCU is stored, and the encoding state can be restored based on the stored encoding state information before encoding each LCU (or each subsequent LCU). On the other hand, in response to the flag "save_state_flag" having a binary value of "0", no encoding state storage and restoration operation is performed.

[0207] Figure 23 A flowchart outlining an encoding process (2300) for encoding geometric and / or attribute information according to some embodiments of this disclosure is shown.

[0208] At (S2310), the encoding of the tree structure at depth (k-1) is completed before the encoding of any occupied nodes at depth k. The occupied nodes at depth k of the tree structure are called LCU nodes, and each LCU node corresponds to a subtree in the tree structure and can be encoded using LCU-based encoding (e.g., LCU-based geometric encoding, LCU-based geometric encoding, and attribute encoding). In some examples, LCU nodes can be encoded in parallel. In some examples, LCUs are encoded one by one, rather than through parallel processing.

[0209] At (S2320), the variable N is set to the number of LCUs (e.g., occupied nodes at depth k) in the tree structure, and the variable I is initialized, for example, to 0. The variable I is used to follow the index of the LCU.

[0210] At (S2330), variable I is compared with variable N. If variable I is less than variable N, the process proceeds to (S2340); otherwise, the LCU encoding is complete, and the process can proceed to another encoding process.

[0211] At (S2340), when variable I equals 0, the corresponding LCU with the value of variable I is the first LCU, and then the process proceeds to (S2350); otherwise, the process proceeds to (S2360).

[0212] At (S2350), if the flag referred to as the state storage flag—for example, the flag “gps_save_state_flag” in the case of LCU-based geometric coding, or the flag “save_state_flag” in the case of LCU-based geometric coding and attribute coding—is equal to the binary value “1”, then the process proceeds to (S2355) to store the coding state; otherwise, the process proceeds to (S2370) without storing the coding state.

[0213] At (S2355), the encoding state (including context and geometric occupancy history information for entropy encoding, as well as other necessary state information for LCU-based encoding) is stored.

[0214] At (S2360), if the state storage flag—for example, the flag “gps_save_state_flag” in the case of LCU-based geometric coding, or the flag “save_state_flag” in the case of LCU-based geometric coding and attribute coding—is equal to the binary value “1”, then the process proceeds to (S2365) to restore the coding state; otherwise, the process proceeds to (S2370) without restoring the coding state.

[0215] At (S2365), the encoding state is restored based on the stored encoding state.

[0216] At (S2370), LCUs with values ​​indexed as variable I are encoded using LCU-based encoding.

[0217] At (S2380), variable I is incremented by 1, and the process returns to (S2330).

[0218] Note that in some examples, when the state storage flag (e.g., the flag "gps_save_state_flag" in the case of LCU-based geometry encoding, or the flag "save_state_flag" in the case of LCU-based geometry encoding and attribute encoding) has a binary value of "1", the LCUs are encoded in parallel; and when the state storage flag (e.g., the flag "gps_save_state_flag" in the case of LCU-based geometry encoding, or the flag "save_state_flag" in the case of LCU-based geometry encoding and attribute encoding) has a binary value of "0", the LCUs are encoded one by one, rather than through parallel processing.

[0219] Note that the processing (2300) can be modified appropriately.

[0220] Figure 24 A flowchart is shown outlining another encoding process (2400) for encoding geometric information and / or attribute information according to some embodiments of this disclosure.

[0221] At (S2410), the encoding of the tree structure at depth (k-1) is completed before the encoding of any occupied nodes at depth k. The occupied nodes at depth k of the tree structure are called LCU nodes, and each LCU node corresponds to a subtree in the tree structure and can be encoded using LCU-based encoding (e.g., LCU-based geometric encoding, LCU-based geometric encoding, and attribute encoding). In some examples, LCU nodes can be encoded in parallel. In some examples, LCUs are encoded one by one, rather than through parallel processing.

[0222] At (S2420), the variable N is set to the number of LCUs at depth k of the tree structure (e.g., the number of occupied nodes at depth k), and the variable I is initialized, for example, to 0. The variable I is used to follow the index of the LCU.

[0223] At (S2430), the encoding state is stored (including context and geometry occupancy history information for entropy encoding, as well as other necessary state information for LCU-based encoding). Note that in some examples, the encoding state is stored when a flag called the state storage flag (e.g., the flag "gps_save_state_flag" in the case of LCU-based geometry encoding, or the flag "save_state_flag" in the case of LCU-based geometry and attribute encoding) is equal to the binary value "1".

[0224] At (S2440), variable I is compared with variable N. If variable I is less than variable N, the process proceeds to (S2445); otherwise, the LCU encoding is complete, and the process can proceed to another encoding process.

[0225] At (S2445), if the flag referred to as the state storage flag (e.g., the flag "gps_save_state_flag" in the case of LCU-based geometric coding, or the flag "save_state_flag" in the case of LCU-based geometric coding and attribute coding) is equal to the binary value "1", then the process proceeds to (S2450) to restore the coding state; otherwise, the process proceeds to (S2460) without restoring the coding state.

[0226] At (S2450), the encoding state is restored based on the stored encoding state.

[0227] At (S2460), LCUs with values ​​indexed as variable I are encoded using LCU-based geometric coding or LCU-based geometric coding and attribute coding.

[0228] At (S2470), variable I is incremented by 1, and the process returns to (S2440).

[0229] Figure 25A flowchart outlining a process (2500) according to an embodiment of the present disclosure is shown. The process (2500) can be used during point cloud decoding. In various embodiments, the process (2500) is executed by processing circuitry such as processing circuitry in a terminal device (110); processing circuitry performing the functions of an encoder (203) and / or decoder (210), an encoder (700) and / or decoder (800), etc. In some embodiments, the process (2500) is implemented as software instructions, and thus, when the processing circuitry executes the software instructions, the processing circuitry executes the process (2500). The process begins at (S2501) and proceeds to (S2510).

[0230] At (S2510), a flag indicating an enable / disable control for saving the encoding state in the maximum coding unit (LCU)-based encoding of the point cloud is determined. In some implementations, the flag is decoded from at least one of the sequence parameter set in the sequence header, the geometry parameter set in the geometry header, and the slice header.

[0231] At (S2520), encoding state information prior to encoding of the first LCU is stored. In this example, the encoding state information is stored in response to a flag indicating that control is enabled. In another example, the storage of encoding state information is skipped in response to a flag indicating that control for saving the encoding state is disabled. In this example, the encoding state information is stored regardless of the flag.

[0232] In some examples, the encoded state information includes at least one of context information, historical geometric occupancy information, and other suitable historical information.

[0233] At (S2530), in response to a flag indicating that control is enabled, the encoding state is restored based on stored encoding state information before encoding of the second LCU. In some examples, in response to a flag indicating that control for saving the encoding state is disabled, the restoration of the encoding state based on stored encoding state information before encoding of the second LCU can be skipped.

[0234] In some implementations, flags, such as the flag "gps_save_state_flag", are decoded from the geometry parameter group in the geometry header. Then, the encoding state information is stored before the geometry encoding of the first LCU; and the encoding state is restored based on the stored encoding state information before the geometry encoding of the second LCU.

[0235] In some implementations, flags, such as the "save_state_flag", are decoded from the sequence parameter group in the sequence header. Then, the encoding state information is stored before the geometric and attribute encoding of the first LCU; and the encoding state is restored based on the stored encoding state information before the geometric and attribute encoding of the second LCU.

[0236] In some implementations, the LCUs can be encoded in parallel. For example, the encoding of the second LCU can begin before the encoding of the first LCU is completed.

[0237] Then, the process proceeds to (S2599) and terminates.

[0238] The techniques disclosed in this disclosure can be used individually or in any combination in any order. Furthermore, each of the techniques (e.g., methods, implementations), encoders, and decoders can be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In some examples, one or more processors execute a program stored on a non-transitory computer-readable medium.

[0239] The techniques described above can be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. For example, Figure 26 A computer system (2600) suitable for implementing certain embodiments of the disclosed subject matter is shown.

[0240] Computer software can be encoded using any suitable machine code or computer language. Machine code or computer language can be subjected to mechanisms such as assembly, compilation, and linking to create code that includes instructions. These instructions can be executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., or through interpretation, microcode execution, etc.

[0241] The instructions can be executed on various types of computers or their components, including personal computers, tablets, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0242] Figure 26 The components shown for the computer system (2600) are exemplary in nature and are not intended to impose any limitation on the scope or functionality of computer software implementing embodiments of this disclosure. The configuration of the components should also not be construed as having any dependency or requirement relating to any one or a combination of components shown in the exemplary embodiments of the computer system (2600).

[0243] The computer system (2600) may include certain human-machine interface input devices. Such human-machine interface input devices can respond to input from one or more human users via, for example, tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., speech, tapping), visual input (e.g., gestures), and olfactory input (not depicted). The human-machine interface devices can also be used to capture certain media that are not necessarily directly related to human conscious input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images obtained from still image capturing devices), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0244] The input human-machine interface device may include one or more of the following (only one of each is depicted): keyboard (2601), mouse (2602), touchpad (2603), touch screen (2610), data glove (not shown), joystick (2605), microphone (2606), scanner (2607), and camera device (2608).

[0245] The computer system (2600) may also include certain human-machine interface output devices. Such human-machine interface output devices can stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include: haptic output devices (e.g., haptic feedback via a touchscreen (2610), data gloves (not shown), or joystick (2605), but haptic feedback devices that are not used as input devices may also exist); audio output devices (e.g., speakers (2609), headphones (not depicted)); visual output devices (e.g., screens (2610), including CRT (Cathode Ray Tube) screens, LCD (Liquid Crystal Display) screens, plasma screens, OLED (Organic Light Emitting Diode) screens, each with or without touchscreen input capability, each with or without haptic feedback capability—some of which may be able to output two-dimensional visual output or more than three-dimensional output in a manner such as stereoscopic image output; virtual reality glasses (not depicted); holographic displays and ashtrays (not depicted); and printers (not depicted).

[0246] The computer system (2600) may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM (Read Only Memory, ROM) / RW (2620) having media such as CD (Compact Disc, CD) / DVD (Digital Video Disk, DVD) (2621), thumb drives (2622), removable hard disk drives or solid-state drives (2623), conventional magnetic media such as magnetic tape and floppy disks (not depicted), devices based on dedicated ROM / ASIC (Application Specific Integrated Circuit, ASIC) / PLD (Programmable Logic Device, PLD) such as security dongles (not depicted), etc.

[0247] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0248] The computer system (2600) may also include interfaces (2654) to one or more communication networks (2655). Networks may be, for example, wireless, wired, or optical. Networks may also be local area, wide area, metropolitan area, vehicular and industrial, real-time, latency-tolerant, etc. Examples of networks include: local area networks, such as Ethernet and wireless LANs; cellular networks, including GSM (Global System for Mobile Communications), 3G (the third generation), 4G (the fourth generation), 5G (the fifth generation), LTE (Long Term Evolution), etc.; wired or wireless wide area digital TV networks, including cable TV, satellite TV, and terrestrial broadcast TV; and vehicle and industrial networks, including CANBus (Controller Area Network Bus), etc. Some networks typically require external network interface adapters that attach to certain general-purpose data ports or peripheral buses (2649) (e.g., USB ports of the computer system (2600)); others are typically integrated into the core of the computer system (2600) via attachment to a system bus (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system (2600) can communicate with other entities. Such communication can be one-way receiving (e.g., broadcasting TV), one-way transmitting (e.g., to a CANBus device), or bidirectional, e.g., using a local area digital network or a wide area digital network to other computer systems. Certain protocols and protocol stacks can be used on each of these networks and network interfaces as described above.

[0249] The human-machine interface device, human-accessible storage device and network interface mentioned above can be attached to the core (2640) of the computer system (2600).

[0250] The core (2640) may include one or more Central Processing Units (CPUs) (2641), Graphics Processing Units (GPUs) (2642), Dedicated Programmable Processing Units in the form of Field Programmable Gate Areas (FPGAs) (2643), hardware accelerators (2644) for certain tasks, graphics adapters (2650), etc. These devices, along with Read-Only Memory (ROM) (2645), Random Access Memory (2646), and internal mass storage devices such as internal non-user-accessible hard disk drives, SSDs, etc. (2647), may be connected via a system bus (2648). In some computer systems, the system bus (2648) may be accessed as one or more physical plugs to allow for expansion with additional CPUs, GPUs, etc. Peripheral devices may be attached directly or via a peripheral bus (2649) to the core's system bus (2648). In the example, a screen (2610) may be connected to a graphics adapter (2650). Peripheral bus architectures include PCI (Peripheral Component Interconnect, PCI) and USB (Universal Serial Bus, USB).

[0251] The CPU (2641), GPU (2642), FPGA (2643), and accelerator (2644) can execute certain instructions that, when combined, constitute the computer code mentioned above. This computer code can be stored in ROM (2645) or RAM (2646). Transient data can also be stored in RAM (2646), while permanent data can be stored, for example, in an internal mass storage device (2647). Fast storage and retrieval of any memory device can be achieved by using a cache memory, which can be closely associated with one or more CPUs (2641), GPUs (2642), mass storage devices (2647), ROMs (2645), RAMs (2646), etc.

[0252] Computer-readable media may contain computer code for performing operations of various computer implementations. The media and computer code may be specifically designed and constructed for the purposes of this disclosure, or the media and computer code may be of a type known and available to those skilled in the art of computer software.

[0253] By way of example and not limitation, a computer system (2600) having an architecture, and particularly a core (2640), can provide the functionality of a processor (including a CPU, GPU, FPGA, accelerator, etc.) to execute software embodied in one or more tangible computer-readable media. Such computer-readable media can be media associated with user-accessible mass storage devices as described above, as well as certain storage devices of the core (2640) having non-transitory characteristics, such as a mass storage device (2647) or ROM (2645) within the core. Software implementing various embodiments of this disclosure can be stored in such devices and executed by the core (2640). Depending on specific needs, the computer-readable media may include one or more memory devices or chips. The software can cause the core (2640), and particularly the processors therein (including CPUs, GPUs, FPGAs, etc.), to execute specific processes or specific portions of specific processes described herein, including defining data structures stored in RAM (2646) and modifying such data structures according to the processes defined by the software. Alternatively or concurrently, the computer system may provide functionality provided by or otherwise embodied in circuitry (e.g., an accelerator (2644)) due to hardwired logic, which may replace or operate with software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, references to software may include logic, and references to logic may include software. Where appropriate, references to computer-readable media may include circuitry (e.g., an integrated circuit (IC)) storing software for execution, circuitry embodying logic for execution, or both. This disclosure includes any suitable combination of hardware and software.

[0254] While several exemplary embodiments have been described in this disclosure, there are changes, substitutions, and various alternatives that fall within the scope of this disclosure. Therefore, it will be appreciated that, although not explicitly shown or described herein, those skilled in the art will be able to conceive of many systems and methods that embody the principles of this disclosure and are therefore within its spirit and scope.

Claims

1. A method for point cloud decoding, characterized in that, The method includes: A first flag is determined, which indicates an enable / disable control for whether to save the encoding state in the encoding of the point cloud based on the maximum coding unit (LCU); In response to the first flag indicating the enable control for whether to save the encoding state, the encoding state is stored.

2. The method according to claim 1, characterized in that, The method further includes: In response to the completion of depth (k-1) encoding of the tree structure of the point cloud, encoding state information is stored after the completion of depth (k-1) encoding and before the start of depth k encoding; and In response to the first flag indicating the enable control for whether to save the encoding state, the first LCU and the second LCU at depth k are encoded using the encoding state information restored after the encoding at depth (k-1) is completed and before the encoding at depth k begins, wherein the second LCU is different from the first LCU.

3. The method according to claim 1, characterized in that, The method further includes: In response to the first flag indicating a disabled control over whether to save the encoding state, the storage of the encoding state information is skipped.

4. The method according to claim 1, characterized in that, The method further includes: In response to the first flag indicating a disabled control over whether to save the encoding state, the restoration of the encoding state is skipped.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: The first flag is decoded from at least one of the sequence parameter set in the sequence header, the geometric parameter set in the geometry header, and the slice header.

6. The method according to any one of claims 2 to 4, characterized in that, The method further includes: Decode the first flag from the set of geometric parameters in the geometry header; Store the encoding state information before geometric encoding of the first LCU; and The encoding state is restored based on the stored encoding state information before geometric encoding of the second LCU.

7. The method according to any one of claims 2 to 4, characterized in that, The method further includes: Decode the first flag from the sequence parameter set in the sequence header; Store the encoding state information before performing geometric encoding and attribute encoding on the first LCU; and The encoding state is restored based on the stored encoding state information before geometric encoding and attribute encoding are performed on the second LCU.

8. The method according to any one of claims 2 to 4, characterized in that, The method further includes: The second LCU is encoded before the encoding of the first LCU is completed.

9. The method according to any one of claims 2 to 4, characterized in that, The method further includes: In response to the first flag indicating a disable control over whether to save the encoding state, the process of restoring the encoding state based on the stored encoding state information is skipped before encoding the second LCU.

10. The method according to claim 9, characterized in that, The method further includes: In response to the first flag indicating a disabled control over whether to save the encoding state, the storage of the encoding state information is skipped.

11. The method according to any one of claims 1 to 4, characterized in that, The encoded status information also includes historical geometric occupancy information.

12. A computer device, characterized in that, The computer device includes: One or more computer-readable non-transitory storage media configured to store computer program code; and One or more computer processors configured to access the computer program code and execute the method according to any one of claims 1 to 11 as instructed by the computer program code.

13. An apparatus for point cloud decoding, characterized in that, The apparatus includes a processing circuit configured to perform the method according to any one of claims 1 to 11.

14. A non-transitory computer-readable medium storing instructions, characterized in that, When executed by a computer, the instructions cause the computer to perform the method according to any one of claims 1 to 11.

15. A method for storing a bit stream, characterized in that, Generate a bit stream by performing the method of any one of claims 1-11; and store the bit stream.

16. A method for transmitting a bit stream, characterized in that, Generate a bit stream by performing the method of any one of claims 1-11; and transmit the bit stream.

17. A computer-readable storage medium storing a computer program / instructions and a bit stream thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1-11 to generate the bit stream.

Images