Point cloud decoding method, computer device, equipment and computer readable medium

Abstract

Various aspects of the present disclosure provide methods and apparatus for point cloud compression and decompression. In some examples, the apparatus for point cloud compression / decompression includes a processing circuit. In some examples, the processing circuit receives a bitstream carrying compressed data of a point cloud. The processing circuit determines that a current node in an octree structure conforms to an isolation pattern. The octree structure corresponds to a three-dimensional (3D) segmentation of the space of the point cloud. The processing circuit then determines a single isolated point flag for the current node based on information from one or more other nodes, the single isolated point flag indicating whether the current node is encoded as a single isolated point.

Description

[0001] Incorporation by reference

[0002] This application claims the benefit of priority to U.S. Patent Application No. 17 / 354,834, filed on June 22, 2021, entitled "METHOD AND APPARATUS FOR POINT CLOUD CODING," which claims the benefit of priority to U.S. Provisional Application No. 63 / 114,451, filed on November 16, 2020, entitled "METHODS OF CODING ISOLATED POINTS FOR POINT CLOUD CODING." The entire disclosure of the prior application is hereby incorporated by reference herein in its entirety. Technical Field

[0003] The present disclosure describes embodiments generally related to point cloud decoding, and more particularly to a method, computer device, apparatus, and non-transitory computer-readable medium for point cloud decoding. Background Art

[0004] The purpose of the background description provided herein is to generally present the context of the present disclosure. The work of the presently named inventors, to the extent that work is described in this background section, and aspects of the description that might not otherwise be considered prior art as of the filing date are not admitted, either explicitly or implicitly, to be prior art with respect to the present disclosure.

[0005] Various technologies have been developed to capture and represent the world in three-dimensional (3D) space, such as objects in the world, the environment in the world, and the like. 3D representations of the world can enable more immersive forms of interaction and communication. Point clouds can be used as 3D representations of the world. A point cloud is a set of points in 3D space, each with associated attributes, such as color, material properties, texture information, intensity attributes, reflectivity attributes, motion-related attributes, morphological attributes, and various other attributes. Such point clouds can include large amounts of data, and storage and transmission can be expensive and time-consuming. Summary of the Invention

[0006] According to an embodiment, a method for point cloud decoding is provided. The method includes: receiving, by a processor, a bitstream carrying compressed data of a point cloud; determining, by the processor, whether a current node in an octree structure conforms to an isolation pattern, the octree structure corresponding to a three-dimensional (3D) segmentation of a space of the point cloud; and determining, by the processor, a single isolated point flag for the current node based on information about another node, the single isolated point flag indicating whether the current node is encoded as a single isolated point.

[0007] According to an embodiment, a computer device is provided. The computer device includes: one or more computer-readable non-transitory storage media configured to store computer program code; and one or more computer processors configured to access the computer program code and execute the above method for point cloud decoding as instructed by the computer program code.

[0008] According to an embodiment, a device for point cloud decoding is provided. The device includes processing circuitry. The processing circuitry is configured to: receive a bitstream carrying compressed data of a point cloud; determine whether a current node in an octree structure corresponds to an isolation pattern, the octree structure corresponding to a three-dimensional (3D) segmentation of a space of the point cloud; and determine a single isolated point flag for the current node based on information about another node, the single isolated point flag indicating whether the current node is encoded as a single isolated point.

[0009] According to an embodiment, a non-transitory computer-readable storage medium is provided, which stores instructions that, when executed by a computer for point cloud decoding, cause the computer to perform the above method for point cloud decoding.

[0010] The point cloud decoding method, computer device, apparatus and non-transitory computer-readable medium of the present invention provide a technology for encoding a single isolated point flag, which is used to indicate whether the current node is encoded as a single isolated point. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Other features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which:

[0012] Figure 1 is a schematic diagram of a simplified block diagram of a communication system according to an embodiment;

[0013] Figure 2 is a schematic diagram of a simplified block diagram of a streaming system according to an embodiment;

[0014] Figure 3 shows a block diagram of an encoder for encoding a point cloud frame according to some embodiments;

[0015] Figure 4 shows a block diagram of a decoder for decoding a compressed bitstream corresponding to a point cloud frame according to some embodiments;

[0016] Figure 5 is a schematic diagram of a simplified block diagram of a video decoder according to an embodiment;

[0017] Figure 6 is a schematic diagram of a simplified block diagram of a video encoder according to an embodiment;

[0018] Figure 7 shows a block diagram of an encoder for encoding a point cloud frame according to some embodiments;

[0019] Figure 8 shows a block diagram of a decoder for decoding a compressed bitstream corresponding to a point cloud frame according to some embodiments;

[0020] Figure 9 A diagram illustrating partitioning of a cube based on an octree partitioning technique according to some embodiments of the present disclosure is shown.

[0021] Figure 10 An example of octree partitioning and an octree structure corresponding to the octree partitioning according to some embodiments of the present disclosure is shown.

[0022] Figure 11 A syntax table according to some embodiments of the present disclosure is shown.

[0023] Figure 12 A flowchart outlining an example of a process according to some embodiments is shown.

[0024] Figure 13 is a schematic diagram of a computer system according to an embodiment. DETAILED DESCRIPTION

[0025] Various aspects of the present disclosure provide point cloud coding (PCC) technology. PCC can be performed according to various schemes, such as a geometry-based scheme called G-PCC (Geometry based Point Cloud Coding, G-PCC), a video coding-based scheme called V-PCC (Video based Point Cloud Coding, V-PCC), and the like. According to some aspects of the present disclosure, G-PCC directly encodes 3D geometry and is a purely geometry-based method that is not much related to video coding, while V-PCC is largely based on video coding. For example, V-PCC can map the points of a 3D cloud to the pixels of a 2D grid (image). The V-PCC scheme can utilize a general video codec for point cloud compression. The Moving Picture Experts Group (MPEG) is studying the G-PCC standard and the V-PCC standard that use the G-PCC scheme and the V-PCC scheme, respectively.

[0026] Aspects of the present disclosure provide techniques for encoding a single outlier flag that indicates whether a current node is encoded as a single outlier.

[0027] Point clouds can be used in a wide range of applications. For example, point clouds can be used for object detection and localization in autonomous vehicles, for map construction in Geographic Information Systems (GIS), and for visualizing and archiving cultural heritage objects and collections in cultural heritage.

[0028] Hereinafter, a point cloud may generally refer to a set of points in 3D space, each point having associated attributes, such as color, material properties, texture information, intensity attributes, reflectivity attributes, motion-related attributes, morphological attributes, and various other attributes. A point cloud can be used to reconstruct an object or scene as a combination of such points. These points can be captured using multiple cameras, depth sensors, and / or lidars in various settings, and may consist of thousands to billions of points to realistically represent the reconstructed scene. A block may generally refer to a continuous subset of a surface described by a point cloud. In an example, a block includes points having surface normal vectors that deviate from each other by less than a threshold amount.

[0029] Compression techniques can reduce the amount of data required to represent point clouds, enabling faster transmission or reduced storage. Therefore, technologies for lossy point cloud compression are needed for real-time communication and six-degrees-of-freedom (6DoF) virtual reality. Furthermore, technologies for lossless point cloud compression are being sought for dynamic mapping applications such as autonomous driving and cultural heritage.

[0030] According to one aspect of the present disclosure, the main principle behind V-PCC is to compress the geometry, occupancy, and texture of a dynamic point cloud into three separate video sequences using existing video codecs. The additional metadata required to interpret these three video sequences is compressed separately. A small portion of the overall bitstream is metadata, which can be efficiently encoded / decoded using software implementations. Most of the information is handled by the video codec.

[0031] Figure 1 A simplified block diagram of a communication system (100) according to an embodiment of the present disclosure is shown. The communication system (100) includes a plurality of terminal devices that can communicate with each other via, for example, a network (150). For example, the communication system (100) includes a pair of terminal devices (110) and (120) interconnected via the network (150). Figure 1In the example of , a first pair of terminal devices (110) and (120) can perform a one-way transmission of point cloud data. For example, the terminal device (110) can compress a point cloud (e.g., points representing a structure) captured by a sensor (105) connected to the terminal device (110). The compressed point cloud can be transmitted to another terminal device (120) via a network (150), for example, in the form of a bit stream. The terminal device (120) can receive the compressed point cloud from the network (150), decompress the bit stream to reconstruct the point cloud, and appropriately display the reconstructed point cloud. One-way data transmission may be common in media service applications, etc.

[0032] exist Figure 1 In the example of , the terminal devices (110) and (120) may be shown as a server and a personal computer, but the principles of the present disclosure may not be limited thereto. Implementations of the present disclosure are applicable to laptop computers, tablet computers, smart phones, game consoles, media players and / or dedicated three-dimensional (3D) equipment. Network (150) represents any number of networks that transmit compressed point clouds between the terminal devices (110) and (120). The network (150) may include, for example, cables (wired) and / or wireless communication networks. The network (150) may exchange data in circuit-switched channels and / or packet-switched channels. Representative networks include telecommunication networks, local area networks, wide area networks and / or the Internet. For the purposes of this discussion, unless explained below, the architecture and topology of the network (150) may not be important for the operation of the present disclosure.

[0033] Figure 2 Shown is a simplified block diagram of a streaming system (200) according to an embodiment. Figure 2 The disclosed subject matter may be equally applicable to other point cloud-enabled applications, such as 3D telepresence applications, virtual reality applications, and the like.

[0034] The streaming system (200) may include a capture subsystem (213). The capture subsystem (213) may include a point cloud source (201), such as a Light Detection and Ranging (LIDAR) system, a 3D camera, a 3D scanner, a graphics generation component in software that generates an uncompressed point cloud, and a similar graphics generation component that generates, for example, an uncompressed point cloud (202). In an example, the point cloud (202) includes points captured by the 3D camera. The point cloud (202) is depicted as a thick line to emphasize the high amount of data compared to the compressed point cloud (204) (a bitstream of the compressed point cloud). The compressed point cloud (204) may be generated by an electronic device (220) that includes an encoder (203) coupled to the point cloud source (201). The encoder (203) may include hardware, software, or a combination thereof to implement or embody aspects of the disclosed subject matter as described in more detail below. The compressed point cloud (204) (or a bitstream of the compressed point cloud (204)), depicted as a thin line to emphasize the lower amount of data compared to the stream of point cloud (202), can be stored on the streaming server (205) for future use. One or more streaming client subsystems, such as Figure 2 The client subsystems (206) and (208) in the embodiment of the present invention can access the streaming server (205) to retrieve the copies (207) and (209) of the compressed point cloud (204). The client subsystem (206) can include a decoder (210), for example, in an electronic device (230). The decoder (210) decodes the input copy (207) of the compressed point cloud and creates an output stream of a reconstructed point cloud (211) that can be rendered on a rendering device (212).

[0035] Note that the electronic device (220) and the electronic device (230) may include other components (not shown). For example, the electronic device (220) may include a decoder (not shown), and the electronic device (230) may also include an encoder (not shown).

[0036] In some streaming systems, the compressed point clouds (204), (207), and (209) (e.g., a bitstream of the compressed point clouds) may be compressed according to some standard. In some examples, a video coding standard is used in the compression of the point clouds. Examples of such standards include High Efficiency Video Coding (HEVC), Versatile Video Coding (VVC), and the like.

[0037] Figure 3A block diagram of a V-PCC encoder (300) for encoding a point cloud frame according to some embodiments is shown. In some embodiments, the V-PCC encoder (300) can be used in the communication system (100) and the streaming system (200). For example, the encoder (203) can be configured and operated in a manner similar to the V-PCC encoder (300).

[0038] The V-PCC encoder (300) receives a point cloud frame as an uncompressed input and generates a bitstream corresponding to the compressed point cloud frame. In some embodiments, the V-PCC encoder (300) may receive the point cloud frame from a point cloud source such as the point cloud source (201).

[0039] exist Figure 3 In an example, the V-PCC encoder (300) includes a block generation module (306), a block packing module (308), a geometry image generation module (310), a texture image generation module (312), a block information module (304), an occupancy map module (314), a smoothing module (336), image filling modules (316) and (318), a group expansion module (320), video compression modules (322), (323) and (332), an auxiliary block information compression module (338), an entropy compression module (334) and a multiplexer (324).

[0040] According to one aspect of the present disclosure, a V-PCC encoder (300) converts a 3D point cloud frame into an image-based representation and some metadata (e.g., occupancy map and block information) that is used to convert the compressed point cloud back into a decompressed point cloud. In some examples, the V-PCC encoder (300) can convert the 3D point cloud frame into a geometry image, a texture image, and an occupancy map, and then encode the geometry image, texture image, and occupancy map into a bitstream using video coding techniques. Typically, a geometry image is a 2D image with pixels filled with geometry values associated with points projected to the pixels, and pixels filled with geometry values can be referred to as geometry samples. A texture image is a 2D image with pixels filled with texture values associated with points projected to the pixels, and pixels filled with texture values can be referred to as texture samples. An occupancy map is a 2D image with pixels filled with values indicating whether a block is occupied or not occupied by a block.

[0041] The patch generation module (306) divides the point cloud into a set of patches that may or may not overlap (e.g., a patch is defined as a continuous subset of the surface described by the point cloud), such that each patch can be described by a depth field relative to a plane in 2D space. In some embodiments, the patch generation module (306) aims to decompose the point cloud into a minimum number of patches with smooth boundaries while also minimizing reconstruction error.

[0042] The block information module (304) can collect block information indicating the size and shape of the block. In some examples, the block information can be packed into an image frame and then encoded by the auxiliary block information compression module (338) to generate compressed auxiliary block information.

[0043] The block packing module (308) is configured to map the extracted blocks to a two-dimensional (2D) grid while minimizing unused space and ensuring that each M×M (e.g., 16×16) block of the grid is associated with a unique block. Effective block packing can directly affect compression efficiency by minimizing unused space or ensuring temporal consistency.

[0044] The geometry image generation module (310) can generate a 2D geometry image associated with the geometry of the point cloud at a given block location. The texture image generation module (312) can generate a 2D texture image associated with the texture of the point cloud at a given block location. The geometry image generation module (310) and the texture image generation module (312) use the 3D to 2D mapping calculated during the encapsulation process to store the geometry and texture of the point cloud as images. To better handle the case where multiple points are projected to the same sample, each block is projected to two images (called layers). In the example, the geometry image is represented by a WxH monochrome frame in YUV420-8 bit format. To generate the texture image, the texture generation process uses the reconstructed / smoothed geometry to calculate the color to be associated with the resampled points.

[0045] The occupancy map module (314) can generate an occupancy map that describes fill information at each cell. For example, the occupancy map includes a binary map that indicates, for each cell of the grid, whether the cell belongs to empty space or to a point cloud. In an example, the occupancy map uses binary information for each pixel that describes whether the pixel is filled. In another example, the occupancy map uses binary information for each pixel block that describes whether the pixel block is filled.

[0046] The occupancy map generated by the occupancy map module (314) can be compressed using lossless encoding or lossy encoding. When lossless encoding is used, the occupancy map is compressed using the entropy compression module (334). When lossy encoding is used, the occupancy map is compressed using the video compression module (332).

[0047] Note that the block packing module (308) can leave some empty space between the packed 2D blocks in the image frame. The image padding module (316) and the image padding module (318) can fill the empty space (referred to as padding) to generate an image frame that can be suitable for 2D video and image codecs. Image padding, also known as background padding, can fill unused space with redundant information. In some examples, good background padding minimally increases the bit rate while not introducing noticeable coding artifacts around block boundaries.

[0048] The video compression modules (322), (323), and (332) can encode 2D images, such as filled geometry images, filled texture images, and occupancy maps, based on a suitable video coding standard, such as HEVC, VVC, etc. In an example, the video compression modules (322), (323), and (332) are separate components that operate separately. Note that in another example, the video compression modules (322), (323), and (332) can be implemented as a single component.

[0049] In some examples, the smoothing module (336) is configured to generate a smoothed image of the reconstructed geometric image. The smoothed image can be provided to the texture image generation module (312). The texture image generation module (312) can then adjust the generation of the texture image based on the reconstructed geometric image. For example, if the block shape (e.g., the geometric shape) is slightly distorted during encoding and decoding, the distortion can be taken into account when generating the texture image to correct for the distortion of the block shape.

[0050] In some embodiments, the group expansion module (320) is configured to fill pixels around object boundaries with redundant low-frequency content to improve the coding gain and visual quality of the reconstructed point cloud.

[0051] The multiplexer (324) may multiplex the compressed geometry image, the compressed texture image, the compressed occupancy map, and the compressed auxiliary block information into a compressed bitstream.

[0052] Figure 4 A block diagram of a V-PCC decoder (400) for decoding a compressed bitstream corresponding to a point cloud frame according to some embodiments is shown. In some embodiments, the V-PCC decoder (400) can be used in the communication system (100) and the streaming system (200). For example, the decoder (210) can be configured to operate in a manner similar to the V-PCC decoder (400). The V-PCC decoder (400) receives a compressed bitstream and generates a reconstructed point cloud based on the compressed bitstream.

[0053] exist Figure 4In an example of the present invention, the V-PCC decoder (400) includes a demultiplexer (432), video decompression modules (434) and (436), an occupancy map decompression module (438), an auxiliary block information decompression module (442), a geometry reconstruction module (444), a smoothing module (446), a texture reconstruction module (448), and a color smoothing module (452).

[0054] The demultiplexer (432) may receive the compressed bitstream and separate it into a compressed texture image, a compressed geometry image, a compressed occupancy map, and compressed auxiliary block information.

[0055] The video decompression modules (434) and (436) can decode the compressed image according to a suitable standard (e.g., HEVC, VVC, etc.) and output the decompressed image. For example, the video decompression module (434) decodes the compressed texture image and outputs the decompressed texture image; and the video decompression module (436) decodes the compressed geometry image and outputs the decompressed geometry image.

[0056] The occupancy map decompression module (438) may decode the compressed occupancy map according to a suitable standard (eg, HEVC, VVC, etc.) and output a decompressed occupancy map.

[0057] The auxiliary block information decompression module (442) can decode the compressed auxiliary block information according to a suitable standard (e.g., HEVC, VVC, etc.) and output decompressed auxiliary block information.

[0058] The geometry reconstruction module (444) may receive the decompressed geometry image and generate a reconstructed point cloud geometry based on the decompressed occupancy map and the decompressed auxiliary patch information.

[0059] The smoothing module (446) can smooth out inconsistencies at the edges of blocks. The smoothing process is intended to mitigate potential discontinuities that may appear at block boundaries due to compression artifacts. In some embodiments, a smoothing filter can be applied to pixels located on block boundaries to mitigate distortion that may be caused by compression / decompression.

[0060] The texture reconstruction module (448) can determine texture information of points in the point cloud based on the decompressed texture image and the smoothed geometry.

[0061] The color smoothing module (452) can smooth out color inconsistencies. Non-adjacent blocks in 3D space are typically packed adjacent to each other in 2D video. In some examples, pixel values from non-adjacent blocks may be blended by a block-based video codec. The purpose of color smoothing is to reduce visible artifacts that appear at block boundaries.

[0062] Figure 5 A block diagram of a video decoder (510) according to an embodiment of the present disclosure is shown. The video decoder (510) can be used in the V-PCC decoder (400). For example, the video decompression modules (434) and (436) and the occupancy map decompression module (438) can be configured similarly to the video decoder (510).

[0063] The video decoder (510) may include a parser (520) to reconstruct symbols (521) from compressed images such as an encoded video sequence. The categories of these symbols include information for managing the operation of the video decoder (510). The parser (520) may parse / entropy decode the received encoded video sequence. The encoding of the encoded video sequence may be performed according to a video coding technique or standard and may follow various principles, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (520) may extract a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder from the encoded video sequence based on at least one parameter corresponding to the group. The subgroup may include a group of pictures (GOP), a picture, a tile, a slice, a macroblock, a coding unit (CU), a block, a transform unit (TU), a prediction unit (PU), etc. The parser (520) may also extract information such as transform coefficients, quantizer parameter values, motion vectors, etc. from the encoded video sequence.

[0064] The parser (520) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory to create symbols (521).

[0065] The reconstruction of the symbol (521) may involve multiple different units depending on the type of the coded video picture or part of the coded video picture (e.g., inter-frame picture and intra-frame picture, inter-frame block and intra-frame block) and other factors. Which units are involved and how they are involved can be controlled by sub-group control information parsed from the coded video sequence by the parser (520). For the sake of simplicity, the flow of this sub-group control information between the parser (520) and the following multiple units is not depicted.

[0066] In addition to the functional blocks already mentioned, the video decoder (510) can be conceptually subdivided into a number of functional units as described below. In actual implementations operating under commercial constraints, many of these units interact closely with each other and may be at least partially integrated with each other. However, for the purposes of describing the disclosed subject matter, the conceptual subdivision into the following functional units is appropriate.

[0067] The first unit is a scaler / inverse transform unit (551). The scaler / inverse transform unit (551) receives quantized transform coefficients as symbols (521) from the parser (520) along with control information including which transform to use, block size, quantization factor, quantization scaling matrix, etc. The scaler / inverse transform unit (551) can output blocks comprising sample values that can be input into an aggregator (555).

[0068] In some cases, the output samples of the scaler / inverse transform unit (551) may belong to an intra-coded block; that is, a block that does not use predictive information from a previously reconstructed picture, but may use predictive information from a previously reconstructed portion of the current picture. Such predictive information may be provided by an intra-picture prediction unit (552). In some cases, the intra-picture prediction unit (552) uses surrounding reconstructed information extracted from the current picture buffer (558) to generate a block of the same size and shape as the block being reconstructed. For example, the current picture buffer (558) buffers, for example, a partially reconstructed current picture and / or a fully reconstructed current picture. In some cases, the aggregator (555) adds the prediction information already generated by the intra-prediction unit (552) to the output sample information as provided by the scaler / inverse transform unit (551) on a per-sample basis.

[0069] In other cases, the output samples of the scaler / inverse transform unit (551) may belong to a block that is inter-coded and potentially motion compensated. In this case, the motion compensated prediction unit (553) may access the reference picture memory (557) to extract samples for prediction. After the extracted samples are motion compensated according to the symbols (521) belonging to the block, these samples may be added to the output of the scaler / inverse transform unit (551) (in this case referred to as residual samples or residual signal) by the aggregator (555) to generate output sample information. The address in the reference picture memory (557) from which the motion compensated prediction unit (553) extracts the prediction samples may be controlled by a motion vector and may be made available to the motion compensated prediction unit (553) in the form of symbols (521) that may have, for example, an X component, a Y component, and a reference picture component. Motion compensation may also include interpolation of sample values extracted from the reference picture memory (557), motion vector prediction mechanisms, etc., when using sub-sample accurate motion vectors.

[0070] The output samples of the aggregator (555) may be subjected to various loop filtering techniques in a loop filter unit (556). The video compression techniques may include in-loop filtering techniques controlled by parameters included in the coded video sequence (also referred to as the coded video bitstream) and available to the loop filter unit (556) as symbols (521) from the parser (520), but the video compression techniques may also be responsive to meta-information obtained during decoding of a previous (in decoding order) portion of the coded picture or coded video sequence, and to previously reconstructed and loop filtered sample values.

[0071] The output of the loop filter unit (556) may be a sample stream that may be output to a rendering device and stored in a reference picture memory (557) for future inter-picture prediction.

[0072] Once fully reconstructed, certain coded pictures can be used as reference pictures for future prediction. For example, once the coded picture corresponding to the current picture is fully reconstructed and the coded picture has been identified as a reference picture (e.g., by the parser (520)), the current picture buffer (558) can become part of the reference picture memory (557) and a new current picture buffer can be reallocated before starting to reconstruct a subsequent coded picture.

[0073] The video decoder (510) may perform decoding operations according to a predetermined video compression technique in a standard such as ITU-T (International Telecommunication Union-Telecommunication Standardization Sector) Recommendation H.265. The coded video sequence may conform to the syntax specified by the video compression technology or standard used, in the sense that the coded video sequence follows the syntax of the video compression technology or standard and the profile recorded in the video compression technology or standard. Specifically, the profile may select certain tools from all the tools available in the video compression technology or standard as the only tools available for use under the profile. For compliance, the complexity of the coded video sequence is also required to be within a range as defined by the hierarchy of the video compression technology or standard. In some cases, the hierarchy limits the maximum picture size, maximum frame rate, maximum reconstruction sampling rate (measured in, for example, megasamples per second), maximum reference picture size, etc. In some cases, the limits set by the hierarchy may be further defined by the Hypothetical Reference Decoder (HRD) specification and metadata signaled in the coded video sequence for HRD buffer management.

[0074] Figure 6 A block diagram of a video encoder (603) according to an embodiment of the present disclosure is shown. The video encoder (603) can be used in the V-PCC encoder (300) for compressing point clouds. In the example, the video compression modules (322) and (323) and the video compression module (332) are configured similarly to the encoder (603).

[0075] The video encoder (603) may receive an image, such as a filled geometry image, a filled texture image, etc., and may generate a compressed image.

[0076] According to an embodiment, the video encoder (603) can encode and compress the pictures (images) of the source video sequence into an encoded video sequence (compressed images) in real time or under any other time constraints required by the application. Implementing an appropriate encoding speed is a function of the controller (650). In some embodiments, the controller (650) controls other functional units as described below and is functionally coupled to the other functional units. For the sake of brevity, such coupling is not depicted. The parameters set by the controller (650) may include rate control related parameters (picture skipping, quantizer, lambda value of rate distortion optimization technique, ...), picture size, picture group (GOP) layout, maximum motion vector search range, etc. The controller (650) can be configured to have other suitable functions related to the video encoder (603) optimized for a specific system design.

[0077] In some embodiments, the video encoder (603) is configured to operate in a coding loop. As an oversimplification, in an example, the coding loop may include a source encoder (630) (e.g., responsible for creating symbols, such as a symbol stream, based on an input picture to be encoded and a reference picture) and a (local) decoder (633) embedded in the video encoder (603). The decoder (633) reconstructs the symbols to create sample data in a manner similar to how a (remote) decoder would also create sample data (because in the video compression or technology considered in the disclosed subject matter, any compression between the symbols and the encoded video bitstream is lossless). The reconstructed sample stream (sample data) is input to a reference picture memory (634). Since the decoding of the symbol stream produces bit-accurate results regardless of the decoder location (local or remote), the contents of the reference picture memory (634) are also bit-accurate between the local encoder and the remote encoder. That is, the reference picture samples "seen" by the prediction portion of the encoder are exactly the same sample values that the decoder will "see" when using prediction during decoding. This basic principle of reference picture synchronization (and the resulting drift when synchronization cannot be maintained, eg due to channel errors) is also used in some related techniques.

[0078] The operation of the "local" decoder (633) can be combined with the above Figure 5 The "remote" decoder of the video decoder (510) described in detail is identical. However, reference is also briefly made to Figure 5 , when symbols are available and the entropy encoder (645) and parser (520) are capable of losslessly encoding / decoding the symbols into an encoded video sequence, the entropy decoding portion of the video decoder (510) including the parser (520) may not be fully implemented in the local decoder (633).

[0079] At this point, it can be observed that any decoder technology other than parsing / entropy decoding present in a decoder must also be present in a substantially identical functional form in the corresponding encoder. For this reason, the disclosed subject matter focuses on decoder operation. The description of encoder technology can be simplified because encoder technology is the inverse of the decoder technology described comprehensively. A more detailed description is required only in certain areas and is provided below.

[0080] During operation, in some examples, the source encoder (630) may perform motion compensated predictive coding, which predictively encodes an input picture with reference to one or more previously encoded pictures from a video sequence designated as "reference pictures." In this manner, the encoding engine (632) encodes the differences between pixel blocks of an input picture and pixel blocks of a reference picture that may be selected as a prediction reference for the input picture.

[0081] The local video decoder (633) can decode the coded video data of the picture that can be designated as the reference picture based on the symbols created by the source encoder (630). The operation of the encoding engine (632) can advantageously be lossy processing. When the coded video data can be decoded at the video decoder ( Figure 6 When the video sequence is decoded at a remote location (not shown), the reconstructed video sequence may typically be a copy of the source video sequence with some errors. The local video decoder (633) repeats the decoding process that may be performed by the video decoder for the reference picture and may store the reconstructed reference picture in the reference picture cache (634). In this way, the video encoder (603) may locally store a copy of the reconstructed reference picture that has common content (absent transmission errors) with the reconstructed reference picture to be obtained by the remote video decoder.

[0082] The predictor (635) can perform a prediction search for the encoding engine (632). That is, for a new picture to be encoded, the predictor (635) can search the reference picture memory (634) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference picture motion vectors, block shapes, etc., that can serve as suitable prediction references for the new picture. The predictor (635) can operate on a pixel block by pixel block basis to find a suitable prediction reference. In some cases, as determined by the search results obtained by the predictor (635), the input picture can have prediction references taken from multiple reference pictures stored in the reference picture memory (634).

[0083] The controller (650) can manage the encoding operations of the source encoder (630), including, for example, setting parameters and sub-group parameters for encoding video data.

[0084] The outputs of all the above functional units may be entropy coded in an entropy encoder (645). The entropy encoder (645) converts the symbols generated by the various functional units into a coded video sequence by losslessly compressing the symbols according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc.

[0085] The controller (650) can manage the operation of the video encoder (603). During encoding, the controller (650) can assign a specific encoding picture type to each encoded picture, which may affect the encoding techniques that can be applied to the corresponding picture. For example, a picture may generally be assigned one of the following picture types:

[0086] An intra picture (I picture) can be a picture that can be encoded and decoded without using any other picture in the sequence as a prediction source. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art are aware of these variations of I pictures and their corresponding applications and features.

[0087] A predictive picture (P picture) may be a picture that can be encoded and decoded using intra prediction or inter prediction, which uses at most one motion vector and a reference index to predict sample values for each block.

[0088] Bidirectionally predictive pictures (B pictures) can be pictures that can be encoded and decoded using intra prediction or inter prediction, which uses up to two motion vectors and reference indices to predict sample values for each block. Similarly, multiple predictive pictures can use more than two reference pictures and associated metadata to reconstruct a single block.

[0089] A source picture may typically be spatially subdivided into blocks of samples (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples, respectively) and encoded on a block-by-block basis. A block may be predictively coded with reference to other (already coded) blocks as determined by the coding allocation applied to the block's corresponding picture. For example, a block of an I picture may be non-predictively coded, or a block of an I picture may be predictively coded (spatial prediction or intra prediction) with reference to already coded blocks of the same picture. A block of pixels of a P picture may be predictively coded via spatial prediction or via temporal prediction with reference to one previously coded reference picture. A block of a B picture may be predictively coded via spatial prediction or via temporal prediction with reference to one or two previously coded reference pictures.

[0090] The video encoder (603) may perform encoding operations according to a predetermined video coding technique or standard, such as ITU-T Recommendation H.265. In operation of the video encoder (603), the video encoder (603) may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancy in the input video sequence. Thus, the encoded video data may conform to a syntax specified by the video coding technique or standard used.

[0091] Video may be in the form of multiple source pictures (images) in a temporal sequence. Intra-picture prediction (often shortened to intra prediction) exploits spatial correlations in a given picture, while inter-picture prediction exploits correlations (temporal or otherwise) between pictures. In an example, a particular picture being encoded / decoded - referred to as the current picture - is divided into blocks. In cases where a block in the current picture is similar to a reference block in a reference picture that was previously encoded in the video and is still buffered, the block in the current picture can be encoded by a vector called a motion vector. The motion vector points to the reference block in a reference picture, and in cases where multiple reference pictures are used, the motion vector may have a third dimension that identifies the reference picture.

[0092] In some embodiments, bidirectional prediction techniques may be used in inter-frame picture prediction. According to bidirectional prediction techniques, two reference pictures are used, for example, a first reference picture and a second reference picture that are both before the current picture in the video in decoding order (but may be in the past and in the future, respectively, in display order). A block in the current picture may be encoded using a first motion vector pointing to a first reference block in the first reference picture and a second motion vector pointing to a second reference block in the second reference picture. The block may be predicted using a combination of the first reference block and the second reference block.

[0093] In addition, merge mode techniques can be used in inter-picture prediction to improve coding efficiency.

[0094] According to some embodiments of the present disclosure, predictions such as inter-picture prediction and intra-picture prediction are performed in units of blocks. For example, according to the HEVC standard, pictures in a video picture sequence are divided into coding tree units (CTUs) for compression, and the CTUs in the pictures have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Generally speaking, a CTU includes three coding tree blocks (CTBs), which are a luminance CTB and two chrominance CTBs. Each CTU can also be recursively split into one or more coding units (CUs) using a quadtree. For example, a 64×64 pixel CTU can be split into a 64×64 pixel CU, or four 32×32 pixel CUs, or sixteen 16×16 pixel CUs. In the example, each CU is analyzed to determine the prediction type for the CU, such as an inter prediction type or an intra prediction type. The CU is split into one or more prediction units (PUs) based on temporal and / or spatial predictability. Typically, each PU includes a luma prediction block (PB) and two chroma PBs. In an embodiment, the prediction operation in decoding (encoding / decoding) is performed in units of prediction blocks. Using a luma prediction block as an example of a prediction block, the prediction block includes a matrix of pixel values (e.g., luma values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0095] Figure 7 A block diagram of a G-PPC encoder (700) according to an embodiment is shown. The encoder (700) can be configured to receive point cloud data and compress the point cloud data to generate a bitstream carrying the compressed point cloud data. In an embodiment, the encoder (700) can include a position quantization module (710), a duplicate point removal module (712), an octree encoding module (730), an attribute conversion module (720), a level of detail (LOD) generation module (740), an attribute prediction module (750), a residual quantization module (760), an arithmetic coding module (770), an inverse residual quantization module (780), an addition module (781), and a memory (790) for storing reconstructed attribute values.

[0096] As shown, an input point cloud (701) may be received at an encoder (700). Positions (e.g., 3D coordinates) of the point cloud (701) are provided to a quantization module (710). The quantization module (710) is configured to quantize the coordinates to generate quantized positions. A duplicate point removal module (712) is configured to receive the quantized positions and perform a filtering process to identify and remove duplicate points. An octree encoding module (730) is configured to receive the filtered positions from the duplicate point removal module (712) and perform an octree-based encoding process to generate a sequence of occupancy codes that describe a 3D grid of voxels. The occupancy codes are provided to an arithmetic coding module (770).

[0097] The attribute conversion module (720) is configured to receive attributes of the input point cloud and perform attribute conversion processing to determine the attribute value of each voxel when multiple attribute values are associated with the corresponding voxel. The attribute conversion processing can be performed on the reordered points output from the octree encoding module (730). The attributes after the conversion operation are provided to the attribute prediction module (750). The LOD generation module (740) is configured to operate on the reordered points output from the octree encoding module (730) and reorganize the points into different LODs. The LOD information is provided to the attribute prediction module (750).

[0098] The attribute prediction module (750) processes the points according to the LOD-based order indicated by the LOD information from the LOD generation module (740). The attribute prediction module (750) generates an attribute prediction for the current point based on the reconstructed attributes of a set of neighboring points of the current point stored in the memory (790). A prediction residual may then be obtained based on the original attribute value received from the attribute conversion module (720) and the locally generated attribute prediction. When a candidate index is used in the corresponding attribute prediction process, the index corresponding to the selected prediction candidate may be provided to the arithmetic coding module (770).

[0099] The residual quantization module (760) is configured to receive the prediction residual from the attribute prediction module (750) and perform quantization to generate a quantized residual. The quantized residual is provided to the arithmetic coding module (770).

[0100] The inverse residual quantization module (780) is configured to receive the quantized residual from the residual quantization module (760) and generate a reconstructed prediction residual by performing an inverse operation of the quantization operation performed at the residual quantization module (760). The addition module (781) is configured to receive the reconstructed prediction residual from the inverse residual quantization module (780) and the corresponding attribute prediction from the attribute prediction module (750). By combining the reconstructed prediction residual with the attribute prediction, a reconstructed attribute value is generated and stored in the memory (790).

[0101] The arithmetic coding module (770) is configured to receive the occupancy code, the candidate index (if used), the quantized prediction residual (if generated), and other information and perform entropy coding to further compress the received values or information. As a result, a compressed bitstream (702) carrying the compressed information can be generated. The bitstream (702) can be transmitted or otherwise provided to a decoder that decodes the compressed bitstream, or can be stored in a storage device.

[0102] Figure 8 A block diagram of a G-PCC decoder (800) according to an embodiment is shown. The decoder (800) can be configured to receive a compressed bitstream and perform point cloud data decompression to decompress the bitstream to generate decoded point cloud data. In an embodiment, the decoder (800) can include an arithmetic decoding module (810), an inverse residual quantization module (820), an octree decoding module (830), an LOD generation module (840), an attribute prediction module (850), and a memory (860) for storing reconstructed attribute values.

[0103] As shown, a compressed bitstream (801) can be received at an arithmetic decoding module (810). The arithmetic decoding module (810) is configured to decode the compressed bitstream (801) to obtain quantized prediction residuals (if generated) and occupancy codes for the point cloud. The octree decoding module (830) is configured to determine reconstructed positions of points in the point cloud based on the occupancy codes. The LOD generation module (840) is configured to reorganize the points into different LODs based on the reconstructed positions and determine an order based on the LODs. The inverse residual quantization module (820) is configured to generate a reconstructed residual based on the quantized residual received from the arithmetic decoding module (810).

[0104] The attribute prediction module (850) is configured to perform attribute prediction processing to determine attribute predictions for a point based on an LOD-based order. For example, the attribute prediction for the current point may be determined based on the reconstructed attribute values of the current point's neighboring points stored in the memory (860). The attribute prediction module (850) may combine the attribute predictions with the corresponding reconstruction residuals to generate a reconstructed attribute for the current point.

[0105] In one example, the sequence of reconstructed attributes generated from the attribute prediction module (850) along with the reconstructed positions generated from the octree decoding module (830) corresponds to the decoded point cloud (802) output from the decoder (800). Additionally, the reconstructed attributes are also stored in the memory (860) and can be subsequently used to derive attribute predictions for subsequent points.

[0106] In various embodiments, the encoder (300), decoder (400), encoder (700) and / or decoder (800) can be implemented using hardware, software, or a combination thereof. For example, the encoder (300), decoder (400), encoder (700) and / or decoder (800) can be implemented using processing circuits such as one or more integrated circuits (ICs) operating with or without software, such as application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), and the like. In another example, the encoder (300), decoder (400), encoder (700) and / or decoder (800) can be implemented as software or firmware including instructions stored in a non-volatile (or non-transitory) computer-readable storage medium. The instructions, when executed by a processing circuit such as one or more processors, cause the processing circuit to perform the functions of the encoder (300), decoder (400), encoder (700) and / or decoder (800).

[0107] Note that the attribute prediction modules (750) and (850) configured to implement the attribute prediction techniques disclosed herein may be included in a computer program that may have a Figure 7 and Figure 8 The decoder or encoder shown may be used in other decoders or encoders with similar or different structures. In addition, in various examples, the encoder (700) and decoder (800) may be included in the same device or in separate devices.

[0108] According to some aspects of the present disclosure, the geometric information and associated attributes of a point cloud, such as color and reflectivity, can be compressed separately (e.g., in the Test Model 13 (TMC13) model). The geometric information of a point cloud, including the 3D coordinates of points in the point cloud, can be encoded using an octree segmentation and occupancy information of the segmentation. Attributes can be compressed based on the reconstructed geometry using techniques such as prediction, lifting, and region-adaptive hierarchical transformations.

[0109] According to some aspects of the present disclosure, octree segmentation can be used to segment a three-dimensional space. An octree is a three-dimensional analog of a quadtree in a two-dimensional space. Octree segmentation technology refers to a segmentation technology that recursively subdivides a three-dimensional space into eight octants, and an octree structure refers to a tree structure that represents the segmentation. In an example, each node in the octree structure corresponds to a three-dimensional space, and the node can be an end node (no longer segmented, also referred to as a leaf node in some examples) or a node with further segmentation. The segmentation at a node can segment the three-dimensional space represented by the node into eight octants. In some examples, a node corresponding to the segmentation of a particular node can be referred to as a child node of the particular node.

[0110] Figure 9 A diagram illustrating partitioning of a 3D cube (900) (corresponding to nodes) based on an octree partitioning technique according to some embodiments of the present disclosure is shown. The partitioning may divide the 3D cube (900) into eight smaller cubes of equal size 0 to 7, such as Figure 9 shown.

[0111] Octree segmentation technology (e.g., in TMC13) can recursively divide the original 3D space into smaller units, and can encode the occupancy information of each subspace to represent the geometric position.

[0112] In some embodiments (e.g., in TMC13), an octree geometry codec is used. The octree geometry codec can perform geometry encoding. In some examples, the geometry encoding is performed on a cubic box. For example, a cubic box can be formed by two points (0,0,0) and (2 M-1 ,2 M-1 ,2 M-1 ) is defined by the axis-aligned bounding box B, where 2 M-1 The size of the bounding box B is defined, and M can be specified in the bitstream.

[0113] Then, an octree structure is constructed by recursively subdividing the cubic box. For example, from two points (0,0,0) and (2 M -1 ,2 M-1 ,2 M-1 ) is divided into 8 sub-cubes, and then an 8-bit code is generated, which is called an occupancy code. Each bit in the occupancy code is associated with a sub-cube, and the value of the bit is used to indicate whether the associated sub-cube contains any points of the point cloud. For example, a bit value of 1 indicates that the sub-cube associated with the bit contains one or more points of the point cloud; while a bit value of 0 indicates that the sub-cube associated with the bit does not contain any points of the point cloud.

[0114] In addition, for an empty sub-cube box (e.g., the value of the bit associated with the sub-cube box is 0), no further division is applied to the sub-cube box. When a sub-cube box has one or more points of the point cloud (e.g., the value of the bit associated with the sub-cube box is 1), the sub-cube box is further divided into 8 smaller sub-cube boxes, and an occupancy code is generated for the sub-cube box to indicate the occupancy of the smaller sub-cube boxes. In some examples, the subdivision operation can be repeatedly performed on a non-empty sub-cube box until the size of the sub-cube box equals a predetermined threshold, such as a size of 1. In some examples, a sub-cube box with a size of 1 is referred to as a voxel, and a sub-cube box with a size larger than a voxel can be referred to as a non-voxel.

[0115] Figure 10 An example of an octree partition (1010) and an octree structure (1020) corresponding to the octree partition (1010) is shown according to some embodiments of the present disclosure. Figure 10 Two levels of partitioning in the octree partitioning (1010) are shown. The octree structure (1020) includes nodes (N0) corresponding to the cube boxes used for the octree partitioning (1010). At the first level, the cube boxes are partitioned into Figure 9 The numbering technique shown in FIG 8 is used to number the 8 sub-cube boxes from 0 to 7. The occupancy code for the partition of node N0 is “10000001” in binary form, which indicates that the first sub-cube box represented by node N0-0 and the eighth sub-cube box represented by node N0-7 include points in the point cloud, while the other sub-cube boxes are empty.

[0116] Then, at the second level of segmentation, the first child cube box (represented by node N0-0) and the eighth child cube box (represented by node N0-7) are further subdivided into eight octants, respectively. For example, the first child cube box (represented by node N0-0) is segmented into Figure 9 The numbering technique shown in FIG is used to number the eight smaller sub-cube boxes from 0 to 7. The occupancy code for the partition of node N0-0 is "00011000" in binary form, which indicates that the fourth smaller sub-cube box (represented by node N0-0-3) and the fifth smaller sub-cube box (represented by node N0-0-4) include points in the point cloud, while the other smaller sub-cube boxes are empty. At the second level, the eighth sub-cube box (represented by node N0-7) is similarly partitioned into eight smaller sub-cube boxes, as shown in FIG. Figure 10 shown.

[0117] exist Figure 10 In the example of , nodes corresponding to non-empty cubic spaces (eg, cubic boxes, sub-cubic boxes, smaller sub-cubic boxes, etc.) are colored gray and are referred to as shaded nodes.

[0118] According to some aspects of the present disclosure, suitable coding techniques can be used to appropriately compress the occupancy code. In some embodiments, an arithmetic encoder is used to compress the occupancy code of the current node in the octree structure. The occupancy code can be represented as S, which is an 8-bit integer, and each bit in S indicates the occupancy state of the child nodes of the current node. In an embodiment, the occupancy code is encoded using bit-by-bit encoding. In another embodiment, the occupancy code is encoded using byte-by-byte encoding. In some examples (e.g., TMC13), bit-by-bit encoding is enabled by default. Both bit-by-bit encoding and byte-by-byte encoding can perform arithmetic coding with context modeling to encode the occupancy code. The context state can be initialized at the beginning of the entire encoding process for the occupancy code, and the context state is updated during the encoding process of the occupancy code.

[0119] In a bit-by-bit encoding embodiment for encoding the occupancy code of the current node, the eight bits in S for the current node are encoded in a specific order. Each bit in S is encoded by referring to the occupancy status of the current node's neighboring nodes and / or the neighboring nodes' children. The neighboring nodes are at the same level as the current node and can be referred to as sibling nodes of the current node.

[0120] In a byte-by-byte encoding embodiment that encodes the occupancy code of the current node, the occupancy code S (one byte) can be encoded by referring to the following: (1) an adaptive lookup table (A-LUT) that tracks the P (e.g., 32) most frequently used occupancy codes; and (2) a cache that tracks the last observed different Q (e.g., 16) occupancy codes.

[0121] In some examples of byte-by-byte encoding, a binary flag indicating whether S is in the A-LUT is encoded. If S is in the A-LUT, the index in the A-LUT is encoded using a binary arithmetic encoder. If S is not in the A-LUT, a binary flag indicating whether S is in the cache is encoded. If S is in the cache, the binary representation of its index in the cache is encoded using a binary arithmetic encoder. Otherwise, if S is not in the cache, the binary representation of S is encoded using a binary arithmetic encoder.

[0122] In some embodiments, at the decoder side, the decoding process can begin by parsing the dimensions of a bounding box from the bitstream. The bounding box indicates a cubic box corresponding to the root node in the octree structure, and the cubic box is segmented according to the geometric information of the point cloud (e.g., the occupancy information of the points in the point cloud). The octree structure is then constructed by subdividing the cubic box according to the decoded occupancy code.

[0123] According to one aspect of the present disclosure, a single isolated point is defined as a single point in a node and can be encoded using the geometric coordinates of the single isolated point. In some examples, when the current node meets the conditions for encoding a single isolated point (e.g., isolation mode is turned on in the high-level syntax and other suitable conditions are met), a flag (e.g., referred to as a single isolated point flag) is signaled to indicate whether the current node has a single isolated point (also referred to as being in single isolated point mode). If the single isolated point flag at the current node is true, the current node is encoded in single isolated point mode and the geometric coordinates of the single isolated point are directly encoded without further octree splitting. If the single isolated point flag is false, the current node can be further divided until a leaf node is reached.

[0124] In some related examples, a single isolated point flag is encoded using arithmetic coding with a single context. The single context refers to the probability of the single isolated point flag itself. Various aspects of the present disclosure provide techniques for encoding a single isolated point flag using information from other nodes, such as occupancy information of a parent node and information of neighboring nodes (also known as sibling nodes), thereby improving the coding efficiency of the single isolated point flag.

[0125] According to one aspect of the present disclosure, if the parent node of the current node conforms to the isolated mode and the parent node has only one child node (e.g., the current node), the current node cannot be encoded in the single isolated point mode. Because if the current node is encoded in the single isolated point mode, since the current node is the only child node of the parent node, the parent node has a single isolated point. Therefore, the parent node should be encoded in the single isolated point mode and the parent node will not be further split, and the current node will not be reached. By utilizing the above comments, the single isolated point flag of the current node can be inferred to be false (e.g., having a value of "0") instead of being explicitly signaled, thereby saving bits used to encode the single isolated point flag.

[0126] Figure 11 An example of a syntax table (1100) according to some embodiments of the present disclosure is shown. The syntax table (1100) may be used to decode and determine geometry information at the current node.

[0127] In the syntax table (1100), the variable "depth" indicates the split depth of the current node, and the current node can be identified using the index "nodeIdx". In the syntax table (1100), information such as the value of the variable "numSiblings" is received from the parent node. The variable "numSiblings" specifies the number of sibling nodes (e.g., including the current node) of the current node and is a positive integer. The value of the variable "numSiblings" is the sum of the non-empty child nodes in the occupancy code of the parent node. For example, when the occupancy code of the parent node is "10000011", the value of the variable "numSiblings" is 3; and when the occupancy code of the parent node is "00000010", the value of the variable "numSiblings" is 1.

[0128] In the syntax table (1100), the parameter "geomIsolatedModeFlag" is specified in the high-level syntax to indicate whether isolated mode is allowed, and the parameter "geomIsolatedModeMaxDepth" is specified in the high-level syntax to indicate the maximum segmentation depth for isolated mode. The condition shown by (1101) checks the compliance of isolated mode for the current node. For example, when the parameter "geomIsolatedModeFlag" is true (e.g., a value of "1") and the variable "depth" is less than or equal to the parameter "geomIsolatedModeMaxDepth", isolated mode is complied at the current node. Note that the compliance test is not limited to the condition shown by (1101) and can have other suitable forms.

[0129] exist Figure 11 In the example of , when the current node meets the isolated mode, and if the variable "numSiblings" is equal to 1 (indicating that the parent node has only one child node, which is the current node), it can be inferred that the flag "geom_isolated_flag" for indicating the single isolated point mode is false (for example, the value is "0") to indicate that the current node is not encoded in the single isolated point mode, as shown by Figure 11 As shown in (1102) in .

[0130] exist Figure 11 In the example of , when the current node conforms to the isolated mode, and if the variable "numSiblings" is not equal to 1 (greater than 1, if the parent node has two or more child nodes), the flag "geom_isolated_flag" is signaled and can be decoded from the bitstream, as shown by Figure 11 As shown in (1103) in .

[0131] In addition, Figure 11In the example of , variables "isolated_position_x", "isolated_position_y", and "isolated_position_z" are used to specify the position coordinates of a single isolated point. For example, when the flag "geom_isolated_flag" is true (e.g., the value is "1"), the variables "isolated_position_x", "isolated_position_y", and "isolated_position_z" can be decoded from the bitstream, as shown by Figure 11 As shown in (1104) in .

[0132] In addition, Figure 11 In the example of FIG, when the flag “geom_isolated_flag” is false (e.g., the value is “0”), the current node is further segmented, and then the occupancy code can be decoded from the bit stream, as shown by Figure 11 As shown in (1105) in .

[0133] According to another aspect of the present disclosure, a single isolated point flag can be encoded using arithmetic coding with additional context. Based on the additional context, a context model can be determined and the single isolated point flag can be encoded with better coding efficiency. For example, information from adjacent coding nodes and / or information from the parent node can be used as additional context. In an embodiment, the number of sibling nodes, such as the variable "numSiblings", is used as additional context for encoding the single isolated point flag. In another example, when the parent node has multiple child nodes and the current node is encoded after some sibling nodes, the information of the encoded sibling nodes before the current node can be used as additional context for encoding the single isolated point flag.

[0134] Figure 12 A flow chart outlining a process (1200) according to an embodiment of the present disclosure is shown. The process (1200) can be used during the decoding process of the point cloud. In various embodiments, the process (1200) is performed by processing circuitry such as a processing circuit in a terminal device (110); a processing circuit that performs the functions of an encoder (203) and / or a decoder (210); a processing circuit that performs the functions of an encoder (300), a decoder (400), an encoder (700), and / or a decoder (800); or the like. In some embodiments, the process (1200) is implemented as software instructions, such that when the processing circuitry executes the software instructions, the processing circuitry performs the process (1200). The process begins at (S1201) and proceeds to (S1210).

[0135] At (S1210), a bitstream carrying compressed data of a point cloud is received.

[0136] At (S1220), it is determined whether a current node in an octree structure corresponding to a three-dimensional (3D) segmentation of the space of the point cloud complies with an isolated pattern.

[0137] At (S1230), a single isolated point flag of the current node is determined based on information of another node, the single isolated point flag indicating whether the current node is encoded as a single isolated point.

[0138] In some embodiments, a single outlier flag is determined based on information about a parent node of the current node. In some examples, the single outlier flag of the current node is inferred to have a false value (e.g., "0") in response to the parent node having only one child node. In some examples, a context model is determined based on the number of child nodes of the parent node, and the single outlier flag of the current node is decoded from the bitstream based on the context model.

[0139] In some embodiments, the single isolated point flag is determined based on information about the sibling nodes of the current node. For example, a context model is determined based on the sibling node information of the current node, and the single isolated point flag of the current node is decoded from the bitstream based on the context model.

[0140] In some embodiments, in response to the single isolated point flag having a true value, coordinates of the single isolated point are decoded from the bitstream.

[0141] In some implementations, in response to the single outlier flag having a false value, an occupancy code of the current node is decoded from the bitstream.

[0142] Then, the process proceeds to (S1299) and terminates.

[0143] In one embodiment, the method further includes determining the single isolated point flag based on information of at least one of the parent node, grandparent node, and sibling node of the current node, wherein the single isolated point flag indicates whether the current node is encoded as a single isolated point.

[0144] In one embodiment, the method further includes: in response to the parent node having one child node, inferring that the single outlier flag of the current node has a false value.

[0145] In one embodiment, the method further comprises: determining a context model based on the number of child nodes of the parent node; and decoding a single isolated point flag of the current node from the bitstream based on the context model.

[0146] In one embodiment, the method further comprises: in response to the single isolated point flag having a true value, decoding the coordinates of the single isolated point from the bit stream.

[0147] In one embodiment, the method further comprises: in response to the single outlier flag having a false value, decoding an occupancy code of the current node from the bit stream.

[0148] In one embodiment, the method further includes: determining a context model based on information of sibling nodes of the current node; and decoding a single isolated point flag of the current node from the bitstream based on the context model.

[0149] The techniques disclosed in this disclosure can be used alone or in combination in any order. In addition, each of the techniques (e.g., methods, implementations), encoders, and decoders can be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In some examples, one or more processors execute a program stored in a non-transitory computer-readable medium.

[0150] The techniques described above can be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. For example, Figure 13 A computer system (1300) suitable for implementing certain embodiments of the disclosed subject matter is shown.

[0151] Computer software may be encoded using any suitable machine code or computer language, which may be subjected to mechanisms such as assembly, compilation, and linking to create code comprising instructions that may be executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., or through interpretation, microcode execution, and the like.

[0152] The instructions may be executed on various types of computers or components thereof including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, IoT devices, and the like.

[0153] Figure 13 The components shown for the computer system (1300) are exemplary in nature and are not intended to suggest any limitation on the scope of use or functionality of computer software implementing embodiments of the present disclosure. Neither should the configuration of components be interpreted as having any dependency or requirement relating to any one or combination of components shown in the exemplary embodiment of the computer system (1300).

[0154] The computer system (1300) may include certain human-machine interface input devices. Such human-machine interface input devices may respond to input from one or more human users through, for example, tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, tapping), visual input (e.g., gestures), and olfactory input (not depicted). The human-machine interface devices may also be used to capture certain media that are not necessarily directly related to human conscious input, such as audio (e.g., voice, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still image camera), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0155] The input human-machine interface device may include one or more of the following (only one of each described): keyboard (1301), mouse (1302), touchpad (1303), touch screen (1310), data gloves (not shown), joystick (1305), microphone (1306), scanner (1307), camera (1308).

[0156] The computer system (1300) may also include certain human interface output devices. Such human interface output devices may stimulate one or more senses of a human user through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include: tactile output devices (e.g., tactile feedback via a touch screen (1310), a data glove (not shown), or a joystick (1305), although tactile feedback devices that do not function as input devices may also be present); audio output devices (e.g., speakers (1309), headphones (not depicted)); visual output devices (e.g., screens (1310), including CRT (Cathode Ray Tube, CRT) screens, LCD (Liquid Crystal Display, LCD) screens, plasma screens, OLED (Organic Light Emitting Diode, OLED) screens, each with or without touch screen input capability, each with or without tactile feedback capability—some of which may be capable of outputting two-dimensional visual output or more than three-dimensional output, such as through stereoscopic image output); virtual reality glasses (not depicted); holographic displays and cigarette cans (not depicted); and printers (not depicted).

[0157] The computer system (1300) may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM (Read Only Memory, ROM) / RW (1320) with media (1321) such as CD (Compact Disc, CD) / DVD (Digital Video Disk, DVD), thumb drives (1322), removable hard drives or solid-state drives (1323), traditional magnetic media such as tapes and floppy disks (not depicted), dedicated ROM / ASIC (Application Specific Integrated Circuit, ASIC) / PLD (Programmable Logic Device, PLD)-based devices such as security dongles (not depicted), etc.

[0158] Those skilled in the art will also understand that the term "computer-readable media" used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0159] The computer system (1300) may also include an interface (1354) to one or more communication networks (1355). The network may be, for example, wireless, wired, or optical. The network may also be local, wide, metropolitan, vehicular and industrial, real-time, delay-tolerant, etc. Examples of networks include: local area networks, such as Ethernet, wireless LAN (Local Area Network, LAN); cellular networks, including GSM (Global System for Mobile Communications, GSM), 3G (The third Generation, 3G), 4G (The fourth Generation, 4G), 5G (The fifth Generation, 5G), LTE (Long Term Evolution), etc.; TV wired or wireless wide area digital networks, including cable TV, satellite TV, and terrestrial broadcast TV; vehicle and industrial networks including CANBus (Controller Area Network Bus, CANBus), etc. Some networks typically require an external network interface adapter attached to some general-purpose data port or peripheral bus (1349) (e.g., a USB port of the computer system (1300)); other networks are typically integrated into the core of the computer system (1300) by attaching to the system bus as described below (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system (1300) can communicate with other entities. Such communications can be one-way receive only (e.g., broadcast TV), one-way send only (e.g., CANBus to some CANBus devices), or two-way, for example, using a local area digital network or a wide area digital network to other computer systems. Certain protocols and protocol stacks can be used on each of these networks and network interfaces as described above.

[0160] The above-mentioned human-machine interface device, human-accessible storage device, and network interface may be attached to the core ( 1340 ) of the computer system ( 1300 ).

[0161] The core (1340) may include one or more central processing units (CPUs) (1341), graphics processing units (GPUs) (1342), specialized programmable processing units in the form of field programmable gate areas (FPGAs) (1343), hardware accelerators (1344) for certain tasks, a graphics adapter (1350), and the like. These devices, along with read-only memory (ROM) (1345), random access memory (1346), and internal mass storage devices such as internal non-user accessible hard drives, SSDs, and the like (1347), may be connected via a system bus (1348). In some computer systems, the system bus (1348) may be accessed in the form of one or more physical plugs to enable expansion with additional CPUs, GPUs, and the like. Peripheral devices may be attached to the core's system bus (1348) directly or via a peripheral bus (1349). In an example, a screen (1310) may be connected to a graphics adapter (1350). Peripheral bus architectures include PCI (Peripheral Component Interconnect, PCI), USB (Universal Serial Bus, USB), and the like.

[0162] The CPU (1341), GPU (1342), FPGA (1343), and accelerator (1344) can execute certain instructions, which, when combined, can constitute the computer code mentioned above. The computer code can be stored in ROM (1345) or RAM (1346). Transient data can also be stored in RAM (1346), while permanent data can be stored in, for example, an internal mass storage device (1347). Fast storage and retrieval of any of the memory devices can be achieved by using a cache memory, which can be closely associated with one or more CPUs (1341), GPUs (1342), mass storage devices (1347), ROM (1345), RAM (1346), etc.

[0163] The computer readable medium may have computer code thereon for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of a type well known and available to those skilled in the computer software arts.

[0164] By way of example and not limitation, a computer system (1300) having an architecture and in particular a core (1340) can provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be media associated with a user-accessible mass storage device as described above, as well as certain storage devices of the core (1340) having non-transitory properties, such as a mass storage device (1347) or ROM (1345) internal to the core. Software implementing various embodiments of the present disclosure can be stored in such a device and executed by the core (1340). Depending on specific needs, the computer-readable medium may include one or more memory devices or chips. The software can cause the core (1340) and in particular the processor therein (including a CPU, GPU, FPGA, etc.) to perform specific processing or specific parts of specific processing described herein, including defining data structures stored in RAM (1346) and modifying such data structures according to processing defined by the software. Additionally or alternatively, the computer system may provide functionality provided as a result of hard-wired logic or otherwise embodied in circuitry (e.g., accelerator (1344)) that may operate in place of or in conjunction with software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, references to software may include logic, and references to logic may include software. Where appropriate, references to computer-readable media may include circuitry (e.g., an integrated circuit (IC)) storing software for execution, circuitry embodying logic for execution, or both. The present disclosure encompasses any suitable combination of hardware and software.

[0165] Although the present disclosure has described several exemplary embodiments, there are changes, permutations, and various substitute equivalents that fall within the scope of the present disclosure. It will therefore be appreciated that although not explicitly shown or described herein, those skilled in the art will be able to devise many systems and methods that embody the principles of the present disclosure and are therefore within the spirit and scope of the present disclosure.

Claims

1. A method for point cloud decoding, characterized in that: The method comprises: receiving, by a processor, a bit stream carrying compressed data of the point cloud; The processor determines, based on a value of an isolation mode flag and a segmentation depth of the current node, whether a current node in an octree structure conforms to an isolation mode, the octree structure corresponding to a three-dimensional (3D) segmentation of the space of the point cloud; and Determining, by the processor, a single isolated point flag of the current node based on information of another node, the single isolated point flag indicating whether the current node is encoded as a single isolated point, wherein the information of the other node includes a parent node of the current node, and the information of the other node indicates whether the parent node of the current node conforms to an isolated mode; If the parent node of the current node conforms to the isolation mode and has one child node, the processor infers that the single isolated point flag of the current node has a false value, indicating that the current node is not encoded as a single isolated point.

2. The method according to claim 1, characterized in that The method further comprises: The processor determines the single isolated point flag based on information of at least one of a grandparent node and a sibling node of the current node, the single isolated point flag indicating whether the current node is encoded as a single isolated point.

3. The method according to claim 1, wherein The method further comprises: determining, by the processor, a context model based on the number of child nodes of the parent node; and The processor decodes a single isolated point flag of the current node from the bit stream based on the context model.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: In response to the single isolated point flag having a true value, the coordinates of the single isolated point are decoded from the bitstream by the processor.

5. The method according to any one of claims 1 to 3, characterized in that The method further comprises: In response to the single outlier flag having a false value, the processor decodes an occupancy code of the current node from the bit stream.

6. The method according to any one of claims 1 to 2, characterized in that The method further comprises: Determining, by the processor, a context model based on information of sibling nodes of the current node; and The processor decodes a single isolated point flag of the current node from the bit stream based on the context model.

7. A method for point cloud coding, characterized in that The method comprises: The processor obtains the point cloud; The processor determines, based on a value of an isolation mode flag and a segmentation depth of the current node, whether a current node in an octree structure conforms to an isolation mode, the octree structure corresponding to a three-dimensional (3D) segmentation of the space of the point cloud; and Determining, by the processor, a single isolated point flag of the current node based on information of another node, the single isolated point flag indicating whether the current node is encoded as a single isolated point, wherein the information of the other node includes a parent node of the current node, and the information of the other node indicates whether the parent node of the current node conforms to an isolated mode; If the parent node of the current node conforms to the isolation mode and has one child node, the processor infers that the single isolated point flag of the current node has a false value, indicating that the current node is not encoded as a single isolated point; A bitstream carrying compressed data of the point cloud is generated by the processor.

8. A computer device, characterized in that: The computer device comprises: one or more computer-readable non-transitory storage media configured to store computer program code; and One or more computer processors configured to access the computer program code and execute the method according to any one of claims 1 to 7 as instructed by the computer program code.

9. A device for point cloud decoding, characterized in that: The apparatus comprises a processing circuit configured to: receiving a bitstream carrying compressed data of a point cloud; determining whether a current node in an octree structure conforms to an isolation mode based on a value of an isolation mode flag and a segmentation depth of the current node, the octree structure corresponding to a three-dimensional (3D) segmentation of a space of the point cloud; as well as Determining a single isolated point flag of the current node based on information of another node, the single isolated point flag indicating whether the current node is encoded as a single isolated point, wherein the information of the other node includes a parent node of the current node, and the information of the other node indicates whether the parent node of the current node conforms to an isolated mode; If the parent node of the current node conforms to the isolation mode and has one child node, it is inferred that the single isolated point flag of the current node has a false value, indicating that the current node is not encoded as a single isolated point.

10. The device according to claim 9, characterized in that The processing circuit is configured to: The single isolated point flag is determined based on information of at least one of a grandparent node and a sibling node of the current node, and the single isolated point flag indicates whether the current node is encoded as a single isolated point.

11. The device according to claim 9, wherein The processing circuit is configured to: determining a context model based on the number of child nodes of the parent node; and A single isolated point flag of the current node is decoded from the bitstream based on the context model.

12. The device according to any one of claims 9 to 11, characterized in that The processing circuit is configured to: In response to the single isolated point flag having a true value, the coordinates of the single isolated point are decoded from the bitstream.

13. The device according to any one of claims 9 to 11, characterized in that The processing circuit is configured to: In response to the single outlier flag having a false value, an occupancy code of the current node is decoded from the bit stream.

14. The device according to any one of claims 9 to 10, characterized in that The processing circuit is configured to: Determine a context model based on information of sibling nodes of the current node; and A single isolated point flag of the current node is decoded from the bitstream based on the context model.

15. A non-transitory computer-readable medium storing instructions, characterized in that: When the instructions are executed by a computer for point cloud decoding, the computer is caused to perform the method according to any one of claims 1 to 7.

16. A method for storing a bit stream, characterized in that: Execute the method of claim 7 to generate a bit stream; and store the bit stream.

17. A method for transmitting a bit stream, characterized in that: Execute the method of claim 7 to generate a bit stream; and transmit the bit stream.

18. A computer-readable storage medium having stored thereon a computer program / instruction and a bit stream, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to claim 7 are implemented to generate the bit stream.

Images