Adaptive selection of occupancy map precision

Abstract

A coding device, a coding method and a decoding device for point cloud compression of a 3D point cloud. The coding device is configured to generate at least a set of geometry frames and a set of occupancy map frames of points of a 3D point cloud for a three-dimensional (3D) point cloud. The coding device is also configured to select an occupancy precision value based on a quantization parameter (QP) associated with at least one generated geometry frame in the set of geometry frames, subsample at least one occupancy map frame in the set of occupancy map frames based on the selected occupancy precision value, and encode the set of geometry frames and the set of occupancy map frames into a bitstream for transmission.

Description

Technical Field

[0001] The present disclosure relates generally to processing multimedia data. More specifically, the present disclosure relates to methods and apparatus for adaptive selection of occupancy map precision for downsampling and upsampling of occupancy maps during processing of point clouds. Background Art

[0002] A point cloud is a collection of three-dimensional (3D) points with attributes. Attributes may include geometric coordinates, colors, normals, texture coordinates, material attributes, motion, etc. In a video-based point cloud (V-PCC) method for processing point clouds, for each input point cloud frame, the geometry and attributes are clustered and projected onto different planes (e.g., xy, yz, zx planes) based on some criteria (e.g., normal direction, distance to the projection frame, continuity, etc.). Each cluster in the cluster is displayed as a patch when projected onto their respective planes. The patches are classified and packed into 2D video frames that store the geometry and texture information of the point cloud. Each of the 2D video frames is then encoded with a video codec such as HEVC, AVC, VP9, ​​VP8, JVET, AVC, etc. The patches may be irregularly shaped, so a binary occupancy map is used to indicate the pixels containing valid points in the geometry and texture video. The compressed video and the compressed occupancy map are multiplexed together with some auxiliary information for tile creation to form an output bitstream. The output bitstream can be received by a decoder to reconstruct the point cloud. Summary of the invention

[0003] Technical Solution

[0004] Embodiments of the present disclosure include an encoding device, an encoding method, and a decoding device for point cloud compression of a 3D point cloud.

[0005] In one embodiment, an encoding device is provided. The encoding device includes a communication interface configured to send a bitstream, and a processor operably connected to the communication interface, the processor configured to cause the encoding device to generate at least a set of geometry frames for a three-dimensional (3D) point cloud, the set of geometry frames providing coordinates of points in the 3D point cloud; generate a set of occupancy map frames indicating whether the points of the 3D point cloud are valid for encoding; and select an occupancy precision value based on a quantization parameter (QP) associated with at least one of the generated geometry frames in the set of geometry frames, wherein when the QP is less than a first QP threshold, the selected occupancy precision value is a first value, wherein when the QP is between the first QP threshold and the second QP threshold, the selected occupancy precision value is a second value higher than the first value, and wherein when the QP is greater than the second threshold, the selected occupancy precision value is a third value higher than the second value. The processor also causes the encoding device to subsample at least one occupancy map frame in the set of occupancy map frames based on the selected occupancy precision value; encode the set of geometry frames and the set of occupancy map frames into a bitstream; and send the bitstream.

[0006] In another embodiment, a method for point cloud compression of a 3D point cloud is provided. The method includes generating at least a set of geometry frames for a three-dimensional (3D) point cloud, the set of geometry frames providing coordinates of points in the 3D point cloud; generating a set of occupancy map frames indicating whether points of the 3D point cloud are valid for encoding; and selecting an occupancy precision value based on a quantization parameter (QP) associated with at least one generated geometry frame in the set of geometry frames, wherein when the QP is less than a first QP threshold, the selected occupancy precision value is a first value, wherein when the QP is between the first QP threshold and the second QP threshold, the selected occupancy precision value is a second value higher than the first value, and wherein when the QP is greater than the second threshold, the selected occupancy precision value is a third value higher than the second value. In addition, the method includes subsampling at least one occupancy map frame in the set of occupancy map frames based on the selected occupancy precision value; encoding the set of geometry frames and the set of occupancy map frames into a bitstream; and transmitting the bitstream.

[0007] In another embodiment, a decoding device for decoding point cloud compression of a 3D point cloud is provided. The decoding device includes a communication interface configured to receive a bitstream, and a processor operably connected to the communication interface, the processor being configured to cause the decoding device to decode the bitstream into a set of geometry frames, the set of geometry frames providing coordinates of points in a 3D point cloud; decode the bitstream into a set of occupancy map frames indicating whether the points of the 3D point cloud are valid for encoding; and obtain an occupancy precision value based on a quantization parameter (QP) associated with at least one decoded geometry frame in the set of geometry frames, wherein when the QP is less than a first QP threshold, the identified occupancy precision value is a first value, wherein when the QP is between the first QP threshold and the second QP threshold, the identified occupancy precision value is a second value higher than the first value, and wherein when the QP is greater than the second threshold, the identified occupancy precision value is a third value higher than the second value. The processor also causes the decoding device to upsample at least one occupancy map frame in the set of occupancy map frames based on the obtained occupancy precision value; and generate a 3D point cloud using the set of geometry frames and the set of occupancy map frames.

[0008] Other technical features may be easily apparent to those skilled in the art from the following drawings, descriptions and claims.

[0009] Before the following detailed description, it may be advantageous to set forth the definitions of certain words and phrases used throughout this patent document. The term "coupling" and its derivatives refer to any direct or indirect communication between two or more elements, regardless of whether those elements are in physical contact with each other. The terms "send", "receive" and "communication" and their derivatives include direct and indirect communication. The terms "include" and "comprise" and their derivatives mean including but not limited to. The term "or" is inclusive, meaning and / or. The phrase "associated with..." and its derivatives refer to including, being included, interconnected with..., including, being included, being connected to or with..., being connected to or with..., being coupled to or with..., being communicable with..., being collaborative with..., being interlaced, being juxtaposed, being close to, being combined with or with..., being combined with, having, having... attributes, being related to..., etc. The term "controller" represents any device, system or part thereof that controls at least one operation. Such a controller may be implemented in hardware, or in a combination of hardware and software and / or firmware. The functions associated with any particular controller may be centralized or distributed, whether locally or remotely. The phrase "at least one of" when used with a list of items means that different combinations of one or more of the listed items may be used, and only one item in the list may be required. For example, "at least one of A, B, and C" includes any of the following combinations: A, B, C, A and B, A and C, B and C, and A and B and C. Likewise, the term "set" means one or more. Thus, a set of items can be a single item or a set of two or more items.

[0010] In addition, the various functions described below can be implemented or supported by one or more computer programs, each of which is formed by a computer-readable program code and embodied in a computer-readable medium. The terms "application" and "program" refer to one or more computer programs, software components, instruction sets, programs, functions, objects, classes, instances, related data or parts thereof suitable for implementation in a suitable computer-readable program code. The phrase "computer-readable program code" includes any type of computer code, including source code, object code and executable code. The phrase "computer-readable medium" includes any type of medium that can be accessed by a computer, such as a read-only memory (ROM), a random access memory (RAM), a hard drive, a compact disk (CD), a digital video disk (DVD) or any other type of memory. "Non-transitory" computer-readable media does not include wired, wireless, optical or other communication links that propagate temporary electrical signals or other signals. Non-transitory computer-readable media include media in which data can be permanently stored and media in which data can be stored and subsequently rewritten, such as rewritable optical disks or erasable storage devices.

[0011] Definitions for other specific words and phrases are provided throughout this patent document. Those of ordinary skill in the art should understand that in many, if not most instances, such definitions apply to prior and future uses of such defined words and phrases.

[0012] Advantageous Effects of the Invention

[0013] The present disclosure relates to methods and apparatus for adaptive selection of occupancy map accuracy for downsampling and upsampling of occupancy maps during processing of point clouds. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] For a more complete understanding of the present disclosure and its advantages, reference is now made to the following description taken in conjunction with the accompanying drawings, in which:

[0015] Figure 1 An exemplary networked computing system according to various embodiments of the present disclosure is shown;

[0016] Figure 2 An exemplary server according to various embodiments of the present disclosure is shown;

[0017] Figure 3 An exemplary electronic device according to various embodiments of the present disclosure is shown;

[0018] Figure 4 shows a point cloud and an exemplary mesh according to an embodiment of the present disclosure;

[0019] Figure 5A shows an exemplary 3D point cloud according to various embodiments of the present disclosure, Figure 5B shows a corresponding 2D frame of an exemplary 3D point cloud according to various embodiments of the present disclosure, Figure 5C shows a corresponding 2D frame of an exemplary 3D point cloud according to various embodiments of the present disclosure, and Figure 5D shows a corresponding 2D frame of an exemplary 3D point cloud according to various embodiments of the present disclosure;

[0020] Figure 6 shows distortion rate curves for various occupancy accuracy values ​​according to various embodiments of the present disclosure;

[0021] Figure 7 shows comparative bit curves for encoding occupancy map frames and geometry frames according to various embodiments of the present disclosure;

[0022] Figure 8 An exemplary encoding unit according to various embodiments of the present disclosure is shown;

[0023] Fig. 9 An exemplary decoding unit according to various embodiments of the present disclosure is shown;

[0024] Fig.10 A flowchart showing a process of adaptively selecting an occupancy accuracy value according to various embodiments of the present disclosure is shown;

[0025] Fig.11 A flowchart showing another process for adaptively selecting an occupancy accuracy value according to various embodiments of the present disclosure is shown;

[0026] Fig. 12A shows an occupancy map processed using a sub-block indicator according to various embodiments of the present disclosure, Fig. 12B shows an occupancy map processed using a sub-block indicator according to various embodiments of the present disclosure, and Fig. 12C An occupancy map processed using a sub-block indicator according to various embodiments of the present disclosure is shown;

[0027] Fig.13 A flow chart for generating an upsampled occupancy map using a sub-block indicator according to various embodiments of the present disclosure is shown;

[0028] Fig.14A shows valid pixels and invalid pixels in an occupancy map according to various embodiments of the present disclosure, Fig. 14B shows valid pixels and invalid pixels in an occupancy map according to various embodiments of the present disclosure, and Fig. 14C shows valid pixels and invalid pixels in an occupancy map according to various embodiments of the present disclosure;

[0029] Fig.15 A flowchart of selecting an occupancy accuracy value based on a comparison of valid pixels and invalid pixels according to various embodiments of the present disclosure is shown;

[0030] Fig.16 A flow chart illustrating restricted occupancy map filtering at a decoder according to various embodiments of the present disclosure is shown;

[0031] Fig.17 A flowchart of a process for encoding a 3D point cloud according to various embodiments of the present disclosure is shown;

[0032] Fig.18 A flowchart of a process for decoding a 3D point cloud according to various embodiments of the present disclosure is shown;

[0033] Fig.19 A flowchart of occupancy map trimming at an encoder according to various embodiments of the present disclosure is shown;

[0034] Fig. 20 A flow chart showing occupancy map grooming at a decoder according to various embodiments of the present disclosure is shown;

[0035] Fig.21Ashows the types of boundary blocks used in occupancy map trimming based on predefined occupancy patterns according to various embodiments of the present disclosure, Fig.21B shows the types of boundary blocks used in occupancy map trimming based on predefined occupancy patterns according to various embodiments of the present disclosure, Fig. 21C shows the types of boundary blocks used in occupancy map trimming based on predefined occupancy patterns according to various embodiments of the present disclosure, and Fig.21D shows the types of boundary blocks used in occupancy map trimming based on predefined occupancy patterns according to various embodiments of the present disclosure;

[0036] Fig. 22 A flowchart of occupancy map shaping at an encoder based on a predefined occupancy pattern according to various embodiments of the present disclosure is shown;

[0037] Fig.23 A flowchart of occupancy map shaping at a decoder based on predefined occupancy patterns according to various embodiments of the present disclosure is shown;

[0038] Fig.24A shows exemplary predefined occupancy modes according to various embodiments of the present disclosure, and Fig. 24B shows exemplary predefined occupancy patterns according to various embodiments of the present disclosure;

[0039] Fig.25A Shown using the Fig.24A downsampled and upsampled occupancy maps of predefined occupancy patterns, Fig.25B Shown using the Fig. 24B downsampled and upsampled occupancy maps of predefined occupancy patterns, and Fig.25C Shown using the Figures 24A to 24B downsampled and upsampled occupancy maps of predefined occupancy patterns;

[0040] Fig.26A shows additional predefined occupancy patterns for boundary blocks according to various embodiments of the present disclosure, and Fig.26B Additional predefined occupancy patterns for boundary blocks according to various embodiments of the present disclosure are shown;

[0041] Fig. 27 A flowchart of occupancy map grooming at an encoder according to various embodiments of the present disclosure is shown;

[0042] Fig.28 A flow chart showing occupancy map grooming at a decoder according to various embodiments of the present disclosure is shown;

[0043] Fig.29 A flow chart showing occupancy map grooming at a decoder according to various embodiments of the present disclosure is shown;

[0044] Fig.30 A flow chart for determining trimming of an occupancy map according to various embodiments of the present disclosure is shown;

[0045] Fig.31 A flow chart showing occupancy map grooming at a decoder according to various embodiments of the present disclosure is shown;

[0046] Fig.32A shows an occupancy map of a boundary block with trimming based on outer corner points according to various embodiments of the present disclosure, Fig.32B An occupancy map of a boundary block with trimming based on outer corner points according to various embodiments of the present disclosure is shown, and Fig.32C An occupancy map of a boundary block with trimming based on exterior corner points according to various embodiments of the present disclosure is shown;

[0047] Fig.33 A flowchart for identifying a boundary block for trimming based on an outer corner point according to various embodiments of the present disclosure is shown;

[0048] Fig.34 A flowchart of boundary block trimming based on external corner points at a decoder according to various embodiments of the present disclosure is shown;

[0049] Fig.35 A flow chart showing occupancy map grooming at an encoder using zero-order hold according to various embodiments of the present disclosure; and

[0050] Fig.36 A flow chart of occupancy map grooming at a decoder using zero-order hold is shown according to various embodiments of the present disclosure. DETAILED DESCRIPTION

[0051] Included in this article Figure 1-36 The various embodiments used to describe the principles of the present disclosure are merely exemplary and should not be interpreted in any way as limiting the scope of the present disclosure. Those skilled in the art will appreciate that the principles of the present disclosure may be implemented in any appropriately arranged wireless communication system.

[0052] Point clouds represent volumetric visual data. Point clouds are composed of multiple 3D points located in 3D space. Each point in a 3D point cloud includes a geometric position represented by a triplet (X, Y, Z) coordinate value. When each point is identified by three coordinates, the exact position in the 3D environment or space is identified. The position of each point in the 3D environment or space can be relative to the origin, other points of the point cloud, or a combination thereof. The origin is the location where the X-axis, Y-axis, and Z-axis intersect. In some embodiments, the point is located on the outer surface of the object. In other embodiments, the point is located on the entire internal structure and outer surface of the object. In addition, depending on the application, each point in the point cloud can also include additional attributes, such as color (also known as texture), reflectivity, intensity, surface normal, etc. In some embodiments, a single point of a 3D point cloud can have multiple attributes. For example, in addition to the geometric positioning of the point (such as the position of the point in 3D space), each point can include an attribute corresponding to the color of the point, another attribute corresponding to the reflectivity of the point, and additional attributes such as intensity, surface normal, or any combination thereof. In some applications, point clouds can also be used to approximate light field data, where each point includes multiple view-dependent color information (R, G, B or Y, U, V triplets).

[0053] A single point cloud can contain billions of points, each associated with a geometric location and one or more attributes. The geometric location and each additional attribute associated with the point takes up a certain number of bits. For example, the geometric location of a single point in a point cloud can consume 30 bits. For example, if each geometric location of a single point is defined by an X value, a Y value, and a Z value, then each coordinate (X, Y, and Z) uses ten bits, totaling thirty bits. Similarly, a texture corresponding to the color of a point cloud can consume 24 bits. For example, if the color component of a single point is defined based on a red value, a green value, and a blue value, then each color component (red, green, and blue) uses eight bits, totaling twenty-four bits. Therefore, a single point with 10 bits of geometric attribute data per coordinate and 8 bits of color attribute data per color value takes up 54 bits. Each additional attribute increases the number of bits required for a single point. If a frame includes one million points, the number of bits per frame is fifty-four million bits (fifty-four bits per point multiplied by one million points per frame). If the frame rate is 30 frames per second, and there is no compression, 1.62 gigabytes per second (54 million bits per second times 30 frames per second) are sent from one electronic device to another electronic device in order for the second device to display the point cloud. Therefore, due to the size and complexity of the data associated with a single point cloud, sending an uncompressed point cloud from one electronic device to another electronic device uses a large amount of bandwidth. As a result, the point cloud is compressed before transmission.

[0054] Since point clouds contain a lot of data, they need to be compressed for efficient storage and transmission. In order to be able to use existing hardware, the Point Cloud Compression (PCC) codec can operate based on projecting (mapping) 3D coordinates and attributes to 2D video frames (images), and then compressing the 2D video frames (images) using existing video codecs (such as HEVC or AVC). In particular, 3D coordinates can be mapped to a set of geometry frames, and attributes can be mapped to a set of attribute frames.

[0055] When a 2D frame is generated, an occupancy map is also generated. As used herein, the term "occupancy map" can be used interchangeably with "occupancy map frame". Typically, for each pixel in a corresponding 2D frame (e.g., a geometry frame and / or an attribute frame), the occupancy map indicates whether the pixel is a valid pixel or an invalid pixel. For example, if the pixel at coordinate (u, v) in the occupancy map is valid, the corresponding pixel in the geometry frame and the corresponding attribute frame at coordinate (u, v) is also valid. If the pixel at coordinate (u, v) in the occupancy map is invalid, the decoder skips the corresponding pixel in the geometry and / or texture frame at coordinate (u, v). In some embodiments, when the occupancy map is a binary occupancy map, the occupancy value at position (u, v) can be 1 or 0. Typically the occupancy map is binary, so the value of each pixel is either 1 or 0. In a non-limiting embodiment, when the pixel value at position (u, v) of the occupancy map is 1, it indicates that the pixel at position (u, v) of the attribute frame and the geometry frame is valid. In contrast, when the pixel value at the location (u, v) of the occupancy map is zero, it indicates that the pixel at the location (u, v) of the attribute frame and the geometry frame is invalid.

[0056] In order to reduce overhead, MPEG V-PCC allows the occupancy map to be subsampled in the encoder by a factor equal to the occupancy precision value before encoding. The occupancy precision value is a sampling factor that determines the number of bits required to encode the occupancy map. As used herein, "occupancy precision" is a syntax element that can be used interchangeably with "occupancy precision value" or "occupancy precision value". The lower the value of occupancy precision, the more bits of the occupancy map and the higher the resolution. The higher the occupancy precision value, the fewer bits of the occupancy map and the lower the resolution. In the decoder, the occupancy map is upsampled back to the original resolution based on the occupancy precision value and is used. The default occupancy precision value of 4 is usually used (i.e., the occupancy map is subsampled by a factor of 4 in both the horizontal and vertical directions). In general, occupancy precision = 4 will reduce the bit rate, but will have a negative impact on distortion (quality) because in the decoder, pseudo points will be introduced into the reconstructed point cloud. Similarly, occupancy precision = 2 will have a higher bit rate than occupancy precision = 4, but less distortion because fewer pseudo points are introduced on average.

[0057] In the MPEG-V-PCC reference software, the value of occupancy precision for any point cloud is set externally via user input. For some point cloud compression scenarios, the default values ​​of occupancy precision and the user-defined occupancy precision values ​​may not be sufficient. Embodiments of the present disclosure recognize the foregoing deficiencies and provide systems and methods for adaptive selection of occupancy precision values ​​for encoding occupancy maps.

[0058] Figure 1 An exemplary networked computing system according to various embodiments of the present disclosure is shown. Communication system 100 includes a network 102 that facilitates communication between various components in communication system 100. For example, network 102 can transmit IP packets, frame relay frames, asynchronous transfer mode (ATM) cells, or other information between network addresses. Network 102 includes one or more local area networks (LANs), metropolitan area networks (MANs), wide area networks (WANs), all or part of a global network such as the Internet, or one or more other communication systems at one or more locations.

[0059] In this example, the network 102 facilitates communication between the server 104 and various client devices 106-116. The client devices 106-116 may be, for example, smart phones, tablet computers, laptop computers, personal computers, wearable devices, head-mounted displays, etc. The server 104 may represent one or more servers. Each server 104 includes any suitable computing or processing device capable of providing computing services to one or more client devices (such as client devices 106-116). Each server 104 may, for example, include one or more processing devices, one or more memories storing instructions and data, and one or more network interfaces facilitating communication through the network 102. As described in more detail below, the server 104 may send a compressed bitstream representing a point cloud to one or more display devices, such as client devices 106-116. In some embodiments, each server 104 may include an encoder and / or decoder for processing a 3D point cloud.

[0060] Each client device 106-116 represents any suitable computing or processing device that interacts with at least one server (such as server 104) or other computing device through network 102. Client devices 106-116 include desktop computers 106, mobile phones or mobile devices 108 (such as smart phones), PDAs 110, laptop computers 112, tablet computers 114, and head-mounted displays 116. However, any other or additional client devices may be used in the communication system 100. Smart phones represent a class of mobile devices 108, which are handheld devices with mobile operating systems and integrated mobile broadband cellular network connections for voice, short message service (SMS), and Internet data communications. In some embodiments, any of the client devices 106-116 may include an encoder, a decoder, or both. For example, the mobile device 108 may record a video and then encode the video so that the video can be sent to one of the client devices 106-116. In another example, the laptop computer 112 may be used to generate a virtual 3D point cloud, which is then encoded and sent to one of the client devices 106-116.

[0061] In this example, some of the client devices 108-116 communicate indirectly with the network 102. For example, the mobile device 108 and the PDA 110 communicate via one or more base stations 118, such as cellular base stations or eNodeBs (eNBs). In addition, the laptop computer 112, the tablet computer 114, and the head mounted display 116 communicate via one or more wireless access points 120, such as IEEE 802.11 wireless access points. Note that these are for illustration only, and each client device 106-116 may communicate directly with the network 102, or indirectly with the network 102 via any suitable intermediate device or network. In some embodiments, the server 104 or any of the client devices 106-116 may be used to compress the point cloud, generate a bitstream representing the point cloud, and send the bitstream to another client device, such as any of the client devices 106-116.

[0062] In certain embodiments, any client device can securely and efficiently send information to another device, such as server 104. In addition, any of client devices 106-116 can trigger the transfer of information between itself and server 104.

[0063] In some embodiments, any of the client devices 106-116 or the server 104 may create a 3D point cloud, compress a 3D point cloud, send a 3D point cloud, receive a 3D point cloud, render a 3D point cloud, or a combination thereof. For example, the server 104 receives the 3D point cloud, decomposes the 3D point cloud to fit a 2D frame, and compresses the frame to generate a bitstream. The bitstream may be sent to a storage device, such as an information library, or one or more of the client devices 106-116. For another example, one of the client devices 106-116 may receive the 3D point cloud, decompose the 3D point cloud to fit a 2D frame, compress the frame to generate a bitstream, which may be sent to a storage device, such as an information library, another of the client devices 106-116, or to the server 104.

[0064] Although Figure 1 One example of a communication system 100 is shown, but may be Figure 1 Various changes may be made. For example, communication system 100 may include any number of each component in any suitable arrangement. In general, computing and communication systems come in a variety of configurations, and Figure 1 The scope of the present disclosure is not limited to any particular configuration. Figure 1 One operating environment is shown in which the various features disclosed in this patent document may be used, but these features may be used in any other suitable system.

[0065] Figure 2 An exemplary server according to various embodiments of the present disclosure is shown. The server 200 may represent Figure 1 Server 200 may represent one or more encoders, decoders, local servers, remote servers, cluster computers, and components that act as a single pool of seamless resources, cloud-based servers, etc. Server 200 may be Figure 1 One or more client devices 106-116 or another server for access.

[0066] Server 200 includes a bus system 205 that supports communication between at least one processing device, such as processor 210 , at least one storage device 215 , at least one communication interface 220 , and at least one input / output (I / O) unit 225 .

[0067] The processor 210 executes instructions that may be stored in the memory 230. The processor 210 may include any suitable number and type of processors or other devices in any suitable arrangement. Example types of processors 210 include microprocessors, microcontrollers, digital signal processors, field programmable gate arrays, application specific integrated circuits, and discrete circuits. In some embodiments, the processor 210 may execute instructions to cause the encoding unit 800 to encode the 3D point cloud stored in the storage device 215 according to various embodiments discussed in more detail in the following figures. For example, the encoding unit 800 may adaptively select an occupancy precision value based on a predetermined criterion to reduce the overhead required to send an encoded bitstream of the 3D point cloud.

[0068] Memory 230 and permanent storage 235 are examples of storage 215, which represents any structure capable of storing and facilitating retrieval of information, such as data, program code, or other suitable information on a temporary or permanent basis. Memory 230 may represent random access memory or any other suitable volatile or non-volatile storage device. For example, the instructions stored in memory 230 may include instructions for decomposing a point cloud into tiles, instructions for packing tiles on a 2D frame, instructions for compressing a 2D frame, instructions for determining an occupancy precision value for subsampling a set of occupancy maps, and instructions for encoding a 2D frame to generate a bitstream. Permanent storage 235 may include one or more components or devices that support longer-term data storage, such as a read-only memory, a hard drive, a flash memory, or an optical disk.

[0069] Communication interface 220 supports communication with other systems or devices. For example, communication interface 220 may include a network interface card or a wireless transceiver to facilitate communication via Figure 1 The communication interface 220 may communicate with the network 102 of FIG. 106 . The communication interface 220 may support communication via any suitable physical or wireless communication link. For example, the communication interface 220 may send a bitstream containing a 3D point cloud to another device, such as one of the client devices 106 , 116 .

[0070] I / O unit 225 allows input and output of data. For example, I / O unit 225 can provide a connection for user input via a keyboard, mouse, keypad, touch screen or other suitable input device. I / O unit 225 can also send output to a display, printer or other suitable output device. However, it is noted that I / O unit 225 can be omitted, such as when I / O interaction with server 200 occurs via a network connection.

[0071] Note that although Figure 2 Described as indicating Figure 1104, but the same or similar structure can be used for one or more of the various client devices 106-116. For example, a desktop computer 106 or a laptop computer 112 can have a Figure 2 The same or similar structure as shown.

[0072] Figure 3 An exemplary electronic device according to various embodiments of the present disclosure is shown. The electronic device 300 may represent Figure 1 The electronic device 300 may be a mobile communication device such as, for example, a mobile station, a subscriber station, a wireless terminal, a desktop computer (similar to Figure 1 desktop computer 106), portable electronic devices (similar to Figure 1 In some embodiments, Figure 1 One or more of the client devices 106-116 may include the same or similar configuration as the electronic device 300. In some embodiments, the electronic device 300 may be an encoding device, a decoder device, or both. For example, the electronic device 300 may be used for data transmission, image or video compression, image or video decompression, encoding, decoding, and media rendering applications.

[0073] like Figure 3 As shown in , the electronic device 300 includes an antenna 305, a radio frequency (RF) transceiver 310, a transmission (TX) processing circuit 315, a microphone 320, and a reception (RX) processing circuit 325. The RF transceiver 310 may include, for example, an RF transceiver, a BLUETOOTH transceiver, a WI FI transceiver, a ZIGBEE transceiver, an infrared transceiver, and various other wireless communication signals. The electronic device 300 also includes a speaker 330, a processor 340, an input / output (I / O) interface (IF) 345, an input 350, a display 360, a memory 365, and a sensor 365. The memory 360 includes an operating system (OS) 361 and one or more applications 362.

[0074] The RF transceiver 310 receives an incoming RF signal from an antenna 305 transmitted from an access point (such as a base station, a WI FI router, or a BLUETOOTH device) or other devices of the network 102 (such as WI-FI, BLUETOOTH, cellular, 5G, LTE, LTE-A, WiMAX, or any other type of wireless network). The RF transceiver 310 down-converts the incoming RF signal to produce an intermediate frequency or baseband signal. The intermediate frequency or baseband signal is transmitted to the RX processing circuit 325, which generates a processed baseband signal by filtering, decoding, and / or digitizing the baseband or intermediate frequency signal. The RX processing circuit 325 transmits the processed baseband signal to the speaker 330 (such as for voice data) or to the processor 340 for further processing (such as for web browsing data).

[0075] The TX processing circuit 315 receives analog or digital voice data from the microphone 320, or receives other outgoing baseband data from the processor 340. The outgoing baseband data may include network data, email, or interactive video game data. The TX processing circuit 315 encodes, multiplexes, and / or digitizes the outgoing baseband data to generate a processed baseband or intermediate frequency signal. The RF transceiver 310 receives the outgoing processed baseband or intermediate frequency signal from the TX processing circuit 315 and up-converts the baseband or intermediate frequency signal to an RF signal that is transmitted via the antenna 305.

[0076] The processor 340 may include one or more processors or other processing devices. The processor 340 may execute instructions stored in the memory 360, such as OS 361, to control the overall operation of the electronic device 300. For example, the processor 340 can control the reception of the forward channel signal and the transmission of the reverse channel signal through the RF transceiver 310, the RX processing circuit 225, and the TX processing circuit 315 according to well-known principles. The processor 340 may include any suitable number and type of processors or other devices of any suitable arrangement. For example, in some embodiments, the processor 340 includes at least one microprocessor or microcontroller. Example types of the processor 340 include microprocessors, microcontrollers, digital signal processors, field programmable gate arrays, application specific integrated circuits, and discrete circuits.

[0077] The processor 340 is also capable of executing other processes and programs resident in the memory 360, such as operations to receive and store data. The processor 340 can move data into or out of the memory 360 as needed for the execution process. In some embodiments, the processor 340 is configured to execute one or more applications 362 based on the OS 361 or in response to a signal received from an external source or operator. Exemplary applications 362 may include encoder applications, decoder applications, VR or AR applications, camera applications (for still images and videos), video phone call applications, email clients, social media clients, SMS messaging clients, virtual assistants, etc. In some embodiments, the processor 340 is configured to receive and send media content.

[0078] Processor 340 is also coupled to I / O interface 345, which provides electronic device 300 with the ability to connect to other devices, such as client devices 106-114. I / O interface 345 is the communication path between these accessories and processor 340.

[0079] The processor 340 is also coupled to the input 350 and the display 355. The operator of the electronic device 300 can use the input 350 to enter data or input to the electronic device 300. The input 350 can be a keyboard, a touch screen, a mouse, a trackball, a voice input, or other devices that can act as a user interface to allow the user to interact with the electronic device 300. For example, the input 350 may include a voice recognition process, thereby allowing the user to enter a voice command. In another example, the input 350 may include a touch pad, a (digital) pen sensor, a key, or an ultrasonic input device. The touch panel can recognize, for example, a touch input performed in at least one scheme, such as a capacitive scheme, a pressure-sensitive scheme, an infrared scheme, or an ultrasonic scheme. By providing additional inputs to the processor 340, the input 350 can be associated with a sensor 365 and / or a camera. In some embodiments, the sensor 365 includes one or more inertial measurement units (IMUs) (such as accelerometers, gyroscopes, and magnetometers), motion sensors, optical sensors, cameras, pressure sensors, heart rate sensors, altimeters, etc. The input 350 may also include a control circuit. In the capacitive scheme, the input 350 can recognize touch or proximity.

[0080] Display 355 may be a liquid crystal display (LCD), a light emitting diode (LED) display, an organic LED (OLED), an active matrix OLED (AMOLED), or other display capable of rendering text and / or graphics, such as from a website, video, game, image, etc. Display 355 may be sized for a head mounted display. Display 355 may be a single display screen or multiple display screens capable of producing a stereoscopic display. In some embodiments, display 355 is a head up display (HUD). Display 355 may display 3D objects, such as a 3D point cloud.

[0081] Memory 360 is coupled to processor 340. A portion of memory 360 may include RAM, and another portion of memory 360 may include flash memory or other ROM. Memory 360 may include permanent storage (not shown), which represents any structure capable of storing and facilitating retrieval of information (such as data, program code, and / or other suitable information). Memory 360 may include one or more components or devices that support long-term storage of data, such as read-only memory, a hard drive, flash memory, or an optical disk. Memory 360 may also include media content. Media content may include various types of media, such as images, videos, three-dimensional content, VR content, AR content, 3D point clouds, etc.

[0082] The electronic device 300 also includes one or more sensors 365, which can measure physical quantities or detect the activation state of the electronic device 300, and convert the measured or detected information into electrical signals. For example, the sensor 365 may include one or more buttons for touch input, a camera, a gesture sensor, an IMU sensor (such as a gyroscope or gyroscope sensor and an accelerometer), an eye tracking sensor, an air pressure sensor, a magnetic sensor or a magnetometer, a grip sensor, a proximity sensor, a color sensor, a biophysical sensor, a temperature / humidity sensor, an illumination sensor, an ultraviolet (UV) sensor, an electromyogram (EMG) sensor, an electroencephalogram (EEG) sensor, an electrocardiogram (ECG) sensor, an IR sensor, an ultrasonic sensor, an iris sensor, a fingerprint sensor, a color sensor (such as a red, green, and blue (RGB) sensor), etc. The sensor 365 may also include a control circuit for controlling any sensor included therein.

[0083] As will be discussed in more detail below, in some illustrative embodiments, electronic device 300 may receive an encoded / compressed bitstream. Decoding unit 900 of electronic device 300 may decode / decompress the received bitstream into a plurality of 2D frames. In some embodiments, the decoded bitstream also includes an occupancy map, which decoding unit 900 may upsample using a selected occupancy precision value adaptively selected according to various embodiments disclosed herein.

[0084] The plurality of 2D frames may include a set of frames indicating coordinates, such as the geographic location of each point of the point cloud. For example, a frame may include a graphical depiction, such as one or more tiles of each geometric point of the point cloud represented in 2D. Another set of frames may include a texture (i.e., or other attributes) associated with each point, such as the color of each point. The decoding unit 900 of the electronic device 300 may then reconstruct and render the point cloud in three dimensions.

[0085] In some embodiments, the electronic device 300 can be similar to the server 200 and encode the point cloud. The electronic device 300 can generate multiple 2D frames representing the geometry and texture of the point cloud. The point cloud can be mapped to one or more 2D frames. For example, a set of frames can include geometric points depicted in a tile. In another example, another set of frames can include the texture or color or both of the point cloud. In some embodiments, the data representing the point cloud on the 2D frame can be structured, semi-structured or unstructured.

[0086] although Figure 2 and 3 An example of an electronic device is shown, but the Figure 2 and 3 Make various changes. For example, Figure 2 and Figure 3 The various components in the embodiment may be combined, further subdivided, or omitted, and additional components may be added as required. As a specific example, processor 340 may be divided into multiple processors, such as one or more central processing units (CPUs) and one or more graphics processing units (GPUs). In addition, as with computing and communications, electronic devices and servers may have a variety of configurations, and Figure 2 and 3 The present disclosure is not limited to any particular electronic device or server.

[0087] Figure 4 A point cloud 405 and an example grid 410 are shown according to an embodiment of the present disclosure. Point cloud 405 depicts an illustration of a point cloud. A point cloud is digitized data that visually defines an object in 3D space. As shown, point cloud 405 includes a plurality of points, each of which represents an external coordinate (i.e., geographic location) of an object, similar to a topographic map. In addition, each point may include one or more attributes, such as color, intensity, texture, motion, material properties, etc.

[0088] Similarly, mesh 410 depicts a diagram of a 3D mesh. Mesh 410 is digital data that visually defines an object in 3D space. The graphical depiction of mesh 410 is defined by many polygonal or triangular interconnections of information between points. Each polygon may include various information, such as geometric information that defines the geometric position of each vertex in 3D space. In addition, each polygon may include attributes such as color reflectivity, motion, etc. For example, topological data provides connectivity information between vertices, such as adjacency of vertices, edges, and faces. Attribute information provides normals, colors, and application-related information for each individual vertex. The vertices of each polygon are similar to points in point cloud 405. Each polygon of mesh 410 represents the outer surface of an object.

[0089] Point clouds (such as point cloud 405) and meshes (such as mesh 410) are sent from one computing device to another computing device using considerable bandwidth. Compression is necessary to reduce storage and bandwidth requirements. For example, lossy compression can compress point clouds and meshes while reducing the size of the data while keeping distortion at a tolerable level.

[0090] Figure 5A An exemplary 3D point cloud corresponding to a 2D frame representing the 3D point cloud is shown according to various embodiments of the present disclosure. In particular, Figure 5A A 3D point cloud 510 is shown. A 3D point cloud 510 is a collection of data points in a 3D space. Each point of the 3D point cloud 510 includes at least a geometric position and optionally one or more attributes that provide information about each point, such as color, reflectivity, material, etc.

[0091] Figure 5B , 5C 5D and 5D each show a 2D frame including a plurality of tiles. In particular, Figure 5B A geometric frame 520 is depicted, Figure 5C depicts a property frame 530, Figure 5D An occupancy map frame 540 is depicted. Geometry frame 520 includes information representing geometric locations of points of 3D point cloud 510. In this illustrative example, attribute frame 530 includes information representing at least a color associated with a point of 3D point cloud 510. Other attributes may also be represented in attribute frame 530. Occupancy map frame 540 includes information indicating whether pixels included in the geometry frame and / or attribute frame at various locations in the occupancy map frame represent points of the 3D point cloud. To reiterate, the occupancy map frame indicates whether points of the 3D point cloud in the set of geometry frame and attribute frame are valid for encoding.

[0092] The geometry frame 520 includes a plurality of tiles (such as tile 522) representing depth values ​​of the 3D point cloud 510. The attribute frame 530 includes a plurality of tiles (such as tile 532) representing attributes (such as the color of the 3D point cloud 510). Each color pixel in the attribute frame 530 corresponds to a particular geometry pixel in the geometry frame 520. A mapping may be generated between each pixel in the geometry frame 520 and the attribute frame 530.

[0093] The tiles in the occupancy map frame 540 identify which points of the point cloud 510 are valid for encoding. For example, black pixels are invalid and white pixels are valid. For lossless compression, the tiles in the occupancy map frame 540 correspond directly to similar tiles in the geometry frame 520 and the attribute frame 530. However, in some embodiments disclosed herein, lossy compression is provided, in which case the tiles in the occupancy map frame 540 are slightly different from the tiles in the geometry frame 520 and the attribute frame 530, and the pixels are excluded from the encoding / decoding process.

[0094] Figure 6 Distortion rate curves for various occupancy precision values ​​according to various embodiments of the present disclosure are shown. In particular, graph 600 shows distortion rate curves at different bit rates controlled by QP values. The y-axis of graph 600 is the peak signal-to-noise ratio and is a measure of quality. The x-axis of graph 600 is the bit rate. Curve 602 represents occupancy precision = 4. Curve 604 represents occupancy precision = 2. Curve 606 represents occupancy precision = 1. Graph 600 shows that when considering the combined distortion rate performance, occupancy precision = 4 performs better than occupancy precision = 2 at lower bit rates, where occupancy precision = 1 and 2 perform better at higher bit rates.

[0095] At the decoder, the subsampled occupancy map is upsampled back to the original resolution, resulting in invalid pixels (i.e., points that are not in the original point cloud), which can be filtered out using geometric smoothing filtering to achieve good reconstruction quality. As shown in the comparison of Tables 1 and 2, the proportion of invalid pixels is higher when the occupancy precision value = 4 compared to when the occupancy precision value = 2. Tables 1 and 2 depict the valid and invalid pixels of the 3D point cloud frames of various objects after downsampling and upsampling. Therefore, it is desirable to use a lower occupancy precision value to obtain better reconstruction quality.

[0096] Table 1. Occupancy precision value = 2.

[0097]

Table 1

[0098] Object Number of effective pixels Number of invalid pixels ratio robbery 560283 14033 2.50％ Red and Black 500502 20790 4.15％ soldier 726181 24659 3.40％ Queen 575820 25728 4.47％ Long skirt 534978 13366 2.50％ basketball 2000571 30821 1.50％ Dancer 1767959 32809 1.86％

[0099] Table 2. Occupancy precision value = 4.

[0100]

Table 2

[0101] Object Number of effective pixels Number of invalid pixels ratio robbery 560283 40453 7.22％ Red and Black 500502 57882 11.56％ soldier 726181 71595 9.86％ Queen 575820 63492 11.03％ Long skirt 534978 37982 7.10％ basketball 200571 89061 4.45％ Dancer 1767959 96617 5.46％

[0102] It may be desirable to use lower occupancy precision values ​​to reduce the number of invalid pixels and improve quality. However, the number of bits required to encode the occupancy map increases as the occupancy precision value decreases, as shown in Table 3.

[0103] Table 3. Number of bits required to encode the occupancy map.

[0104]

Table 3

[0105] Occupied precision value = 4 Occupied precision value = 2 Occupied precision value = 1 robbery 21648 40592 87992 Red and Black 31608 58664 125104 soldier 40528 73648 155336 Queen 31672 58464 143000 Long skirt 19760 36560 76384 basketball 43456 84864 183832 Dancer 46632 92520 19979

[0106] Figure 7 Comparative bit curves for encoding an occupancy map frame and a geometry frame according to various embodiments of the present disclosure are shown. When the occupancy map is losslessly encoded, the bits required to encode the occupancy map remain the same, regardless of the bit rate. Curve 702 represents occupancy precision = 4. Curve 704 represents occupancy precision = 2. Curve 706 represents occupancy precision = 1. Curve 708 represents a representative geometry frame.

[0107] As can be seen from graph 700, smaller occupancy precision values ​​provide better quality at the expense of increased bitrate used to encode the occupancy map. The bits used for the occupancy map can be a significant portion of the total bits used to encode the geometry (occupancy map bits + geometry video bits). There is a tradeoff between increased bits and improved quality.

[0108] In addition to improving rate-distortion performance, the disclosed method will improve the visual quality of the reconstructed point cloud at high bitrates. Since smaller occupancy precision values ​​will result in a smaller number of spurious points, the visual quality of the reconstructed point cloud will improve even without smoothing the geometry. An adaptive selection of the occupancy precision value can be used to balance the quality of the reconstructed 3D point cloud and its corresponding bitrate.

[0109] Figure 8 800 is an exemplary encoding unit according to various embodiments of the present disclosure. The encoder 800 may be incorporated into a server, such as Figure 2 or incorporated into one or more electronic devices, such as Figure 3 The electronic device 300 in FIG. 8 shows a high-level overview of an embodiment of the present disclosure of an electronic device for encoding and compressing a point cloud for transmission. In some embodiments, the encoder 800 packages the point cloud for transmission to one or more decoders such as a bitstream. Fig. 9 Decoder 900 in.

[0110] In general, the encoder 800 can compress, encode, and transmit the point cloud, mesh, or both as an encoded bitstream. In some embodiments, the encoder 800 generates multiple 2D frames, such as in Figure 5B and5C In some embodiments, the encoder 800 generates an occupancy map (such as a Figure 5D In some embodiments, the encoder 800 is a network server, a server computer such as a management server, or any other electronic computing system capable of mapping the three dimensions of a point cloud into two dimensions, compressing frames, and encoding images for transmission. In some embodiments, the encoder 800 can be a "cloud" of computers interconnected by one or more networks, each of which utilizes clustered computers and components to communicate over a network. Figure 1 The network 102 acts as a single pool of seamless resources when accessed by computing systems.

[0111] The encoder 800 is configured to receive an input point cloud frame 802 representing a point cloud, such as Figure 5A The point cloud 510 in FIG. The point cloud may be stored in a memory associated with the encoder 800, such as Figure 2 The storage device 230 in the image processing unit 804 is a memory device. The tile generator 804 decomposes the point cloud frame 802 into a plurality of tiles. The tiles can be organized based on geometry and attributes. The geometry is the geographic location of each point of the point cloud, and the attributes represent various aspects of each point of the point cloud, such as color, intensity, normal, reflectivity, etc. The tile generator 804 clusters the geometric attributes and texture attributes. The clustering can be based on criteria such as normal direction, distance to the projection frame, proximity, etc. Each generated cluster is represented as a tile, such as Figure 5B , 5C and as shown in 5D.

[0112] In some embodiments, the tile generator 804 projects and maps the points of the point cloud onto the 2D frame via a one-to-one mapping. In some embodiments, the tile generator 804 projects and maps the points of the point cloud onto the 2D frame (e.g., the geometry frame 808 and the attribute frame 810) by projecting the points onto different planes. In some embodiments, the projection and mapping of the points of the point cloud onto the 2D frame can be based on raster scan mapping, mapping based on X geometric coordinates, mapping based on Y geometric coordinates, mapping based on Z geometric coordinates, color-based mapping, normal direction-based mapping, etc.

[0113] In some embodiments, after projecting and mapping the points of the point cloud onto the 2D frame, the points can be classified and packed in the 2D frame to improve coding efficiency. The packing unit 612 classifies and packs the points within the tile. The packing unit 806 also classifies and packs the tiles into one or more geometry frames 808 and one or more attribute frames 810. In some embodiments, the packing unit 806 creates one or more geometry frames 808 and one or more attribute frames 810.

[0114] In some embodiments, the packing unit 806 may create an occupancy map 812. The occupancy map indicates pixel locations in frames (such as one or more geometry frames 808 and one or more attribute frames 810) that contain valid points of the point cloud that are projected or mapped onto the respective frames. For example, the occupancy map 812 indicates whether each pixel on each frame (such as one or more geometry frames 808 and one or more attribute frames 810) is a valid pixel or blank. The valid pixels on the occupancy map indicate the pixels on the 2D frame that correspond to the 3D points in the point cloud. In some embodiments, the occupancy map 812 may be the same for each frame, such as one or more geometry frames 808 and one or more attribute frames 810. In some embodiments, the occupancy map 812 represents multiple occupancy maps, where each occupancy map corresponds to one frame (such as one or more geometry frames 808 and one or more attribute frames 810). The occupancy map 812 may be included in another frame, similar to the one or more geometry frames 808 or the one or more attribute frames 810. In some embodiments, the occupancy map 812 may be sent as metadata as a separate bitstream.

[0115] The tile generator 804 also generates auxiliary tile information 814. The auxiliary tile information 814 is information associated with each generated tile. In some embodiments, the auxiliary tile information 814 is metadata about the tile. For example, each tile has associated therewith auxiliary information such as (i) a spatial positioning offset, (ii) a depth offset, (iii) the position of the tile in the video frame, (iv) the size of the tile (such as the height and width of the tile), (v) an index of a projection plane, and the like. A decoder (such as Fig. 9 The decoder 900 in FIG. 8 reconstructs the point cloud using the auxiliary tile information 814. To reduce overhead, the auxiliary tile information 814 may be compressed by an auxiliary tile information compression unit 824 before transmission.

[0116] 2D frames (e.g., geometry frame 808, attribute frame 810, and occupancy map frame) may be sent to an image filling unit 816 to fill unfilled areas between projected tiles using an image filling algorithm, thereby reducing the number of sharp edges in the projected video frame, which reduces the compression bit rate.

[0117] One or more geometry frames 808 are compressed by a compression unit 818 using a conventional 2D codec, such as High Efficiency Video Coding (HEVC) or Advanced Video Coding (AVC). To encode attribute frames including color attributes, a set of geometry frames are first decoded to reconstruct 3D coordinates. The reconstructed geometry coordinates are then smoothed using a smoothing unit 820, and their color values ​​are interpolated from the color values ​​of the input coordinates. The resulting colors are then packed into a set of attribute frames 810.

[0118] To reduce overhead, the occupancy map subsampler 822 subsamples the set of occupancy map frames 812 using methods described in various embodiments below. The subsampled set of occupancy map frames may also be referred to as low-resolution occupancy map frames.

[0119] The set of subsampled occupancy map frames 830, compressed geometry frames 826, compressed attribute frames 828, and compressed auxiliary tile information 832 may be multiplexed via a multiplexer 834 to create a decoder such as Fig. 9 A single bit stream 836 of the decoder 900 in FIG.

[0120] Fig. 9 900 shows an exemplary decoding unit according to various embodiments of the present disclosure. The decoding unit 900 can be incorporated into an electronic device, such as Figure 3 electronic device 300 in, or incorporated into one or more servers, such as Figure 2 Server 200 in.

[0121] The decoding unit 900 is configured to decode / decompress an encoded bitstream, such as the encoded bitstream 836 including a representation of a point cloud, a mesh, or both. In some embodiments, the decoding unit 900 generates a plurality of point clouds from a received bitstream including a plurality of 2D frames. For example, each point of the point cloud may be mapped based on information received in the 2D frames and a received occupancy map.

[0122] Upon receiving the encoded bitstream 836 , the demultiplexer 902 separates the encoded bitstream 836 into a compressed geometry frame 826 , a compressed attribute frame 828 , a low-resolution occupancy map frame 830 , and compressed auxiliary tile information 832 .

[0123] The compressed geometry frame 826 and the compressed attribute frame 828 are decompressed by the video decompression unit 904 to generate 2D video frames of geometry 926 and color 928, respectively. The compressed auxiliary tile information 832 is decompressed by the auxiliary path decompression unit 910 to form decompressed auxiliary tile information 932. As used herein, decompression may be used interchangeably with decoding.

[0124] The occupancy map upsampler 906 upsamples the set 830 of low resolution occupancy map frames to form upsampled occupancy map frames 930, which can be passed to the geometry reconstruction unit 908 along with the decompressed auxiliary tile information 932. The set 930 of upsampled occupancy map frames indicates valid pixel locations within the 2D frame to reconstruct a 3D point cloud.

[0125] The geometry reconstruction unit 908 reconstructs the geometry frame data based on the data received from the demultiplexer 902 and / or the video decompression unit 904. The position and size of the regular tile are retrieved from the decompressed auxiliary tile information unit 910. For example, the point is reconstructed from the regular tile by adding the tile offset (from the decompressed auxiliary tile information 932) to the spatial position of the point in the video and adding the depth offset of the tile to the pixel value at the spatial position.

[0126] The geometry reconstruction unit 908 may send the reconstructed geometry frame data to the smoothing unit 912 and then to the attribute reconstruction unit for combining with the information in the decompressed attribute frame 928 to reconstruct a 3D point cloud.

[0127] Fig.10 1000 is a flowchart showing a process for adaptively selecting occupancy map precision according to various embodiments of the present disclosure. The operations described in flowchart 1000 may be implemented in an encoder, such as Figure 8 The encoder 800 in. The encoder may be included in a server, such as Figure 2 Server 200 in, or electronic device, such as Figure 3 The electronic device 300 in.

[0128] In one embodiment of the present invention, an occupancy precision is automatically selected based on a quantization parameter used to quantize a geometric video frame. Flowchart 1000 begins at operation 1002 where a determination is made as to whether the QP is less than QPTHRESH1. If the QP is not less than QPTHRESH1, another determination is made in operation 1004 as to whether the QP is less than QPTHRESH2. If the QP is not less than QPTHRESH2, an occupancy precision of 4 is selected in operation 1006. In operation 1008, the occupancy map frame is subsampled based on the selected occupancy precision, and the process terminates.

[0129] Returning to operation 1002, if QP is less than QPTHRESH1, then occupancy precision 1 is selected in operation 1010 and the flowchart proceeds to operation 1008 where the occupancy map frame is sub-sampled based on the selected occupancy precision before the process terminates.

[0130] At operation 1004, if QP is less than QPTHRESH2, then at operation 1012 an occupancy precision of 2 is selected and the flow chart proceeds to operation 1008 where the occupancy map frame is sub-sampled based on the selected occupancy precision before the process terminates.

[0131] Fig.11 1100 is a flowchart of another process for selecting a variable occupancy precision value according to various embodiments of the present disclosure. Flowchart 1100 may be implemented in an encoder, such as Figure 8 The encoder 800 in. The encoder may be included in a server, such as Figure 2 Server 200 in, or electronic device, such as Figure 3 The electronic device 300 in.

[0132] In the process according to this embodiment, the following variables are defined:

[0133] B o = the number of bits occupied by the image frame;

[0134] B g =Number of bits in the geometry frame;

[0135] R=B o / B g ;

[0136] R T = threshold value of ratio R;

[0137] R TL =low threshold of ratio R;

[0138] R TH = high threshold of ratio R;

[0139] V = percentage of invalid pixels when occupied precision = 4; and

[0140] V T =Threshold value for invalid pixel percentage.

[0141] V determines the quality of the geometry when the occupancy precision = 4. V can be compared with a predefined threshold V T The ratio is compared to determine the threshold R T If V is low, it is better to use a higher occupancy precision value, because the cost of increasing the rate by reducing the occupancy precision value does not offset the improvement in quality. Therefore, the ratio threshold R T is adaptively selected based on V. If V is lower, then the bar (R TH ) is higher to switch to a lower occupancy accuracy. If V is higher, then the bar (R TL ) to switch to a lower occupancy accuracy.

[0142] In one embodiment, the ratio R to R T to determine the occupancy accuracy. g Significantly greater than B o , use a lower occupancy precision value. This logic is used because there is a trade-off between rate and distortion, and using too many bits for the occupancy map by using a lower occupancy precision value may not offset the improvement in geometry quality. The final occupancy map is obtained by subsampling the input occupancy map by a factor of the occupancy precision.

[0143] Back to Fig.11 Flowchart 1100 begins at operation 1102 and makes a decision as to whether V is less than V T If V is less than V T , then in operation 1104, R T Set to R TH However, if V is not less than V T , then in operation 1106, R T Set to R TL .

[0144] In operation 1108, a decision is made as to whether R is less than R T If R is not less than RT, then in operation 1110, the occupied precision 4 is selected. Otherwise, if R is less than R T , then in operation 1112, occupancy precision 2 is selected. In operation 1114, the occupancy map is subsampled using the selected occupancy precision. Thereafter, the process terminates.

[0145] In one or more variations of the embodiments disclosed herein, multiple thresholds may be used to determine other occupancy precision values; the occupancy precision may be automatically selected based on quantization parameters used to quantize geometry, texture, and auxiliary video; the occupancy precision may be automatically selected based on the bits consumed by geometry, texture, and occupancy maps; the occupancy precision may be automatically selected based on the number of tiles; the occupancy precision may be automatically selected based on the number of tile boundary pixels; and the occupancy precision may be selected based on the number of invalid pixels created after upsampling a low-resolution occupancy map in a decoder.

[0146] Fig. 12C An occupancy map processed using sub-block indicators according to various embodiments of the present disclosure is shown. This embodiment will use a larger occupancy precision value for the inner blocks of the occupancy map to improve coding efficiency, while a smaller occupancy precision value will be used for the outer blocks. Since a smaller occupancy precision value will be used for the outer blocks, almost no geometric smoothing is required. This results in a significant reduction in complexity and memory usage in the decoder.

[0147] exist Fig. 12CIn the illustrative example in , the occupancy precision is set to a higher value, such as 2 or 4, but a smaller occupancy precision value (e.g., 2 or 1) can be used for partially filled 4×4 blocks close to tile boundaries. The occupancy values ​​of the sub-blocks of 2×2 pixels are added to the low-resolution occupancy map and used at the decoder to trim the occupancy map near tile boundaries. The order of the outer blocks will also be signaled to the decoder to determine the corresponding sub-block occupancy values. Alternatively, instead of Fig. 12C The sub-block occupancy values ​​may be signaled to the decoder using SEI messages or other mechanisms.

[0148] Fig. 12A An occupancy map 1200a is depicted having a tile 1202 with 4x4 border blocks 1204, 1206, and 1208. Border blocks 1204 and 1208 are partially filled border blocks, and border block 1206 is a fully filled border block. Fig. 12B An occupancy map 1200b is depicted, which can be obtained by downsampling the occupancy map 1200a at an occupancy precision of 2 and then upsampling. However, according to this embodiment, the encoder can create an occupancy map frame that includes an occupancy map 1200b using a high occupancy precision value (such as 4) for internal blocks and a lower occupancy precision value for external blocks. The occupancy map frame can be encoded using a standard-compliant video encoder (such as HEVC / AVC, etc.). Then, the external 4×4 blocks (e.g., blocks 1204, 1206, and 1208) in the occupancy map are identified. One method is that if any adjacent 4×4 blocks (above, below, left, or right) are not occupied, the current 4×4 block is classified as an external block. Then, for each external 4×4 block, the occupancy information of each 2×2 sub-block (e.g., sub-blocks 1204a, 1204b, and 1208a) can be encoded as side information in the tile frame bitstream, or alternatively encoded as an SEI message. The outer 4x4 blocks are scanned in a predetermined order known to the decoder (e.g., raster scan). This process can be continued for another level, where the outer 2x2 blocks are determined, and for each outer 2x2 block, occupancy information is sent for each pixel location. Alternatively, the initial occupancy map video can be created with an occupancy precision value of 2.

[0149] Fig. 12C 1200c is a downsampled occupancy map 1200 at an occupancy precision of 4, where the value of a block is 1 if the corresponding block in the original occupancy map includes valid pixels, or 0 if the corresponding block in the original occupancy map does not include valid pixels. For example, blocks 1204, 1206, and 1208 in occupancy map 1200c include 1 because blocks 1204, 1206, and 1208 in occupancy map 1200a include valid pixels. The downsampled occupancy map 1200c also includes the same 1204, 1206, and 1208 as appended at the end. Fig. 12B Thus, the decoder can use the downsampled occupancy map 1200c to create an upsampled occupancy map 1200b, where the occupancy precision value of the inner block or the fully filled outer block is 4, and the occupancy precision value of the partially filled outer block is 2.

[0150] At the decoder, a standard-compliant low-resolution occupancy map bitstream (e.g., occupancy precision value of 4) is decoded. Based on the decoded 2D occupancy map frame, the external blocks (4×4 in this case) are determined to be in the encoder. Finally, for each external block, the sub-block occupancy map (2×2 in this case) is parsed from the bitstream (tile frame bitstream or SEI message) to determine the occupancy information of the pixels belonging to the external 4×4 blocks. The process can be repeated to identify the external 2×2 blocks and decode the pixel level (1×1 level) occupancy map for each external block. The corresponding flowchart is in Fig.13 A more detailed description is given in .

[0151] Fig.13 1300 is a flowchart for generating an upsampled occupancy map using a sub-block indicator according to various embodiments of the present disclosure. The operations described in flowchart 1300 may be implemented in a decoder, such as Fig. 9 The decoding unit 900 in the embodiment of the present invention. The decoding unit may be included in an electronic device, such as Figure 3 The electronic device 300 in, or included in a server, such as Figure 2 Server 200 in.

[0152] Flowchart 1300 begins at operation 1302, where a low-resolution occupancy map is decoded from an encoded bitstream. In operation 1304, the low-resolution occupancy map is upsampled by a factor of 4. In operation 1306, outer 4x4 boundary blocks are identified, and for each of the outer 4x4 boundary blocks, a sub-block occupancy map is decoded at a 2x2 level.

[0153] In V-PCC, the occupancy map can be encoded using a standards-based video encoder such as HEVC or AVC. Some newer standards, such as Versatile Video Coding (VVC) and Open Media Codec Alliance (AV-1), support frames with different resolutions in the same video sequence. This feature can be utilized in conjunction with the above alternative embodiments.

[0154] For example, the above alternative embodiment can be used to determine an appropriate occupancy precision for each frame. The encoder can then generate an occupancy map frame of the appropriate resolution. For example, if the optimal occupancy precision is determined to be 2, the size of the occupancy map frame will be equal to (asps_frame_width / 2)×(asps_frame_height / 2). To generate an occupancy map frame at a reduced resolution, the occupancy map value for a particular location is set to 1 if the spatially corresponding 2×2 sub-block at the full resolution occupancy frame has at least one non-zero value. The occupancy map video generated in this manner is compressed using a video encoder (such as VVC or AV-1) that allows frames with different resolutions. The V-PCC decoder decodes the occupancy map video and, if necessary, upsamples each occupancy map frame using pixel replication to obtain an occupancy map frame of size asps_frame_width x asps_frame_height. An advantage of this embodiment is that the decision about the appropriate occupancy precision to use can be adapted for each frame.

[0155] Fig. 14C Valid pixels and invalid pixels generated during encoding of an occupancy map according to various embodiments of the present disclosure are shown. Fig.14A A full resolution occupancy map 1400 having a plurality of pixels 1402 spanning 4×4 blocks 1404 , 1406 , and 1408 is depicted. Fig. 14B Depicted is a low-resolution occupancy map 1410 (i.e., a downsampled occupancy map) obtained by downsampling the full-resolution occupancy map 1400 at an occupancy precision of 4. The low-resolution occupancy map 1410 includes values ​​of 1 in blocks 1414, 1416, and 1418 in positions corresponding to blocks 1404, 1406, and 1408, respectively. Values ​​of 0 are filled into the remaining blocks of the low-resolution occupancy map 1410 to indicate the absence of valid pixels.

[0156] Fig. 14C An upsampled occupancy map 1420 is depicted that is generated by upsampling the low resolution occupancy map 1410 rows at an occupancy precision of 4. As a result, blocks 1424, 1426, and 1428 are fully pixelated. A comparison of the upsampled occupancy map 1420 and the original occupancy map 1400 shows Fig.14A Multiple pixels 1410 in are invalid pixels that are considered valid pixels at the decoder.

[0157] exist Fig. 14C In this illustrative embodiment, the occupancy precision selection is based on the ratio of invalid pixels to the number of valid pixels. If the ratio of invalid pixels to the number of valid pixels is less than a threshold, the occupancy precision can be set to 4. Otherwise, a smaller occupancy precision value can be selected. The corresponding flow chart of this embodiment is shown in the following figure.

[0158] Fig.15 1500 is a flowchart for selecting an occupied precision value based on a ratio of invalid pixels to invalid pixels according to various embodiments of the present disclosure. The operations described in flowchart 1500 may be implemented in an encoder, such as Figure 8 The encoder 800 in. The encoder may be included in a server, such as Figure 2 Server 200 in, or electronic device, such as Figure 3 The electronic device 300 in.

[0159] Flowchart 1500 begins with selecting an occupancy precision value at operation 1502. In operation 1504, the original occupancy map is downsampled using the selected occupancy precision. In operation 1506, the downsampled occupancy map is upsampled using the selected occupancy precision, and in operation 1508, the ratio of invalid pixels to valid pixels is calculated.

[0160] In operation 1510, a determination is made as to whether the calculated ratio is less than a predetermined pixel ratio. If the calculated ratio is less than the predetermined pixel ratio, the selected occupancy precision is maintained in operation 1512. Otherwise, if the calculated ratio is greater than the predetermined pixel ratio, flowchart 1500 returns to operation 1502 to select another occupancy precision value.

[0161] In another embodiment, the upsampled occupancy map will be filtered to remove invalid pixels. The filtering will be limited to filtering out only invalid pixels and retaining valid pixels. Since there are no invalid pixels in the upsampled occupancy map for an occupancy precision value of 1, no filtering is performed on the upsampled occupancy map. Fig.16 A flow chart of restricted occupancy map filtering based on occupancy precision value at the decoder is shown. However, in alternative embodiments, the decision may be made at the encoder and signaled to the decoder.

[0162] Fig.16 1600 is a flowchart of restricted occupancy map filtering at a decoder according to various embodiments of the present disclosure. The operations of flowchart 1600 may be implemented in a decoder such as Fig. 9 The decoder 900 in FIG. 1 may be included in an electronic device such as Figure 3 The electronic device 300, or Figure 3 Server 200 in.

[0163] Flowchart 1600 begins at operation 1602, where a downsampled occupancy map is received. At operation 1604, an occupancy precision value is received. At operation 1606, a determination is made as to whether the occupancy precision value is equal to 1. If the occupancy precision value received at operation 1604 is not equal to 1, then at operation 1608, the received occupancy map is upsampled using the received occupancy precision value. Thereafter, at operation 1610, the upsampled occupancy map is subjected to restricted filtering.

[0164] Returning to operation 1606 , if it is determined that the occupancy precision value is equal to 1, then flow diagram 1600 proceeds to operation 1612 , where the received occupancy map is upsampled using the occupancy precision value 1. As previously described, the upsampled occupancy map is not subject to further filtering.

[0165] Fig.17 1700 is a flowchart of a process for encoding a 3D point cloud according to various embodiments of the present disclosure. The operations of flowchart 1700 may be implemented in an encoder, such as Figure 8 In addition, the encoder may be included in a server, such as Figure 2 Server 200 in, or electronic device, such as Figure 3 The electronic device 300 in.

[0166] Flowchart 1700 begins with operation 1702, generating at least a set of geometry frames, the set of geometry frames providing coordinates of points in a 3D point cloud. In operation 1704, a set of occupancy map frames is generated, which indicates whether the points of the 3D point cloud in the set of geometry frames are valid for encoding. In some embodiments, a set of geometry frames and / or a set of atlas frames may also be generated.

[0167] In operation 1706, an occupancy precision value is selected based on a quantization parameter (QP) associated with at least one generated geometry frame in the set of geometry frames. In at least one embodiment, when the QP is less than a first QP threshold, the selected occupancy precision value is a first value; when the QP is between the first QP threshold and the second QP threshold, the selected occupancy precision value is a second value higher than the first value; and when the QP is greater than the second threshold, the selected occupancy precision value is a third value higher than the second value. For example, the first value may be 1, the second value may be 2, and the third value may be 4.

[0168] In operation 1708, at least one occupancy map frame in the set of occupancy map frames is subsampled based on the selected occupancy precision value. In some embodiments, the occupancy map frames may be subsampled using different occupancy precision values. For example, inner blocks of at least one occupancy map frame may be subsampled using a higher occupancy precision value (e.g., 2 or 4), and when outer blocks of the occupancy map frame are partially filled, they may be subsampled using a lower occupancy precision value (e.g., 1 or 2).

[0169] Thereafter, at least the set of geometry frames and the set of occupancy map frames are encoded into a bitstream. As discussed in other illustrative embodiments, the encoded bitstream may also include a set of attribute frames, a set of atlas frames, and auxiliary tile information. At operation 1712, the encoded bitstream is sent.

[0170] In some embodiments, the method comprises the further step of sending one or more sub-block occupancy indicators in at least one occupancy map frame that has been sub-sampled.Alternatively, the one or more sub-block occupancy indicators may be sent in a Supplemental Enhancement Information (SEI) message.

[0171] In other embodiments, different occupancy map frames in the set of occupancy map frames may be sub-sampled with different occupancy precision values. Fig.17 The process may include determining another occupancy accuracy value different from the selected occupancy accuracy value, and subsampling another occupancy map frame in the set of occupancy map frames with the other occupancy accuracy value.

[0172] In some other embodiments, the occupancy precision value may be determined or modified based on a percentage of invalid pixels at a highest occupancy precision value and a ratio of bits of at least one occupancy map frame to bits of at least one geometry frame corresponding to at least one geometry frame.

[0173] Fig.18 1800 is a flowchart of a process for decoding a 3D point cloud according to various embodiments of the present disclosure. The operations of flowchart 1800 may be implemented in a decoder, such as Fig. 9 In addition, the decoder may be included in an electronic device, such as Figure 3 The electronic device 300 in, or included in a server, such as Figure 2 Server 200 in.

[0174] Flowchart 1800 begins at operation 1802 where a bitstream is received. At operation 1804, the bitstream is decoded into a set of geometry frames. At operation 1806, the bitstream is also decoded into a set of occupancy picture frames.

[0175] In operation 1808, an occupancy precision value is obtained based on a quantization parameter (QP) associated with at least one decoded geometry frame in the set of geometry frames. In at least one embodiment, when the QP is less than a first QP threshold, the obtained occupancy precision value is a first value; when the QP is between the first QP threshold and the second QP threshold, the obtained occupancy precision value is a second value higher than the first value; and when the QP is greater than the second threshold, the obtained occupancy precision value is a third value higher than the second value. For example, the first value may be 1, the second value may be 2, and the third value may be 4.

[0176] In operation 1810, at least one occupancy map frame in the set of occupancy map frames is upsampled based on the obtained occupancy precision value. In some embodiments, the occupancy map frames may be upsampled using different occupancy precision values. For example, inner blocks of at least one occupancy map frame may be upsampled with a higher occupancy precision value (e.g., 2 or 4), and outer blocks of the occupancy map frame may be upsampled with a lower occupancy precision value (e.g., 1 or 2) when the outer blocks are partially filled based on one or more sub-block occupancy indicators. In some embodiments, the sub-block occupancy indicator is obtained in at least one occupancy map frame. In other embodiments, the one or more sub-block occupancy indicators may be obtained from a supplemental enhancement information (SEI) message.

[0177] In operation 1812 , a 3D point cloud is generated using the set of geometry frames and the set of occupancy map frames.

[0178] In some embodiments where the set of occupancy map frames includes at least one other occupancy map frame subsampled with a different occupancy precision value, the method further includes the steps of obtaining another occupancy precision value different from the selected occupancy precision value and upsampling the other occupancy map frame with the another occupancy precision value.

[0179] Limited trimming

[0180] The expansion of the full-resolution occupancy map usually occurs at the boundary blocks where the blocks of NxN points are adjacent to the empty blocks. Recently, some techniques have been introduced to filter the upsampled occupancy map at the decoder and trim invalid points from the occupancy map. Although these methods successfully trim many invalid points, they also delete some valid points. This has an adverse effect on the visual quality of the reconstructed point cloud. Some embodiments of the present disclosure are related to limited occupancy map trimming that avoids removing valid points. Instead of using a binary occupancy map (i.e., taking values ​​of 0 or 1), an m-ary occupancy map is used, and the value of the occupancy map is defined according to the occupancy pattern of the boundary blocks in the high-resolution occupancy map. Some embodiments allow for more aggressive occupancy map trimming, which removes a limited number of valid points from the occupancy map while removing more invalid points. The following sections describe various embodiments of limited trimming of occupancy maps.

[0181] In one approach to pruning based on a restricted occupancy map, a decision is made at the encoder whether and how to prune the occupancy map in each block. This decision can be communicated to the decoder in various ways. Some exemplary mechanisms for communicating the decision to the decoder include: sending the decision in supplemental enhancement information (SEI); sending a specific number for each block to the decoder to indicate whether and how to prune the block; sending a binary map or an m-ary map to the decoder to indicate which blocks will be pruned; embedding the decision in various videos by changing the values ​​of video samples (e.g., occupancy map values, etc.) or adding it to the video as a separate tile; and including the decision in a tile data unit. In addition, the number of bits used to indicate the decision regarding occupancy map pruning for each tile can be signaled at the frame level to eliminate the parsing dependency on occupancy map decoding.

[0182] In a first embodiment, for any occupancy map pruning technique (e.g., techniques based on predefined occupancy patterns, techniques based on upsampling / filtering, etc.), it will be determined at the encoder whether any (significant) points of a block in the original high-resolution occupancy map will be removed when pruning the block. If valid points in the block will be removed when the block is pruned, the block will be excluded from pruning, and the decision will be communicated to the decoder. Fig.19 and Fig. 20 Flowcharts of the encoder and decoder of the first embodiment respectively.

[0183] Fig.19 1900 is a flowchart of occupancy map trimming at an encoder according to various embodiments of the present disclosure. The operations described in flowchart 1900 may be implemented in an encoder such as Figure 8 The encoder 800 in. The encoder may be included in a server, such as Figure 2 Server 200 in, or electronic device, such as Figure 3 The electronic device 300 in.

[0184] Flowchart 1900 begins at operation 1902, where a boundary block in an occupancy map is identified. In operation 1904, a determination is made as to whether points in the boundary block in the original occupancy map are pruned. If points in the boundary block of the original occupancy map are not pruned, then pruning is allowed in operation 1906, and the decision is communicated to the decoder. However, if a determination is made in operation 1904 that valid points in the boundary block in the original occupancy map are pruned, then pruning is not allowed in operation 1908, and the decision is communicated to the decoder.

[0185] Fig. 20 2000 is a flowchart of occupancy map trimming at a decoder according to various embodiments of the present disclosure. The operations described in flowchart 2000 may be implemented in a decoder such as Fig. 9The decoding unit 900 in the embodiment of the present invention. The decoding unit may be included in an electronic device, such as Figure 3 The electronic device 300 in, or included in a server, such as Figure 2 Server 200 in.

[0186] Flowchart 2000 begins at operation 2002 by identifying boundary blocks in an upsampled occupancy map, and then a determination is made as to whether trimming the occupancy map is allowed in operation 2004. If trimming the occupancy map is not allowed, then the boundary blocks are not trimmed in operation 2006. However, if trimming the occupancy map is allowed, then the boundary blocks are trimmed in operation 2008.

[0187] In another embodiment of restricted pruning, boundary blocks of NxN points (eg, N=4) are identified where at least one side of the block is adjacent to an empty block or a block belonging to another tile in the upsampled occupancy map.

[0188] For each type of boundary block, such as FIG. 21A to FIG. 21D As shown, the most likely occupancy pattern in the original high-resolution occupancy map is found, and in the original occupancy map, if the unoccupied portion of the boundary block covers the unoccupied portion of the predefined occupancy pattern, the occupancy value of the block in the low-resolution occupancy map is set to a value indicating that the block can be trimmed, for example, occupancy value = 1. If the block contains any valid pixels in the area corresponding to the unoccupied portion of the predefined occupancy pattern, the occupancy value of the block in the low-resolution occupancy map is set to a value indicating that the boundary block cannot be trimmed, for example, occupancy value = 2.

[0189] FIG. 21A to FIG. 21D Boundary block types used in occupancy map trimming based on predefined occupancy patterns according to various embodiments of the present disclosure are shown. Fig.21A A boundary block 2100a is depicted which is adjacent to an empty block or a block belonging to another tile on only one side. Fig.21B A boundary block 2100b is depicted, which has only two parallel sides adjacent to empty blocks or blocks belonging to another tile. Fig. 21C A border block 2102c is depicted, of which only two crossing sides are adjacent to empty blocks or blocks belonging to another tile. Fig.21D A border block 2102d is depicted, which is adjacent on three sides to empty blocks or blocks belonging to another tile.

[0190] Fig. 22 2200 is a flowchart of occupancy map trimming at an encoder based on a predefined occupancy pattern according to various embodiments of the present disclosure. The operations described in flowchart 2200 may be implemented in an encoder, such as Figure 8 The encoder 800 in. The encoder may be included in a server, such as Figure 2 Server 200 in, or electronic device, such as Figure 3 The electronic device 300 in.

[0191] Flowchart 2200 begins at operation 2202, where a boundary block is identified. At operation 2204, FIG. 21A to FIG. 21D The boundary block type is identified in one of the four types shown. In operation 2206, a determination is made as to whether the unoccupied portion of the boundary block is greater than or equal to the unoccupied portion of the predefined occupancy pattern obtained in operation 2208. If the unoccupied portion of the boundary block is not greater than or equal to the unoccupied portion of the predefined occupancy pattern, then in operation 2210, the occupancy value of the boundary block is set to a value that does not allow trimming. If the unoccupied portion of the boundary block is greater than or equal to the unoccupied portion of the predefined occupancy pattern, then in operation 2212, the occupancy value of the boundary block is set to a value that allows trimming.

[0192] Note that the more occupancy patterns are defined for the boundary blocks, the larger the size of the occupancy map file will be, since the values ​​in the occupancy map will be multiple values ​​instead of binary values. Therefore, there is a trade-off between removing extended points from the upsampled occupancy map and the size of the m-ary occupancy map file. Otherwise, the objectness score will be reduced.

[0193] If multiple predefined occupancy patterns are defined for any type of boundary block, the occupancy pattern will be selected based on certain criteria, such as selecting the pattern that removes a higher number of invalid pixels, introduces the least number of artifacts, etc.

[0194] The set of occupancy patterns for each type of boundary block can be defined at different levels, i.e., tile level, frame level, or sequence level. If the occupancy patterns are defined at a lower level (i.e., tile level), more precise shaping of the occupancy map can be performed at the expense of more metadata to be sent to the decoder.

[0195] Fig.23 2300 is a flowchart of occupancy map trimming at a decoder based on predefined occupancy patterns according to various embodiments of the present disclosure. The operations described in flowchart 2300 may be implemented in a decoder, such as Fig. 9 The decoding unit 900 in the embodiment of the present invention. The decoding unit may be included in an electronic device, such as Figure 3 The electronic device 300 in, or included in a server, such as Figure 2 Server 200 in.

[0196] Flowchart 2300 begins with operation 2302, where a boundary block in an upsampled occupancy map is identified. In operation 2304, the boundary block is identified from FIG. 21A to FIG. 21DThe boundary block type is identified in one of the four types shown in FIG. 2306. In operation 2308, a determination is made as to whether the occupancy value of the identified boundary block corresponds to the predefined occupancy pattern value obtained in operation 2308. If the occupancy value of the identified boundary block does not correspond to the value of the predefined occupancy block, then the boundary block is not trimmed in operation 2310. Otherwise, if the occupancy value of the identified boundary block does correspond to the value of the predefined occupancy block, then the boundary block is trimmed in operation 2312.

[0197] In a non-limiting embodiment, an occupancy value of 1 allows trimming, while an occupancy value of 2 does not allow trimming. Thus, at the decoder, if the occupancy value of a boundary block is equal to 1, then the points in the corresponding boundary block of the upsampled occupancy map are removed according to the occupancy pattern. Otherwise, if the occupancy value is equal to 2, then no points of the corresponding boundary block of the upsampled occupancy map are removed.

[0198] Figures 24A to 24B Exemplary predefined occupancy patterns according to various embodiments of the present disclosure are shown. In particular, Fig.24A Depicts the corresponding Fig. 21C A predefined occupancy pattern 2402a for a boundary block of a boundary block is shown. Fig. 24B Depicts the corresponding Fig.21D A predefined occupancy pattern 2402b for a boundary block of a boundary block is shown.

[0199] The ternary occupancy map may be defined in the range of [0, 2]. In this example, the boundary blocks 2402a and 2402b are the most likely occupancy patterns of the original high-resolution occupancy map. For the boundary blocks, if all points in the unoccupied portion of the predefined occupancy pattern are unfilled, the occupancy value of the block is set to 1, otherwise the occupancy value is assigned to 2. At the decoder, if the occupancy value of the boundary block is equal to 1, the upsampled occupancy map is trimmed according to the predefined occupancy pattern, otherwise no points are trimmed from the upsampled occupancy pattern. Fig.25A Shows the use of Fig.24A Examples of full-resolution occupancy patterns of predefined occupancy patterns of boundary blocks and the resulting downsampled and upsampled occupancy maps shown in .

[0200] Fig.25B Shows the use of Fig. 24B Occupancy maps for down-sampling and up-sampling of predefined occupancy patterns in . Encoders, e.g. Figure 8 The encoder 800 in can generate Fig.25A The original full-resolution occupancy map 2500a and Fig.25B Downsampled occupancy map 2500b in. The decoder, e.g. Fig. 9 The decoder 900 in FIG. Figures 24A to 24B The predefined occupancy patterns shown in Fig.25CUpsampled occupancy map 2500c.

[0201] exist Fig.25A In this example of FIG. 1 , the encoder identifies boundary blocks 2502a and 2502b in the occupancy map 2500a and generates a ternary downsampled occupancy map 2500b that includes a value of 0, 1, or 2 in each block. An occupancy value of 0 indicates that there is no pixel in the original occupancy map. An occupancy value of 1 indicates correlation with a predefined occupancy pattern, and a value of 2 indicates no correlation with a predefined occupancy pattern. The decoder can then trim the boundary blocks 2502a and 2502b to match the occupancy values ​​of 0, 1, or 2, respectively. Figures 24A to 24B The predefined occupancy modes 2402a and 2402b in. Fig.25C A trimmed occupancy map 2500c is shown.

[0202] Figures 24A to 24B The predefined boundary blocks shown in are two non-limiting examples of boundary blocks. Figures 26A to 26B Two additional predefined occupancy patterns for boundary blocks according to various embodiments of the present disclosure are shown. In particular, Fig.26A Depicted include corresponding to Fig. 21C Occupancy map 2600a of a predefined occupancy pattern 2602a for a boundary block type 2102c in FIG. Fig.26B Depicted include corresponding to Fig.21D Occupancy map 2600b of a predefined occupancy pattern 2602b of a boundary block type 21D in FIG.

[0203] In the previous limited trimming embodiment, points from the original high-resolution occupancy map are not trimmed. As a result, many extended points in the upsampled occupancy map will remain because many border blocks cannot be trimmed. In another embodiment, a few points in the original high-resolution occupancy map may be trimmed (e.g., 1 or 2 original points in a block of NxN points). For example, in this embodiment, if the unoccupied portion of the border block in the original occupancy map overlaps the corresponding unoccupied portion of one of the predefined occupancy patterns, or differs by a predefined number of points, such as 1 or 2 points, the occupancy value of the block in the downsampled occupancy map, i.e., the low-resolution occupancy pattern, may be set to a value corresponding to the occupancy pattern. Otherwise, the occupancy value may be set to a predefined value indicating that the border block cannot be trimmed.

[0204] This will result in removing many unwanted points in the upsampled occupancy map. The operation of the decoder in this embodiment will be based on Fig.23 The operation of the encoder in this embodiment will be performed according to the following Fig. 27 The operations of the flowchart are discussed in more detail in FIG.

[0205] Fig. 272700 is a flowchart of occupancy map trimming at an encoder according to various embodiments of the present disclosure. The operations of flowchart 2700 may be performed by an encoder, such as Figure 8 Encoder 800 in.

[0206] Flowchart 2700 begins at operation 2702, where a boundary block is identified. In operation 2704, a boundary block type is identified. In operation 2706, a determination is made as to whether the unoccupied portion of the predefined occupancy pattern obtained from operation 2708 matches the corresponding pixel in the boundary block, or differs by less than some occupancy threshold. If the unoccupied portion of the predefined occupancy pattern does not match the corresponding pixel in the boundary block, or differs by less than some occupancy threshold, then flowchart 2700 proceeds to operation 2710, where the occupancy value of the boundary block is set to a predefined value, and trimming is not allowed. Otherwise, flowchart 2700 proceeds to operation 2712, where the occupancy value of the boundary block is set to a predefined value, and trimming is allowed.

[0207] In another embodiment, each block boundary is divided into 4 sub-blocks of 2×2 points. The occupancy of each sub-block can be determined (e.g., 1: occupied, 0: empty). As a result, the occupancy of the boundary block is specified by four bits. These four bits can be converted into a decimal value and used as the occupancy value of the boundary block in the downsampled occupancy pattern. Alternatively, these four bits can be sent to the decoder through other mechanisms including extensions to occupancy map video. As an alternative to this embodiment, the occupancy of the boundary block can be characterized at the pixel level, which will require NxN bits to do so for blocks of NxN pixels.

[0208] In another embodiment, other types of boundary blocks can be identified by considering the occupancy of the upper left, lower left, upper right and lower right neighboring blocks. A total of 8 neighboring blocks (i.e., left, right, upper, lower left, upper right and lower right neighboring blocks) will be used to identify the type of boundary blocks.

[0209] In another embodiment, in order to reduce the size of the m-ary occupancy map video (compared to the binary occupancy map), the occupancy map video will be compressed in a lossy mode. This will cause the values ​​of the occupancy map of some boundary blocks to change, so some unwanted trimming will be performed on some boundary blocks. If the occupancy map is compressed at a high bit rate, the lossy compression may not cause a significant degradation in visual quality while improving the target score.

[0210] In another embodiment, the upsampled occupancy map may be trimmed by upsampling the low-resolution occupancy map by a factor of 4 in both horizontal and vertical directions (for occupancy resolution = 4), applying 2D filtering to the upsampled occupancy map, and trimming any values ​​less than a predefined threshold in the filtered occupancy map.

[0211] For the above embodiment, the m-ary occupancy map can still be used to identify the occupancy pattern of the boundary blocks. Only pixels in the unfilled part of the occupancy pattern are allowed to be trimmed (if the value is less than the threshold), otherwise pixels in the filled part of the occupancy pattern will not be trimmed. The operation of the encoder in this embodiment will be based on Fig. 22 The operation of the decoder in this embodiment will be as follows: Fig.28 The flow chart described in .

[0212] Fig.28 2800 is a flowchart for trimming an occupancy map at a decoder using 2D filtering according to various embodiments of the present disclosure. The operations described in flowchart 2800 may be implemented in a decoder of an electronic device, such as Figure 8 The decoder may be included in a server, such as Figure 2 Server 200 in, or electronic device, such as Figure 3 The electronic device 300 in.

[0213] Flowchart 2800 begins at operation 2802, where a low-resolution occupancy map is upsampled by a factor corresponding to an occupancy precision value selected for the occupancy map at the encoder. In this illustrative embodiment, the low-resolution occupancy map is upsampled by a factor of 4. In operation 2806, 2D filtering is applied to the upsampled occupancy map, and boundary blocks are identified in operation 2804.

[0214] At operation 2808, a determination is made as to whether the occupancy value of the boundary block is equal to the predefined occupancy pattern value obtained from operation 2810. If the occupancy value of the boundary block is not equal to the predefined occupancy pattern value, the flowchart 2800 proceeds to operation 2812, where trimming of the boundary block is not allowed. Otherwise, the flowchart 2800 proceeds to operation 2814, where if the value of the pixel is less than the threshold, the boundary block is trimmed based on the occupancy pattern.

[0215] In another embodiment, according to Fig.28 The operations of flowchart 2800 in the example above are performed to trim the upsampled occupancy map at the decoder. However, some original pixels in the high-resolution occupancy map will be allowed to be trimmed. In the process of identifying the occupancy pattern of the boundary block, if in the original occupancy map, the occupied portion of the boundary block matches one of the previously defined occupancy patterns or differs by a predefined number of points (e.g., 1 or 2 points), the occupancy value of the boundary block in the downsampled occupancy map (i.e., the low-resolution occupancy pattern) can be set to a value corresponding to the occupancy pattern. At the decoder, if the boundary block has been identified as a predefined occupancy pattern, the pixels in the unfilled portion of the occupancy pattern can be trimmed if the pixel value is less than a threshold. The encoder of this embodiment can follow Fig. 27 The operation described in the flowchart 2700 and the decoder of this embodiment may follow Fig.28 The operations are described in flowchart 2800.

[0216] In another embodiment, Fig. 27 As described in the flowchart 2700 in FIG. 2 , the occupancy pattern of the boundary block can be identified in the encoder, but in the decoder, if the value of the pixel corresponding to the unfilled portion of the predefined occupancy pattern is less than a predefined threshold (e.g., threshold_u), the pixel can be trimmed. If the value of the pixel corresponding to the filled portion of the occupancy pattern is less than a predefined threshold (e.g., threshold_f), the pixel can also be trimmed. The value of threshold_f will be much larger than threshold_u to limit the number of trimmed pixels in the filled portion of the occupancy pattern. Fig.29 The operation of the decoder of this embodiment is shown.

[0217] Fig.29 2900 is a flowchart for trimming an occupancy map at a decoder using upper and lower thresholds according to various embodiments of the present disclosure. The operations of flowchart 2900 may be performed by a decoder, such as Figure 8 The decoding unit 800 in.

[0218] Flowchart 2900 begins at operation 2902, where a low-resolution occupancy map is upsampled by a factor corresponding to an occupancy precision value selected for the occupancy map at the encoder. In this illustrative embodiment, the low-resolution occupancy map is upsampled by a factor of 4. Boundary blocks are identified in operation 2904, and 2D filtering is applied to the upsampled occupancy map in operation 2906.

[0219] In operation 2908, a determination is made as to whether the occupancy value of the boundary block is equal to the predefined occupancy pattern value obtained from operation 2910. If the occupancy value of the boundary block is not equal to the predefined occupancy pattern value, the flowchart 2900 proceeds to operation 2912, where trimming of the boundary block is not allowed. Otherwise, the flowchart 2900 proceeds from operation 2908 to operation 2914 to perform the following operations: if the value is less than Threshold_U, trimming the pixels of the boundary block based on the unfilled portion of the occupancy pattern, or if the value is less than Threshold_F, trimming the filled portion of the boundary block based on the filled portion of the occupancy pattern.

[0220] In another embodiment, no occupancy pattern is defined for the boundary blocks. At the encoder, in any boundary block, if only the value of the extended pixels is less than a predefined threshold, the occupancy value of the boundary block in the downsampled occupancy map (i.e., the low-resolution occupancy map) will be set to a specific value (e.g., 1). This occupancy value means that the boundary block can be trimmed at the decoder. Otherwise, if the value of any pixel from the original high-resolution occupancy map is below the threshold, no trimming will be performed on the block, and the occupancy value of the boundary block in the downsampled occupancy map (i.e., the low-resolution occupancy map) will be set to another specific value (e.g., 2). Fig.30 is a flowchart of the encoder of this embodiment, and Fig.31 is a flowchart of the decoder of this embodiment.

[0221] Fig.30 3000 is a flowchart for determining a trim of an occupancy map according to various embodiments of the present disclosure. The operations of flowchart 3000 may be performed by an encoder, such as Figure 2 The encoding unit 700 in the server 200 in.

[0222] Flowchart 3000 begins at operation 3002, where a low-resolution occupancy map is upsampled by a factor corresponding to an occupancy precision value selected for the occupancy map at the encoder. In this illustrative embodiment, the low-resolution occupancy map is upsampled by a factor of 4. In operation 3004, 2D filtering is applied to the upsampled occupancy map. In operation 3006, a determination is made as to whether the value of any valid point in the boundary block is less than a threshold value. If the value of any valid point in the boundary block is not less than the threshold value, then flowchart 3000 proceeds to operation 3008, where the occupancy value of the boundary block is set to a predefined value indicating that trimming is allowed. Otherwise, flowchart 3000 proceeds to operation 3010, where the occupancy value of the boundary block is set to a predefined value that does not allow trimming.

[0223] Fig.31 3100 is a flowchart for trimming an occupancy map at a decoder according to various embodiments of the present disclosure. The operations of flowchart 3100 may be performed by a decoder, such as Figure 3 The decoding unit 800 in the electronic device 300 in.

[0224] Flowchart 3100 begins at operation 3102, where a low-resolution occupancy map is upsampled by a factor corresponding to an occupancy precision value selected for the occupancy map at the encoder. In this illustrative embodiment, the low-resolution occupancy map is upsampled by a factor of 4. In operation 3104, boundary blocks are identified, and in operation 3106, a determination is made as to whether trimming is allowed based on the occupancy value assigned to the boundary block. If trimming is not allowed based on the occupancy value, flowchart 3100 proceeds to operation 3106, where trimming is not allowed for the boundary block. If trimming is allowed based on the occupancy value, flowchart 3100 proceeds to operation 3108, where the boundary block is trimmed to remove pixels having values ​​less than a threshold.

[0225] In another embodiment, the occupancy map is compressed in a lossy manner. Occupancy map trimming will be disabled, as the occupancy value will be compared to a threshold and trimmed. Alternatively, if occupancy map trimming is enabled, the lossy occupancy map threshold is not signaled and is inferred to be 0. Following is the syntax for the above operation:

[0226] if (lossy_occupancy_map_flag) {

[0227] occupation_map_trimming_flag=0

[0228] }

[0229] else{

[0230] occupation_map_trimming_flag=1

[0231] }

[0232] The syntax for the alternative implementation is as follows:

[0233] if(occupancy_map_trimming_flag){

[0234] lossy_occupancy_map_threshold=0

[0235] }

[0236] else{

[0237] lossy_occupancy_map_flag=1

[0238] }

[0239] Figures 32A to 32CAn occupancy map frame with boundary blocks trimmed based on outer corner points according to various embodiments of the present disclosure is shown. The occupancy map 3200 includes a tile 3202, with white areas representing occupied / valid pixels and surrounding black pixels representing unoccupied / invalid pixels. Region 3200a of the occupancy map 3200 is shown in more detail, with outer corner points 3204 identified.

[0240] For the outer corner point 3204, its horizontal end 3206b is found by going to the right or left, and its vertical end 3206a is found according to Fig.32A The position of the outer corner point shown is found by moving toward the top or bottom side. Fig.32C As shown, the horizontal end and the vertical end are linked to construct the oblique line 3208, and the occupancy value of the pixel outside the oblique line, that is, between the oblique line 3208 and the unoccupied pixel will be set to a value indicating that the pixel is unoccupied. In a non-limiting embodiment, this value is zero. In addition, in the case of precision B0=4, the distance from the outer corner point 3204 to the horizontal end 3206n does not exceed B0 pixels.

[0241] For this embodiment, at the encoder, the boundary block (i.e., 4×4 pixels) can be identified as trimmable only when none of the light-colored pixels outside the diagonal line 3008 are occupied in the original high-resolution occupancy map. Otherwise, trimming of the block will not be allowed, and the occupancy value of the boundary block in the downsampled occupancy map (i.e., low-resolution occupancy map) will be set to another specific value (e.g., 2). Fig.33 and 34 These are respectively the operation flow charts of the encoder and decoder of this embodiment.

[0242] Fig.33 3300 is a flowchart for trimming boundary blocks at an encoder based on external corner points according to various embodiments of the present disclosure. The operations of flowchart 3300 may be performed by an encoder, such as Figure 8 The encoder may be included in a server, such as Figure 2 Server 200 in, or electronic device, such as Figure 3 The electronic device 300 in.

[0243] Flowchart 3300 begins at operation 3302, where an outer corner point in an occupancy map is identified. In operation 3304, the horizontal and vertical ends of the outer corner point are linked with a diagonal line. Thereafter, a determination is made in operation 3306 as to whether the pixels outside the diagonal line are occupied in the original occupancy map. If the pixels outside the diagonal line are not occupied in the original occupancy map, then flowchart 3300 proceeds to operation 3308, and the occupancy value of the boundary block is set to a predefined value that allows trimming. If the pixels outside the diagonal line are occupied in the original occupancy map, then flowchart 3300 proceeds to operation 3310, and the occupancy value of the boundary block is set to a predefined value that does not allow trimming.

[0244] Fig.34 3400 is a flowchart for trimming boundary blocks at a decoder based on external corner points according to various embodiments of the present disclosure. The operations of flowchart 3400 may be performed by a decoder, such as Fig. 9 The decoder may be included in an electronic device such as Figure 3 The electronic device 300 in, or included in a server, such as Figure 2 Server 200 in.

[0245] Flowchart 3400 begins at operation 3402, where an outer corner point in an upsampled occupancy map is identified. In operation 3404, the horizontal and vertical ends of the outer corner point are linked with a diagonal line. In operation 3406, a determination is made as to whether trimming the occupancy map is allowed. If trimming the occupancy map is not allowed, flowchart 3400 proceeds to operation 3408, and trimming is not performed on the boundary blocks. However, if trimming the occupancy map is allowed, flowchart 3400 proceeds to operation 3410, and the points outside the diagonal line are trimmed.

[0246] In another embodiment, occupancy map grooming may be performed using a zero-order hold. Fig.35 describes the operation of the encoder, and Fig.36 The operation of the decoder is described.

[0247] Fig.35 3500 is a flowchart of occupancy map trimming at an encoder using zero-order hold according to various embodiments of the present disclosure. The operations of flowchart 3500 may be performed by an encoder such as Figure 8 The encoder may be implemented in a server such as Figure 2 Server 200 in, or electronic device, such as Figure 3 The electronic device 300 in.

[0248] Flowchart 3500 begins at operation 3502 where the occupancy map is downsampled by a factor of N. In operation 3504, the occupancy map is upsampled and then a determination is made in operation 3506 as to whether any valid samples are discarded in the NxN boundary block. If valid samples are discarded, then flowchart 3500 proceeds to operation 3508 where a zero-order hold is used to generate an upsampled boundary block. In operation 3510, the occupancy value of the boundary block is set to a predefined value that does not allow trimming.

[0249] At operation 3506, if a determination is made that no valid samples are discarded in the NxN boundary block, then the flowchart 3500 proceeds to operation 3512 where the upsampled block is retained and the occupancy value of the block is set to a predefined value that allows trimming.

[0250] Fig.36 3600 is a flowchart of occupancy map trimming at a decoder using zero-order hold according to various embodiments of the present disclosure. The operations of flowchart 3600 may be performed by a decoder, such as Figure 3 The decoding unit 800 in the electronic device 300 in.

[0251] Flowchart 3600 begins at operation 3602, where a low-resolution occupancy map is received. At operation 3604, a determination is made as to whether trimming is allowed for a boundary block of the low-resolution occupancy map. If trimming is not allowed, flowchart 3600 proceeds to operation 3606, where an upsampled block is generated using a zero-order hold. If trimming is allowed at operation 3604, flowchart 3600 proceeds to operation 3608, where an upsampled boundary block is generated using a selected upsampling technique.

[0252] In one or more embodiments disclosed herein, occupancy map signaling can be implemented by using a flag. For example, a flag can be used to signal whether the occupancy map can be trimmed on the decoder side. The flag can be set to 1 to indicate that under certain conditions, the occupancy map can be trimmed on the decoder side. The flag can be at different levels (i.e., sequence and frame levels). Such a flag can be named occupancy map trimming enabled flag (occupancyMapTrimmingEnabledFlag).

[0253] Another flag may be used to signal whether the constrained occupancy map trimming described in the above embodiments is combined with any other occupancy map trimming / filtering techniques. For example, a value of 1 for this flag indicates that the occupancy map trimming / filtering is constrained so that no (or few) valid points in the original occupancy map are discarded. It also indicates that the occupancy map value controls whether the trimming / filtering is applied to the corresponding N×N block. In this case, for some blocks, occupancy map trimming / filtering may be applied, while for other blocks, all occupancy map values ​​in the N×N block are set to the corresponding occupancy map values ​​at a lower resolution. This flag may be at both the sequence level and the frame level, and may be named constrainedocupancymaptrimmingenabledflag.

[0254] In some embodiments, if constrainedoccupancyMapTrimmingEnabledFlag is 1, occupancyMapTrimmingEnabledFlag is also 1. Alternatively, the constrainedOccupancyMapTrimmingEnabledFlag flag can be signaled conditionally. That is, it will be signaled only when occupancyMapTrimmingEnabledFlag is 1.

[0255] In an alternative embodiment, a single flag controls limited occupancy map trimming. If such a flag is 1, occupancy map trimming / filtering may be applied to the NxN block, depending on the value of the corresponding occupancy map. If the flag is 0, occupancy map trimming / filtering is not used.

[0256] Although the present disclosure has been described with exemplary embodiments, various changes and modifications may be suggested to one skilled in the art. It is intended that the present disclosure encompass such changes and modifications as fall within the scope of the appended claims.

Claims

1. A coding device for point cloud compression of a three-dimensional 3D point cloud, the coding device comprising: a communication interface configured to send a bit stream; and a processor operably connected to the communication interface, the processor being configured to: generating at least a set of geometric frames for the 3D point cloud, the set of geometric frames providing coordinates of points in the 3D point cloud; generating a set of occupancy map frames indicating whether points of the 3D point cloud are valid for encoding; selecting an occupancy precision value based on a quantization parameter QP associated with at least one generated geometry frame of the set of geometry frames, wherein when the QP is less than a first QP threshold, the selected occupancy precision value is a first value, wherein when the QP is between the first QP threshold and a second QP threshold, the selected occupancy precision value is a second value higher than the first value, and wherein when the QP is greater than the second QP threshold, the selected occupancy precision value is a third value higher than the second value; sub-sampling at least one occupancy map frame in the set of occupancy map frames based on the selected occupancy precision value; encoding the set of geometry frames and the set of occupancy map frames into a bitstream; and The bitstream is transmitted via a communication interface. 2 . The encoding device of claim 1 , wherein the first value is 1, wherein the second value is 2, and wherein the third value is 4.

3. The encoding device according to claim 1, wherein: When the selected occupancy precision value is the second value or the third value, in order to subsample the at least one occupancy map frame, the processor is further configured to: subsampling internal blocks of the at least one occupancy map frame at the selected occupancy precision value; When an outer block of the at least one occupancy map frame is partially filled, subsampling the outer block with a lower occupancy precision value; and One or more sub-block occupancy indicators are sent via the communication interface, wherein the one or more sub-block occupancy indicators are sent in the at least one occupancy map frame or in a supplemental enhancement information (SEI) message.

4. The encoding device according to claim 1, wherein: The processor is further configured to: Generates a collection of atlas frames containing information about each tile.

5. The encoding device according to claim 1, wherein: The set of occupancy map frames includes at least one further occupancy map frame, and wherein the processor is further configured to: determining another occupancy accuracy value different from the selected occupancy accuracy value; and The another occupancy map frame is sub-sampled with the another occupancy precision value.

6. The encoding device according to claim 1, wherein: The selected occupancy precision value is based on: (i) the percentage of invalid pixels at the highest occupancy precision value and (ii) a ratio of bits of the at least one occupancy map frame to bits of at least one generated geometry frame in the set of generated geometry frames, and wherein the at least one occupancy map frame corresponds to the at least one generated geometry frame.

7. A method for point cloud compression of a three-dimensional 3D point cloud, the method comprising: generating at least a set of geometric frames for the 3D point cloud, the set of geometric frames providing coordinates of points in the 3D point cloud; generating a set of occupancy map frames indicating whether points of the 3D point cloud are valid for encoding; selecting an occupancy precision value based on a quantization parameter QP associated with at least one generated geometry frame of the set of geometry frames, wherein when the QP is less than a first QP threshold, the selected occupancy precision value is a first value, wherein when the QP is between the first QP threshold and a second QP threshold, the selected occupancy precision value is a second value higher than the first value, and wherein when the QP is greater than the second QP threshold, the selected occupancy precision value is a third value higher than the second value; sub-sampling at least one occupancy map frame in the set of occupancy map frames based on the selected occupancy precision value; encoding the set of geometry frames and the set of occupancy map frames into a bitstream; and The bit stream is transmitted. The method of claim 7 , wherein the first value is 1, wherein the second value is 2, and wherein the third value is 4.

9. The method according to claim 7, wherein: When the selected occupancy precision value is the second value or the third value, subsampling the at least one occupancy map frame further comprises: subsampling internal blocks of the at least one occupancy map frame at the selected occupancy precision value; When an outer block of the at least one occupancy map frame is partially filled, subsampling the outer block at a lower occupancy precision value; and One or more sub-block occupancy indicators are sent, wherein the one or more sub-block occupancy indicators are sent in the at least one occupancy map frame or in a supplemental enhancement information (SEI) message.

10. The method according to claim 7, further comprising: Generates a collection of atlas frames containing information about each tile.

11. The method according to claim 7, wherein: The set of occupancy map frames includes at least one other occupancy map frame, the method further comprising: determining another occupancy accuracy value different from the selected occupancy accuracy value; and The another occupancy map frame is sub-sampled with the another occupancy precision value.

12. The method according to claim 7, wherein: The selected occupancy precision value is based on: (i) the percentage of invalid pixels at the highest occupancy precision value and (ii) a ratio of bits of the at least one occupancy map frame to bits of at least one generated geometry frame in the set of generated geometry frames, and wherein the at least one occupancy map frame corresponds to the at least one generated geometry frame.

13. A decoding device for point cloud compression of a 3D point cloud, the decoding device comprising: a communication interface configured to receive a bit stream; and a processor operably connected to the communication interface, the processor being configured to: decoding the bitstream into at least a set of geometry frames, the set of geometry frames providing coordinates of points in the 3D point cloud; decoding the bitstream into a set of occupancy map frames indicating whether points of the 3D point cloud are valid for encoding; obtaining an occupancy precision value based on a quantization parameter QP associated with at least one decoded geometry frame in the set of geometry frames, wherein when the QP is less than a first QP threshold, the obtained occupancy precision value is a first value, wherein when the QP is between the first QP threshold and a second QP threshold, the obtained occupancy precision value is a second value higher than the first value, and wherein when the QP is greater than the second QP threshold, the obtained occupancy precision value is a third value higher than the second value; upsampling at least one occupancy map frame in the set of occupancy map frames based on the obtained occupancy accuracy value; and The 3D point cloud is generated using the set of geometry frames and the set of occupancy map frames. 14 . The decoding device of claim 13 , wherein the first value is 1, wherein the second value is 2, and wherein the third value is 4.

15. The decoding device according to claim 13, wherein: When the obtained occupancy accuracy value is the second value or the third value, in order to upsample the at least one occupancy map frame, the processor is further configured to: upsampling an internal block of the at least one occupancy map frame with the obtained occupancy accuracy; upsampling outer blocks of the at least one occupancy map frame at a lower occupancy precision value based on the one or more sub-block indicators; and wherein the one or more sub-block indicators are received in the at least one occupancy map frame or in a supplemental enhancement information (SEI) message, and wherein the set of occupancy map frames comprises at least one further occupancy map frame, and wherein the processor is further configured to: determining another occupancy accuracy value different from the obtained occupancy accuracy value; and upsampling the another occupancy map frame with the another occupancy precision value, and The determined occupancy precision value is based on: (i) the percentage of invalid pixels at the highest occupancy precision value and (ii) a ratio of bits of the at least one occupancy map frame to bits of at least one decoded geometry frame in a set of decoded geometry frames, and wherein the at least one occupancy map frame corresponds to the at least one decoded geometry frame.

Images