Method and device for processing duplicate points in point cloud compression

Abstract

Decoder, encoder and method for decoding 3D point cloud data. The decoder includes a communication interface and a processor, the communication interface is configured to receive an encoded bitstream including 3D point cloud data. The processor is configured to decode the bitstream to identify a plurality of two-dimensional (2D) frames; based on the positions of the corresponding pixels in the two or more geometric frames, identify corresponding pixels in two or more geometric frames of the plurality of 2D frames; identify depth values ​​of the corresponding pixels in the two or more geometric frames; based on the depth values ​​of the corresponding pixels in the identified two or more geometric frames, identify duplicate points in at least one of the two or more geometric frames; and remove or ignore the identified duplicate points when reconstructing the 3D point cloud data.

Description

Technical Field

[0001] The present disclosure generally relates to point cloud compression of three-dimensional (3D) objects. More specifically, the present disclosure relates to a method and apparatus for handling duplicate points in point cloud compression. Background Art

[0002] Due to the availability of powerful handheld devices such as smartphones, three hundred and sixty degree (360°) video is becoming a new way to experience immersive video. 360° video provides consumers with a “real life”, “be there” immersive experience by capturing a 360° panoramic view of the world. Users can interactively change their viewpoint and dynamically view any part of the captured scene or object they want. Display and navigation sensors track head movement in real time to determine the area of ​​the 360° video that the user wants to view. 360° video provides an immersive experience with three degrees of freedom (3DoF). Six degrees of freedom (6DoF) is the next level of immersive experience, where users can turn their heads and walk around in a virtual / augmented environment. 3D multimedia data in nature, such as point clouds, are required to provide a 6DoF experience.

[0003] Point clouds and meshes are a set of 3D points that represent a model of the surface of a 3D object. Point clouds are common in various applications, such as gaming, 3D mapping, visualization, medical applications, augmented reality, virtual reality, autonomous driving, multi-view replay, 6DoF immersive media, etc. are some of them. However, point clouds consist of a large amount of data that needs to be compressed. Summary of the invention

[0004] Technical issues

[0005] The present disclosure provides processing of duplicate points in point cloud compression.

[0006] Solving the problem

[0007] In one embodiment, a decoder for decoding 3D point cloud data is provided. The decoder includes a communication interface and a processor, the communication interface being configured to receive an encoded bitstream including 3D point cloud data. The processor is configured to decode the bitstream to identify a plurality of 2D frames; based on the positions of the corresponding pixels in two or more geometric frames in the plurality of 2D frames, identify corresponding pixels in two or more geometric frames in the plurality of 2D frames; identify depth values ​​of the corresponding pixels in the two or more geometric frames; based on the depth values ​​of the corresponding pixels in the identified two or more geometric frames, identify duplicate points in at least one of the two or more geometric frames; and remove or ignore the identified duplicate points when reconstructing the 3D point cloud data.

[0008] In another embodiment, an encoder for encoding 3D point cloud data is provided. The encoder includes a processor and a communication interface. The processor is configured to encode a first two-dimensional (2D) frame set of 3D point cloud data into a coded bitstream; and perform prediction on the first 2D frame set to encode a second 2D frame set of 3D point cloud data into a coded bitstream. Wherein, in order to perform the prediction, the processor is also configured to: decode the bitstream to generate a plurality of 2D frames; based on the positions of the corresponding pixels in two or more geometric frames in the plurality of 2D frames, identify the corresponding pixels in two or more geometric frames in the plurality of 2D frames; identify the depth values ​​of the corresponding pixels in the two or more geometric frames; based on the depth values ​​of the corresponding pixels in the identified two or more geometric frames, identify duplicate points in at least one of the two or more geometric frames; and remove or ignore the identified duplicate points when reconstructing the 3D point cloud data. The communication interface is configured to send a coded bitstream including 3D point cloud data.

[0009] Other technical features will be apparent to those skilled in the art from the following drawings, description and claims.

[0010] Advantageous Effects of the Invention

[0011] The present disclosure provides processing of duplicate points in point cloud compression. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] For a more complete understanding of the present disclosure and its advantages, reference is now made to the following description taken in conjunction with the accompanying drawings, wherein like reference numerals represent like parts:

[0013] Figure 1 An example computing system according to the present disclosure is shown;

[0014] Figure 2 and Figure 3 An example device in a computing system according to an embodiment of the present disclosure is shown;

[0015] Figure 4 shows examples of duplicate points in point cloud data according to various embodiments of the present disclosure;

[0016] Figure 5 An example block diagram of an encoder according to various embodiments of the present disclosure is shown;

[0017] Figure 6 shows an example block diagram of a decoder according to various embodiments of the present disclosure;

[0018] Figure 7 A flowchart of a geometry-based duplicate point removal process according to various embodiments of the present disclosure is shown;

[0019] Figure 8An example block diagram of an encoder for a multi-attribute point cloud according to various embodiments of the present disclosure is shown;

[0020] Fig. 9 A flowchart showing a process of removing duplicate points based on geometry and attributes according to various embodiments of the present disclosure; and

[0021] Fig.10 A flowchart of a process for decoding 3D point cloud data according to various embodiments of the present disclosure is shown.

[0022] Specific implementation method

[0023] Before proceeding to the following detailed description, it may be beneficial to set forth the definitions of certain words and phrases used throughout the patent document. The term "coupled" and its derivatives refer to any direct or indirect communication between two or more elements, whether or not the elements are in physical contact with each other. The terms "send," "receive," and "communicate" and their derivatives cover direct and indirect communication. The terms "include" and "comprises" and their derivatives mean including but not limited to. The term "or" is inclusive, meaning and / or. The phrase "associated with..." and its derivatives mean including, included therein, interconnected therewith, containing, contained therein, connected therewith, coupled therewith, communicated therewith, cooperated therewith, interlaced, juxtaposed, approximated therewith, bound therewith, or having, having attributes or having some relationship, having some relationship therewith, etc. The term "controller" refers to any device, system, or part thereof that controls at least one operation. Such a controller may be implemented in hardware or a combination of hardware and software and / or firmware. The functions associated with any particular controller may be centralized or distributed, whether local or remote. The phrase "at least one" when used with a list of items means that different combinations of one or more of the listed items may be used, and only one item in the list may be required. For example, “at least one of A, B, and C” includes any of the following combinations: A; B; C; A and B; A and C; B and C; and A, B, and C.

[0024] In addition, the various functions described below can be implemented or supported by one or more computer programs, each of which is formed by a computer-readable program code and embodied in a computer-readable medium. The terms "application" and "program" refer to one or more computer programs, software components, instruction sets, procedures, functions, objects, classes, instances, related data or a portion thereof, suitable for implementation in a suitable computer-readable program code. The phrase "computer-readable program code" includes any type of computer code, including source code, object code and executable code. The phrase "computer-readable medium" includes any type of medium that can be accessed by a computer, such as a read-only memory (ROM), a random access memory (RAM), a hard drive, a compact disk (CD), a digital video disk (DVD) or any other type of memory. "Non-transitory" computer-readable media do not include wired, wireless, optical or other communication links that transmit transient electrical or other signals. Non-transitory computer-readable media include media that can permanently store data and media that can store data and then overwrite it, such as rewritable optical disks or erasable storage devices.

[0025] Definitions for certain other words and phrases are provided throughout this patent document. Those of ordinary skill in the art should understand that in many, if not most, instances, such definitions apply to prior, as well as future uses of such definitions.

[0026] The following discussion Figures 1 to 10 The various embodiments used to describe the principles of the present disclosure in this patent document are exemplary only and should not be interpreted in any way as limiting the scope of the present disclosure. Those skilled in the art will understand that the principles of the present disclosure can be implemented in any appropriately arranged system or device.

[0027] Embodiments of the present disclosure recognize that decoding and rendering point cloud data, such as 3D objects, can be expensive in terms of resources consumed. For example, processing and storage resources may be intensively used when reconstructing a point cloud. Embodiments of the present disclosure provide improvements to point cloud compression, such as video-based point cloud compression (V-PCC) performed by the Moving Picture Experts Group (MPEG), by reducing the number of redundant and repeated vertices generated, which in turn can also reduce codec complexity.

[0028] Embodiments of the present disclosure provide solutions to reduce the complexity of point clouds. Embodiments of the present disclosure recognize that, although certain point cloud encoding techniques may perform well in both objective and visual tests, they may produce significantly more points in the decoded point cloud than the point cloud encoding techniques in the original (source) point cloud. In addition, embodiments of the present disclosure recognize that some encoding techniques may generate a large percentage of duplicate points, which increases codec complexity.

[0029] Therefore, embodiments of the present disclosure identify two sources of point duplication and provide solutions to reduce the number of redundant and duplicate points. In addition, various embodiments provide such reduction without degrading the visual quality of the reconstructed point cloud.

[0030] Figure 1 An example computing system 100 is shown in accordance with the present disclosure. Figure 1 The embodiment of the system 100 shown is for illustration only. Other embodiments of the system 100 may be used without departing from the scope of the present disclosure.

[0031] System 100 includes a network 102 that facilitates communications between various components in system 100. For example, network 102 can transmit Internet Protocol (IP) packets, frame relay frames, or other information between network addresses. Network 102 includes all or part of one or more local area networks (LANs), metropolitan area networks (MANs), wide area networks (WANs), a global network such as the Internet, or any other communication system at one or more locations.

[0032] The network 102 facilitates communication between the server 104 and various client devices 106-116. The client devices 106-116 may be, for example, smart phones, tablet computers, laptop computers, personal computers, wearable devices, or head-mounted displays (HMDs). The server 104 may represent one or more servers. Each server 104 includes any suitable computing or processing device that can provide computing services to one or more client devices. Each server 104 may, for example, include one or more processing devices, one or more memories for storing instructions and data, and one or more network interfaces for facilitating communication over the network 102. As described in more detail below, the server 104 may encode or decode the point cloud to handle duplicate points. In various embodiments, each server 104 may include an encoder for handling duplicate points in point cloud compression.

[0033] Each client device 106-116 represents any suitable computing or processing device that interacts with at least one server or one or more other computing devices via the network 102. In this example, the client devices 106-116 include a desktop computer 106, a mobile phone or mobile device 108 (e.g., a smartphone), a personal digital assistant (PDA) 110, a laptop computer 112, a tablet computer 114, and an HMD 116. However, any other or additional client devices may be used in the system 100. As described in more detail below, each client device 106-116 may encode or decode a point cloud to handle duplicate points. In various embodiments, each client device 106-116 may include a decoder for handling duplicate points in point cloud compression.

[0034] In this example, some client devices 108-116 communicate indirectly with the network 102. For example, client devices 108 and 110 (mobile device 108 and PDA 110, respectively) communicate via one or more base stations 118, such as cellular base stations or eNodeBs (eNBs). Mobile device 108 includes a smartphone. Moreover, client devices 112, 114, and 116 (laptops, tablet computers, and HMDs, respectively) communicate via one or more wireless access points 120, such as IEEE 802.11 wireless access points. As described in more detail below, HMD 116 can display a 360° scene including one or more point clouds. Note that these are for illustration only, and each client device 106-116 can communicate directly with the network 102, or communicate indirectly with the network 102 via any suitable intermediate device or network. In various embodiments, server 104 or any client device 106-114 can be used to compress the point cloud and send the data to another client device, such as any client device 106-116.

[0035] In various embodiments, the mobile device 108 (or any other client device 106-116) can securely and efficiently send information to another device (e.g., the server 104). The mobile device 108 (or any other client device 106-116) can be used as a VR display when connected to the headset via the cradle, and is similar to the functionality of the HMD 116. The mobile device 108 (or any other client device 106-116) can trigger the transmission of information between itself and the server 104.

[0036] although Figure 1 One example of system 100 is shown, but may be used for Figure 1 Various changes may be made. For example, system 100 may include any number of each component in any suitable arrangement. In general, computing and communication systems have a wide variety of configurations, and Figure 1 The scope of the present disclosure is not limited to any particular configuration. Figure 1 An operating environment is shown in which the various features disclosed in this patent document may be used, as may be used in any other suitable system.

[0037] The processes and systems provided in the present disclosure allow a client device 106-116 or a server 104 to encode, compress, send, receive, render a point cloud, or a combination thereof. For example, the server 104 can then compress, encode, and send the point cloud data to the client devices 106-116. For another example, any client device 106-116 can compress the point cloud data and send it to any client device 106-116 or the server 104.

[0038] Figure 2 and Figure 3 An example device in a computing system according to an embodiment of the present disclosure is shown. In particular, Figure 2 An example server 200 is shown, Figure 3 An example electronic device 300 is shown. The server 200 may represent Figure 1 The server 104, the electronic device 300 may represent Figure 1 One or more of the client devices 106-116.

[0039] Server 200 may represent one or more local servers, one or more compression servers, or one or more encoding servers. Figure 2 As shown, server 200 includes a bus system 205 that supports communication between at least one processor 210, at least one storage device 215, at least one communication interface 220, and at least one input / output (I / O) unit 225. In various embodiments, server 200 is an encoder or a decoder.

[0040] Processor 210 executes instructions that may be stored in memory 230. The instructions stored in memory 230 may include instructions for decomposing a point cloud, compressing a point cloud. The instructions stored in memory 230 may also include instructions for encoding the point cloud to generate a bitstream. The instructions stored in memory 230 may also include instructions for handling duplicate points in point cloud compression and / or for viewing through a VR headset (e.g., Figure 1 The processor 210 may include any suitable number and type of processors or other devices in any suitable arrangement. Example types of processors 210 include microprocessors, microcontrollers, digital signal processors, field programmable gate arrays, application specific integrated circuits, and discrete circuits.

[0041] Memory 230 and persistent storage 235 are examples of storage devices 215, which represent any structure capable of storing and facilitating retrieval of information (e.g., data, program code, or other suitable information, either temporarily or permanently). Memory 230 may represent random access memory or any other suitable volatile or non-volatile storage device. Permanent storage 235 may include one or more components or devices that support long-term storage of data, such as read-only memory, a hard drive, flash memory, or an optical disk.

[0042] Communication interface 220 supports communication with other systems or devices. For example, communication interface 220 may include a network interface card or a wireless transceiver to facilitate communication between Figure 1Communication interface 220 may support communication over network 102. Communication interface 220 may support communication over any suitable physical or wireless communication link.

[0043] I / O unit 225 allows input and output of data. For example, I / O unit 225 may provide a connection for user input through a keyboard, mouse, keypad, touch screen, motion sensor, or any other suitable input device. I / O unit 225 may also send output to a display, printer, or any other suitable output device.

[0044] In various embodiments, the server 200 implements handling of duplicate points in point cloud compression, which will be discussed in more detail below. Figure 2 Describe as representative Figure 1 104, but the same or similar structure can be used in one or more of the various client devices 106-116. For example, a desktop computer 106 or a laptop computer 112 can have a Figure 2 The same or similar structure as shown.

[0045] Figure 3 An electronic device 300 according to an embodiment of the present disclosure is shown. Figure 3 The embodiment of the electronic device 300 shown in FIG. 1 is for illustration only, and other embodiments may be used without departing from the scope of the present disclosure. The electronic device 300 may have a variety of configurations, and Figure 3 The scope of the present disclosure is not limited to any particular implementation of an electronic device. In various embodiments, Figure 1 One or more of the client devices 106-116 may be configured to include the same or similar configuration as the electronic device 300. In various embodiments, the electronic device 300 is an encoder, a decoder, or both.

[0046] In various embodiments, the electronic device 300 may be used for data transmission, image or video compression, image or video decompression, encoding, decoding, and media rendering applications. The electronic device 300 may be a mobile communication device, such as a wireless terminal, a desktop computer (similar to a Figure 1 desktop computer 106), mobile devices (similar to Figure 1 Mobile devices 108), PDAs (similar to Figure 1 PDA 110), laptop computer (similar to Figure 1 portable computer 112), tablet computer (similar to Figure 1 Tablet computer 114), head mounted display (similar to Figure 1 HMD 116) etc.

[0047] like Figure 3As shown, the electronic device 300 includes an antenna 305, a radio frequency (RF) transceiver 310, a transmit (TX) processing circuit 315, a microphone 320, and a receive (RX) processing circuit 325. The electronic device 300 also includes a speaker 330, one or more processors 340, an input / output (I / O) interface (IF) 345, an input 350, a display 355, and a memory 360. The memory 360 includes an operating system (OS) 361, one or more applications 362, and a point cloud 363.

[0048] The RF transceiver 310 receives an incoming RF signal from the antenna 305 that is transmitted by another component on the system. For example, the RF transceiver 310 receives an RF signal transmitted via a Bluetooth or WI-FI signal from an access point (e.g., a base station, a WI-FI router, a Bluetooth device) of the network 102 (e.g., WI-FI, Bluetooth, cellular, 5G, LTE, LTE-A, WiMAX, or any other type of wireless network). The RF transceiver 310 can down-convert the incoming RF signal to generate an intermediate frequency or baseband signal. The intermediate frequency or baseband signal is sent to the RX processing circuit 325, which generates a processed baseband signal by filtering, decoding, or digitizing the baseband or intermediate frequency signal or a combination thereof. The RX processing circuit 325 sends the processed baseband signal to the speaker 330 (e.g., for voice data) or the processor 340 for further processing (e.g., for web browsing data).

[0049] The TX processing circuit 315 receives analog or digital voice data from the microphone 320, or receives other output baseband data from the processor 340. The output baseband data may include network data, email, or interactive video game data. The TX processing circuit 315 encodes, multiplexes, digitizes, or a combination thereof, the output baseband data to generate a processed baseband or intermediate frequency signal. The RF transceiver 310 receives the outgoing processed baseband or intermediate frequency signal from the TX processing circuit 315 and up-converts the baseband or intermediate frequency signal to an RF signal that is transmitted via the antenna 305.

[0050] The processor 340 may include one or more processors or other processing devices, and executes the OS 361 stored in the memory 360 to control the overall operation of the electronic device 300. For example, the processor 340 may control the reception of the forward channel signal, the transmission of the reverse channel signal by the RF transceiver 310, the RX processing circuit 325, and the TX processing circuit 315 according to known principles. The processor 340 is also capable of executing other applications 362 residing in the memory 360, such as for processing duplicate points in point cloud compression.

[0051] Processor 340 may execute instructions stored in memory 360. Processor 340 may include any suitable number and type of processors or other devices in any suitable arrangement. For example, in some embodiments, processor 340 includes at least one microprocessor or microcontroller. Example types of processor 340 include microprocessors, microcontrollers, digital signal processors, field programmable gate arrays, application specific integrated circuits, and discrete circuits.

[0052] Processor 340 is also capable of executing other processes and programs resident in memory 360, such as receiving, storing, and executing instructions by providing image capture and processing. Processor 340 can move data into or out of memory 360 as required by the executed process. In some embodiments, processor 340 is configured to receive, store, and execute instructions based on OS 361 or in response to a request from an eNB (similar to Figure 1 The processor 340 is also coupled to an I / OIF 345 that enables the electronic device 300 to connect to other devices, such as client devices 106-116. The I / OIF 345 is a communication path between these accessories and the processor 340.

[0053] The processor 340 is also coupled to the input 350. The operator of the electronic device 300 can use the input 350 to input data or input into the electronic device 300. The input 350 can be a keyboard, a touch screen, a mouse, a trackball, a voice input, or any other device that can act as a user interface to allow the user to interact with the electronic device 300. For example, the input 350 may include a voice recognition process, thereby allowing the user to enter a voice command through the microphone 320. In another example, the input 350 may include a touch panel, a (digital) pen sensor, a key, or an ultrasonic input device. The touch panel can recognize touch input, for example, with at least one of a capacitive scheme, a pressure-sensitive scheme, an infrared scheme, or an ultrasonic scheme. For example, in a capacitive scheme, the input 350 can recognize touch or proximity. The input 350 may also include a control circuit. By providing additional inputs to the processor 340, the input 350 can be associated with one or more sensors 365 and / or cameras. As discussed in more detail below, sensors 365 include inertial sensors (e.g., accelerometers, gyroscopes, and magnetometers), optical sensors, motion sensors, cameras, pressure sensors, heart rate sensors, altimeters, etc. For example, input 350 may utilize motion detected by a motion sensor associated with sensor 365 as input when rendering 3D video.

[0054] Processor 340 is also coupled to display 355. Display 355 may be a liquid crystal display (LCD), a light emitting diode (LED) display, an organic LED (OLED), an active matrix OLED (AMOLED), or other display capable of rendering text and / or graphics (e.g., from a website, video, game, image, etc.). Display 355 may be sized to fit the HMD. Display 355 may be a single display screen or multiple display screens capable of creating a stereoscopic display. In various embodiments, display 355 is a heads-up display (HUD).

[0055] Memory 360 is coupled to processor 340. A portion of memory 360 may include random access memory (RAM), and another portion of memory 360 may include flash memory or other read-only memory (ROM).

[0056] The memory 360 may include persistent storage representing any structure capable of storing and facilitating retrieval of information (e.g., data, program code, and / or other suitable information, whether temporary or permanent). The memory 360 may include one or more components or devices that support long-term storage of data, such as a read-only memory, a hard drive, a flash memory, or an optical disk. The point cloud data 363 may include various 3D point clouds, 3D meshes, 3D objects, and the like. In various embodiments, the electronic device 300 handles duplicate points in the compression or decompression of the point cloud data 363. The electronic device 300 may reconstruct the point cloud data 363 to display one or more 3D objects or 3D scenes on the display 355.

[0057] The electronic device 300 may further include one or more sensors 365 that measure physical quantities or detect the activation state of the electronic device 300 and convert the measured or detected information into electrical signals. For example, the sensor 365 may include one or more buttons for touch input (located on the headset or the electronic device 300), one or more cameras, gesture sensors, eye tracking sensors, gyroscopes or gyroscope sensors, air pressure sensors, magnetic sensors or magnetometers, acceleration sensors or accelerometers, grip sensors, proximity sensors, color sensors (e.g., red, green, and blue (RGB) sensors), biophysical sensors, temperature / humidity sensors, lighting sensors, ultraviolet (UV) sensors, electromyography (EMG) sensors, electroencephalography (EEG) sensors, electrocardiography (ECG) sensors, infrared (IR) sensors, ultrasound sensors, iris sensors, fingerprint sensors, etc. The sensor 365 may further include a control circuit for controlling at least one of the sensors included therein. As will be discussed in more detail below, one or more of these sensors 365 may be used to control a user interface (UI), detect UI input, determine the user direction of the three-dimensional content display recognition, and the direction toward the user. Any of these sensors 365 may be located within the electronic device 300, within an auxiliary device operably connected to the electronic device 300, within a headset configured to hold the electronic device 300, or within a single device in which the electronic device 300 includes the headset.

[0058] As will be discussed in more detail below, in this illustrative embodiment, the electronic device 300 and / or server 200 receives an encoded bitstream including 3D point cloud data, decodes the bitstream to identify 2D frames, identifies duplicate points in the frames based on depth values, and removes or ignores the duplicate points when reconstructing the 3D point cloud data.

[0059] although Figure 2 and Figure 3 An example of a device in a computing system is shown, but the Figure 2 and Figure 3 Make various changes. For example, Figure 2 and Figure 3 The various components in the embodiment may be combined, further subdivided, or omitted, and other components may be added as required. As a specific example, processor 340 may be divided into multiple processors, such as one or more central processing units (CPUs) and one or more graphics processing units (GPUs). In addition, as with computing and communication networks, electronic devices and servers may have a variety of configurations, and Figure 2 and Figure 3 The present disclosure is not limited to any particular electronic device or server.

[0060] As discussed in this article, a point cloud is a set of (x, y, z) coordinates (also called points or vertices) that represent an object in 3D space (e.g., the outer surface of an object). Point clouds are obtained from 3D scans using LiDAR, RGB+depth cameras, or from 3D reconstruction from multiple images using multi-view stereo. Each point in a point cloud can have multiple attributes associated with that point, such as color, material properties, normals, intensity (a measure of the return intensity of the laser pulse that generated the point during the scan), and more. Since point clouds, especially those used to represent 3D maps, can contain millions of points, the raw data size is often very large and requires compression.

[0061] For example, the MPEG Test Model for Category 2 (TMC2) for compressing dynamic point clouds works by converting each 3D point cloud frame into multiple 2D image / video frames. A dynamic point cloud is a set of point cloud frames used to represent a motion / moving 3D object over time. For example, a dynamic point cloud is a static point cloud when a video is an image. For TMC2, one set of image / video frames is used to capture geometry and another set is used for texture. In the conversion process from 3D to 2D, orthogonal projection is adopted in V-PCC, so two of the three geometric coordinates are implicitly encoded in the 2D position of the projected pixel in the image / video frame, and the third coordinate value (or some value derived from the third coordinate) is stored as the value of the pixel representing the depth of the 3D coordinate from the plane (i.e., the depth value). Specifically, which of the x, y, or z coordinates is stored as the value depends on the direction of the projection (or the direction of the projection plane). The image / video frame is then compressed using an existing image / video codec (e.g., High Efficiency Video Coding (HEVC)), allowing TMC2 to use the existing codec for most of the compression process. At the decoder, a set of image / video frames for geometry and textures are decompressed from the compressed bitstream. The point cloud is then reconstructed by reprojecting the points from the 2D frames into 3D space. TMC2 is just an example, other embodiments and coordinate systems are possible.

[0062] However, such encoding techniques may generate many more points in the decoded point cloud than the original (source) point cloud. In addition, a large number of vertex duplications may occur in the reconstructed point cloud. Typically, a point cloud contains a single attribute of each attribute type for each point. For example, if the only attribute type of a given point cloud is color, then for each geometric point (x, y, z), there is only one associated color, such as a 3-tuple (r, g, b) color. In addition, in some embodiments, each point can only be rendered with one color or attribute. In other embodiments, each point can have multiple attributes. In one or more of these embodiments, duplicate points in the point cloud are points with the same (or similar) geometric values ​​(x, y, z). In other embodiments of these embodiments, duplicate points in the point cloud are points with the same (or similar) geometric values ​​(x, y, z) and some or all attributes (or vertices) are the same (or similar). Here, the same or similar means that the values ​​are exactly the same or the values ​​are within a set threshold, depending on the context. For example, if a floating point representation is used, an "exact" comparison may not make sense, and a threshold similarity is used.

[0063] Embodiments of the present disclosure recognize that there is little benefit in generating duplicate vertices in a point cloud, as the duplication cannot be used to enhance the visual experience (e.g., such as providing field-dependent color or reflectivity modulation like light fields). Instead, these duplicate points result in a waste of computational resources, especially for lossy PCC schemes, as this increases the data size and introduces unnecessary overhead for transmission. Furthermore, a significant increase in the number of vertices increases the complexity of the processing chain proportionally.

[0064] Figure 4 Examples of duplicate points in point cloud data according to various embodiments of the present disclosure are shown. Figure 4 The examples of repeated points in the point cloud data shown in are for illustration only. Other embodiments may be used without departing from the scope of the present disclosure.

[0065] A common situation encountered during projection from a point cloud to multiple planes (depth layers) is that multiple points may be projected onto the same pixel location in the 2D plane. An example where this may occur is using a point cloud to represent clothing that may have multiple folds / layers. To assimilate these points into patches, two depth vectors can be used for each patch: patch.depth[0] and patch.depth[1]. These vectors are used to store the depth values ​​of the points staggered along the projection plane (e.g., along the depth dimension). The other two dimensions of each point are encoded by the projected position on the 2D plane.

[0066] exist Figure 4In , point 1 is projected to pixel position p[i] on the 2D projection plane. The depth relative to the minimum depth d(1) of the patch is stored in the vector patch.depth[0] at index position [i]. After projection, points 2, 3, 4, and 5 share the same position on the projection plane. Similarly, points 6 and 7 share the same position on the projection plane. In this example, the relative depths of points 2 and 4 (d(2) and d(4) respectively) will be stored in patch.depth[0] and patch.depth[1] at index position p[i+1] in the patch. Points 3 and 5 can be reconsidered for projection in subsequent iterations. Between points 6 and 7, since the depth of point 7 from point 6 is greater than the parameter "surfaceThickness", only the relative depth of point 6, d(6), is stored in patch.depth[0] at index position [i+2]. In this example, for the vector patch.depth[1], only one of the three index positions stores the depth value of a unique point (i.e., starting from point 4). The values ​​in the remaining positions in the vector patch.depth[1] are copied from the corresponding positions in patch.depth[0]. Later in the encoding pipeline, when a pair of 2D frames are generated to capture geometry, the first geometry frame is created using the values ​​in patch.depth[1], and in one embodiment, the difference (or delta value) (e.g., patch.depth[1] – patch.depth[0]) is stored in the second geometry frame. In another embodiment, the actual value from patch.depth[1] (instead of the delta value) is stored in the second geometry frame.

[0067] Afterwards, the point cloud geometry is reconstructed from the encoded bitstream. This reconstruction is also replicated at the encoder, and the regenerated point cloud geometry is used for color approximation and texture generation at the encoder. During point cloud geometry reconstruction, multiple duplicate points are generated in the reconstructed point cloud geometry due to multiple values ​​in patch.depth[1] that are copied from patch.depth[0] during patch creation. Additionally, the duplication percentage increases at lower bitrate settings.

[0068] At lower bitrate settings, the increase in point repetition is a result of quantization artifacts of lossy compression. This is the second reason for duplicate points in the reconstructed point cloud. During lossy compression, especially for low bitrates, if the difference between the values ​​stored at corresponding pixel locations in two or more geometry frames is smaller than the quantization step size, the depth values ​​at these pixel locations may become identical after compression. Reconstructing 3D points from these pixel locations results in duplicate points in the point cloud.

[0069] Figure 5An example block diagram of an encoder 500 according to various embodiments of the present disclosure is shown. Figure 6 An example block diagram of a decoder 600 according to various embodiments of the present disclosure is shown. The encoder 500 may be similar to Figure 1 Server 104, Figure 1 Any of the client devices 106-116, and including Figure 2 Server 200 and Figure 3 The decoder 600 may be similar to the internal components of the electronic device 300. Figure 1 Any one of the client devices 106-116, and including Figure 3 The encoder 500 may communicate with the decoder 750 via the network 102. The embodiments of the encoder 500 and the decoder 600 are for illustration only. Other embodiments may be used without departing from the scope of the present disclosure.

[0070] The encoder 500 can compress, encode, and send a point cloud, a mesh, or both as an encoded bitstream 525. In various embodiments, the encoder 500 generates a plurality of 2-D frames in which the point cloud or mesh is mapped or projected. For example, the point cloud is unfolded and mapped onto a plurality of 2-D frames. For example, the point cloud can be unfolded along one axis (e.g., the Y axis), and the image is mapped along the remaining axes (e.g., the X and Z axes). In various embodiments, the encoder 500 generates an occupancy map (such as an occupancy map 522) that indicates where each pixel of the point cloud is located when the point cloud is mapped onto the 2-D frame. For example, the occupancy map indicates valid and invalid points for each frame. An invalid point is a position of a frame that does not correspond to a point of the point cloud, and a valid point is a position of a frame that corresponds to a point in a 3D point cloud. In various embodiments, the encoder 500 is a network server, a server computer (e.g., a management server), or any other electronic computing system capable of mapping three dimensions of a point cloud into two dimensions, compressing frames, and encoding images for transmission. In various embodiments, encoder 500 may be a "cloud" of computers interconnected by one or more networks, each of which is connected via Figure 1 The network 102 is accessed when the computing system utilizes clustered computers and components to act as a single pool of seamless resources.

[0071] The decoder 600 can decode, decompress, and generate a received encoded bitstream (such as Figure 5 In various embodiments, the decoder 550 generates a plurality of point clouds from a received bitstream comprising a plurality of 2-D frames. For example, each pixel of the point cloud may be mapped based on information received in the 2-D frames and a received occupancy map.

[0072] Encoder 500 illustrates a high-level overview of an embodiment of the present disclosure of an electronic device that handles repeated points in point cloud compression. In various embodiments, encoder 500 packages the point cloud for transmission to one or more decoders (e.g., decoder 600) via a bitstream. Encoder 500 receives point cloud 502 and encodes it. To this end, encoder 500 includes a patch generation block 504, an auxiliary patch information compression block 506, a frame packing block 512, a geometry reconstruction block 530, an occupancy map compression block 532, an image padding block 534, a video compression block 536, a smoothing block 538, and a multiplexer 524. When generating a coded bitstream 525, encoder 500 generates various frames (e.g., geometry frames 514 and texture frames 518) and an occupancy map 522.

[0073] Point cloud 502 represents input to encoder 500. Point cloud 502 can be stored in a memory associated with encoder 500. Patch generation block 504 decomposes point cloud 502 and creates multiple patches. Patches can be organized by attributes. For example, attributes can include geometry and texture. Geometry is the geographic location of each point of point cloud 502. Texture represents various aspects of each point of point cloud 502, such as color, intensity, normal, reflection, etc. Patch generation block 504 clusters geometry attributes and texture attributes. Clustering can be based on criteria such as normal direction, distance to projection frame, continuity, etc. Each generated cluster is represented as a patch.

[0074] In various embodiments, the patch generation block 504 projects and maps the points of the point cloud onto the 2-D frame via a one-to-one mapping. In various embodiments, the patch generation block 504 projects and maps the points of the point cloud onto the 2-D frame by projecting the points onto different planes. In various embodiments, projecting and mapping the points of the point cloud onto the 2-D frame may be based on raster scanning mapping, mapping based on X geometric coordinates, mapping based on Y geometric coordinates, mapping based on Z geometric coordinates, color-based mapping, normal direction-based mapping, etc.

[0075] In various embodiments, after projecting and mapping the points of the point cloud onto the 2-D frame, the frame packing block 512 classifies and packs the points into the 2-D frame to increase encoding efficiency. The frame packing block 512 classifies and packs the points into patches. In various embodiments, the encoder 500 may also include a classification engine (not shown) that places adjacent points and points including spatially correlated data adjacent to each other.

[0076] The patch generation block 504 also generates auxiliary patch information compressed by the auxiliary patch information block 506. The auxiliary patch information 506 is information associated with each generated patch. In various embodiments, the auxiliary patch information is metadata about the patch. For example, each patch has auxiliary information associated with it, such as (i) spatial position offset, (ii) depth offset, (iii) the position of the patch in the video frame, (iv) the size of the patch (e.g., the height and width of the patch), (v) the index of the projection plane, etc. The auxiliary patch information is used by a decoder such as the decoder 600 to reconstruct the point cloud.

[0077] The frame packing block 512 classifies and packs the patches into a geometry frame 514 and a texture frame 518. The geometry and texture information corresponding to the patches generated by the patch generation block 504 are packed into separate video frames, such as a geometry frame 514 and a texture frame 518. In various embodiments, the frame packing block 512 creates a geometry frame 514 and a texture frame 518. The frame packing block 512 also determines the height and width of the frame based on how the patches are arranged and packed into the frame. Conventional patches are arranged to occupy the smallest space in a frame. That is, the size of the frame is determined based on the position of each patch within the frame. Therefore, based on how the patches are oriented and positioned within the frame, the frame is the smallest possible size.

[0078] The frame packing block 512 also creates an occupancy map 522. The occupancy map 522 indicates the pixel locations in the frames (e.g., geometry frame 514 and texture frame 518) that contain valid points projected or mapped onto the point cloud on each frame. For example, the occupancy map 522 indicates whether each pixel on each frame (such as geometry frame 514 and texture frame 518) is a valid pixel or a blank. The valid pixels on the occupancy map indicate the pixels on the 2D frame that correspond to the 3D points in the point cloud. In various embodiments, the occupancy map 522 may be the same for each frame, such as geometry frame 514 and texture frame 518. In various embodiments, the occupancy map 522 represents multiple occupancy maps, each of which corresponds to a frame (e.g., geometry frame 514 and texture frame 518). The occupancy map 522 may be included in another frame similar to the geometry frame 514 or the texture frame 518. In various embodiments, the occupancy map 522 may be sent as metadata as a separate bitstream. Image padding block 534 adds padding to geometry frame 514 and texture frame 518 based on the locations indicated by occupancy map 522. Video compression block 536 compresses geometry frame 514 and texture frame 518, for example using a video compression codec such as HEVC or other video codecs.

[0079] In various embodiments, the geometry reconstruction block 530 performs prediction by reconstructing point cloud data for a current set of geometry frames 514 and / or texture frames 518 and comparing the current frame to the next set of frames. In these embodiments, the geometry reconstruction block 530 can generate difference or delta information between sets of frames to provide additional coding efficiency or improvement to related frames in the coded bitstream. After smoothing is performed by the smoothing block 538, the difference or delta information can be encoded as or in the next set of texture frames 518. In such a case, in various embodiments, the geometry reconstruction block 530 provides for handling duplicate points in point cloud compression, as discussed in more detail below. For example, the geometry reconstruction block 530 can identify and ignore or remove duplicate points in the reconstruction of the point cloud while performing prediction to reduce the amount of computing resources required to reconstruct the point cloud.

[0080] In various embodiments, the auxiliary patch information, the occupancy map 522 (after compression by the occupancy map compression block 532), or both are also encoded. Thereafter, the auxiliary patch information, the encoded geometry frame 514 and the encoded texture frame 518, and the occupancy map 522 are multiplexed by a multiplexer 524. The multiplexer 524 combines the auxiliary patch information 506, the encoded geometry frame 514, the encoded texture frame 518, and the occupancy map 522 to create a single encoded bitstream 525.

[0081] Figure 6 A high-level overview of a decoder 600 according to various embodiments of the present disclosure is shown. For example, the decoder 600 may be included in an electronic device to decode and reconstruct a point cloud for rendering. In various embodiments, the decoder extracts compressed geometry and attributes from the bitstream 625. In various embodiments, the decoder 600 maps the geometry and attribute information encoded in the bitstream 625 to regenerate the point cloud 602. The decoder 600 includes a demultiplexer 624, decompression blocks 636, 632, and 606 for video, occupancy map, and patch information decompression, and geometry and texture reconstruction blocks 614 and 618, respectively. To reconstruct the 3D point cloud, the decoder 600 uses information within patches of various video frames (e.g., geometry frames 514 and texture frames 518) and patch metadata demultiplexed and decompressed from the encoded bitstream 625 (e.g., included in the auxiliary patch information and occupancy map 522).

[0082] The decoder 600 receives an encoded bitstream 625, for example, encoded by and received from the encoder 500. Upon receiving the encoded bitstream 625, a demultiplexer 624 separates various data streams from the encoded bitstream 625. For example, the demultiplexer 624 separates various data streams, such as geometry frame information (initially geometry frame 514), texture frame information (e.g., texture frame 518), occupancy map information (e.g., occupancy map 522), and auxiliary patch information. For example, the demultiplexer 662 demultiplexes the encoded bitstream 626 into separate compressed bitstreams for point cloud geometry, texture, other attributes, occupancy map, and patch information.

[0083] The occupancy map information decompressed by the occupancy map decompression block 632 indicates valid pixel locations within the 2D frame to reconstruct the point cloud 602. The locations and sizes of the regular patches are retrieved from the decoded auxiliary patch information decompressed by the patch information decompression block 606. For example, the point locations can be reconstructed from the regular patches by adding the patch offsets (from the decoded auxiliary patch information) to the spatial locations of these points in the video and adding the depth offsets of the patches to the pixel values ​​in the spatial locations. In various embodiments, the decoding engine 672 can be lossless or lossy. In various embodiments, the decoder 60 can reconstruct the point cloud from a lossy or lossless encoded bitstream.

[0084] In various embodiments, the geometry reconstruction block 614 decodes the geometry frame information and identifies duplicate points, for example based on the point geometry and / or other attributes of the point, occupancy map, and / or patch information, as discussed in more detail below, and removes or ignores the duplicate points when reconstructing the point cloud 602. The geometry reconstruction block 614 generates the geometry of the points in the point cloud 602 from the 2D geometry frame smoothed by the smoothing block 620. The texture reconstruction block 618 applies texture (e.g., color, view-dependent color, normal, reflection, etc.) to the points in the point cloud 602 that are smoothed by the color smoothing block 622. When identifying and removing (or ignoring) duplicate points, the amount of hardware resources in geometry and texture reconstruction is greatly reduced in accordance with the present disclosure. In addition, in accordance with embodiments of the present disclosure, the ignored or removed points are duplicates so that their removal does not significantly affect the quality of the rendered 3D point cloud.

[0085] Although Figure 5 and Figure 6 An example of a decoder and an encoder is shown. Figure 5 and Figure 6 Make various changes. For example, Figure 5 and Figure 6 The various components in the Figure 5 and Figure 6Examples in, and other components can be added or deleted according to specific needs. For example, encoding and decoding for MPEG TMC2 (V-PCC) is used as an example, and the embodiments of the present disclosure can be used with other types of point cloud encoding and decoding technologies.

[0086] Figure 7 FIG. 7 is a flowchart of a geometrically-based duplicate point removal process 700 according to various embodiments of the present disclosure. For example, Figure 7 The processes described in the above can be respectively represented by Figure 2 Server 20 or Figure 3 Electronic equipment 3000 or Figure 5 and Figure 6 The geometry reconstruction blocks 530 and / or 614 are performed in, generally referred to herein as a system.

[0087] The system starts at the first pixel (i=0, j=0) in the geometry frame (step 702). If occupied (step 704) (e.g., based on an occupancy map), the system looks up the value in both depth layers (step 706), creates a new point (step 708), and adds the point to the reconstructed point cloud (step 710) for the first depth layer (D0).

[0088] For example, returning a reference Figure 4 In one embodiment, the encoder 500 uses two frames to encode the geometry and texture of each point cloud frame, obtains the value stored in the first geometry video frame from the value in the first depth layer (D0), and stores the value. After subtracting the value in the first depth layer from the corresponding value in the second depth layer (D1), the value stored in the second geometry frame is obtained. In such an implementation, delta (i, j) is defined as the difference between the values ​​of the second depth layer (D1) and the first depth layer (D0) at position (i, j), and delta (i, j) is obtained by directly reading the pixel value at position (i, j) in the second video frame after decoding the compressed bit stream. This pixel encoding is called the actual value or absolute value (i.e., the flag "absolute D1" == 1 at step 712) as the actual depth value (or the actual depth value minus the offset) of the point starting from the projection plane. For example, the offset is used as the minimum value of the depth values ​​of all points in a set of points (i.e., a patch). This is mainly done to ensure that the 10-bit depth value can be "decomposed" to be stored in an 8-bit video frame. The first frame is used to store the actual value minus the offset. For absolute values, the second frame also stores the actual value minus its offset.

[0089] As a result, the delta is calculated as the value of depth1-depth0 (step 716), and if it is not greater than zero (step 718) (or within a set threshold of zero in some embodiments), the system identifies the point as a geometric repeat and removes the point from the point cloud (or ignores / does not include / creates) when the system looks for the next width pixel (i.e., j=j+1) (step 724). However, if the point is not a geometric repeat, the system creates a new point and adds the new point to the reconstructed point cloud of the second depth (d1) layer (steps 720 and 722), similar to the operation performed on the first depth (d0) layer as described above.

[0090] In another embodiment, the encoder 500 uses two (or more) frames to encode the geometry and texture of each point cloud frame, obtains the value stored in the first geometry video frame from the value in the first depth layer (D0), and obtains the value stored in the second geometry frame from the second depth layer (D1). In such an implementation, after decoding the compressed bitstream, delta(i, j) is obtained by subtracting the value of the first geometry video frame at position (i, j) and the value of the second geometry video frame at position (i, j). This value is called the delta value (i.e., the flag "absolute D1" is not set in step 712) as the difference between the actual point depth and the point depth in the previous frame.

[0091] As a result, the delta is the value stored for the pixel, and if it is not greater than zero (step 718) (or within a set threshold of zero in some embodiments), the system identifies the point as a geometric duplicate and removes the point (or ignores / does not include / does not include / creates it from the point cloud when the system looks for the next width pixel (i.e., j=j+1)) (step 724). However, if the point is not a geometric duplicate, the system creates a new point and adds the new point to the reconstructed point cloud of the second depth (d1) layer (steps 720 and 722), similar to the operations performed on the points of the first depth (d0) layer as described above. Thereafter, the system continues the process for all frame width coordinates j (steps 724 and 726), and then all frame height coordinates i (steps 728 and 730) until all frames have been processed.

[0092] As discussed herein, absoluteD1 is a flag used to distinguish between the two encoding configurations described above. That is, if absoluteD1=1, the second video frame stores the value of the second depth layer (D1) (possibly minus the offset if an offset is used), otherwise the second video frame stores the difference layer of the second depth layer (D1) and the first depth layer (D0) (possibly minus the offset if an offset is used).

[0093] In these embodiments, the system avoids adding these geometry duplication points to the reconstructed point cloud geometry by checking the value of the delta, and only adding the point to the cloud if and only if the delta is greater than zero (i.e., not a geometry duplication). The point cloud from the decoded geometry video frame is reconstructed by a function called GeneratePointCloud(). The pseudo code for adding points is shown in the second frame below:

[0094] For each position in the 2D geometry frame:

[0095]

[0096]

[0097] exist Figure 6 The embodiments shown in are intended as examples. For example, when the comparison of two geometry and texture frames is discussed, any number of geometry and texture frames may be compared and duplicate points therein may be handled as discussed. In another example, related projection points (based on projection direction, neighborhood, color similarity, etc.) are grouped in patches. In the process of reconstructing a point cloud from a decoded geometry frame, points are retrieved from the geometry frame on a patch-by-patch basis. As described in the following pseudo-code provided below, duplicate checking is also performed on a patch-by-patch basis, with the variables defined in Table 1 below.

[0098] Pseudo code showing an embodiment of how to prevent duplicate points when projected points are stored in related groups within a geometry frame:

[0099]

[0100]

[0101]

[0102] Table 1: Description of the variables in the pseudocode provided above, which illustrates how duplicate points are handled when the projected points are stored in related groups within the geometry frame.

[0103]

[0104]

[0105] In yet another embodiment, where the projected points are grouped into patches that are packed and encoded in two geometry video frames and two attribute video frames per point cloud frame, and the only attribute present in the point cloud is color (or texture), the point cloud at the decoder is reconstructed from the patches in the two decoded geometry and two decoded attribute frames. Additionally, in this embodiment, the patch size is differentially encoded as auxiliary patch information (i.e., as part of the metadata).

[0106] In this embodiment, first, the patch information is retrieved as follows:

[0107] Patch2dShiftU=patch.patch_2d_shift_u*patch.occpancy_packing_block_size

[0108] Patch2dShiftV=patch.patch_2d_shift_v*patch.occupancy_packing_block_size

[0109] Patch2dSizeU=patch.patch_2d_delta_size_u*patch.occupancy_packing_block_size

[0110] Patch2dSizeV=patch.patch_2d_delta_size_v*patch.occupancy_packing_block_size

[0111] Patch3dShiftX=patch.patch_3d_shift_tangent_axis

[0112] Patch3dShiftY=patch.patch_3d_shift_bitangent_axis

[0113] Patch3dShiftZ=patch.patch_3d_shift_normal_axis

[0114] where,

[0115] occupancy_packing_block_size is the same as occupancyResolution inTable 1.

[0116] patch_2d_shift_u specifies the x-coordinate of the top-left corner ofthe patch bounding box expressed as multiple of occupancy_packing_block_size.The value of patch_2d_shift_u is in the range of 0to(frame_width / occupancy_packing_block_size-1),inclusive.

[0117] patch_2d_shift_v specifies the y-coordinate of the top-left corner ofthe patch bounding box expressed as multiple of occupancy_packing_block_size.The value of patch_2d_shift_v is in the range of 0to(frame_height / occupancy_packing_block_size-1),inclusive.

[0118] patch_2d_delta_size_u is the difference of patch width between thecurrent patch and the previous one.

[0119] patch_2d_delta_size_v is the difference of patch height between thecurrent patch and the previous one.

[0120] patch_3d_shift_tangent_axis is the same as patch.tangentAxis in Table1.

[0121] patch_3d_shift_bitangent_axis is the same as patch.bitangentAxis inTable 1.

[0122] patch_3d_shift_normal_axis is the same as patch.normalAxis in Table1.

[0123] In this step, a point cloud without added color is generated from the decoded geometry frame as well as the decoded frame patch information and the decoded occupancy map. In the second step, the generated (reconstructed) point cloud is input into the geometry smoothing process for lossy encoding. Finally, color is added to the smoothed cloud points from the decoded texture frame. The decoded geometry frame contains two frames for each point cloud corresponding to the near layer and the far layer, and the decoded attribute frame contains two frames for each point cloud corresponding to the near layer and the far layer. If the occupancy value at that pixel is 1, two points are generated for each pixel location, one from the near layer and one from the far layer.

[0124] Two points are generated from each pixel in the patch, where x is from Patch2dShiftU to (Patch2dShiftU + Patch2dSizeU) and y is from Patch2dShiftV to (Patch2DShiftV + Patch2dSizeV). If the occupancy value oFrame[x][y] of the pixel is equal to 1, the first point is derived as follows and added to the reconstructed point cloud:

[0125] 1. First, get x_new and y_new for pixel (x, y) based on the patch packing direction.

[0126] 2. The coordinates of the points are derived from the geometry frame as follows:

[0127] point1[0]=x_new+Patch3dShiftX

[0128] point1[1]=y_new+Patch3dShiftY

[0129] point1[2]=gFrame[0][x_new][y_new]+Patch3dShiftZ

[0130] 3. Color the points by using the attribute frame as follows:

[0131] point1[3]=aFrame[0][0][x_new][y_new]

[0132] point1[4]=aFrame[0][1][x_new][y_new]

[0133] point1[5]=aFrame[0][2][x_new][y_new]

[0134] 4.Then add the point to the reconstructed point cloud as follows:

[0135] addPointToPointCloud(recPc,point1)

[0136] The second point is derived as follows:

[0137] 1. First, get the coordinates of the point from the geometry frame as follows:

[0138]

[0139]

[0140] 2. Color the points by using the attribute frame as follows:

[0141] point2[3]=aFrame[1][0][x_new][y_new]

[0142] point2[4]=aFrame[1][1][x_new][y_new]

[0143] point2[5]=aFrame[1][2][x_new][y_new]

[0144] 3. If the point is not a duplicate of the first point, it is added to the reconstructed point cloud as follows:

[0145] if((point1[0]–point2[0])&&(point1[1]–point2[1]&&(point1[2]–point2[2]))

[0146] addPointToPointCloud(recPc,point2)

[0147] A pseudo-code implementation of the embodiments discussed in the above steps is provided below.

[0148] Pseudocode for an example where projected points are grouped into patches which are then packed and encoded in two geometry video frames and two attribute video frames per point cloud frame. The only attribute present in the point cloud is color (or texture). The point cloud at the decoder is reconstructed from the patches in the two decoded geometry and two decoded attribute frames.

[0149]

[0150]

[0151] Embodiments of the present disclosure recognize that at low bit rates, if point duplication is prevented at the encoder, slight degradation may occur in the reconstructed point cloud. Therefore, in some embodiments, if the bit rate is low (i.e., the encoder is not embedded), duplicate point handling is only implemented at the decoder. Pseudocode for preventing point duplication during point cloud geometry reconstruction only at the decoder is shown below.

[0152] Pseudocode for adding points from the second frame of GeneratePointCloud() to avoid duplication of points at the decoder:

[0153] For each position in the 2D geometry frame:

[0154] --->if(first frame):

[0155] ------>generate 3D point from the position in 2D frame and value ofdepth at that location

[0156] ------>add point to the point cloud

[0157] --->if(second frame):

[0158] ------>if(absoluteD1):

[0159] ---------->delta=value in the current location in frame–value in the current location in the first frame

[0160] ------>else:

[0161] ---------->delta=value in the current location in frame

[0162] ------>if(encoder)

[0163] --------->generate 3D point from the position in the 2D frame and value of the depth at that location

[0164] ------>else if(decoder and delta>0):

[0165] --------->generate 3D point from the position in the 2D frame and value of the depth at that location

[0166] Figure 8 An example block diagram of an encoder 800 for a multi-attribute point cloud according to various embodiments of the present disclosure is shown. The embodiment of the encoder 800 is for illustration only. Other embodiments may be used without departing from the scope of the present disclosure.

[0167] In some high-end immersive media applications, multi-attribute point clouds are used, where there are multiple attribute values ​​for the same geometry point (e.g., view-dependent color, normal, reflectance, etc.). These applications typically use surface light fields (such as point clouds) to render view-dependent attributes. That is, the color of a point in the rendered (or observed) point cloud depends on the view direction. In other cases, such as data obtained from LiDAR, the source point cloud may contain multiple view-dependent attributes per geometry point.

[0168] In these embodiments, the source point cloud contains multiple attributes for each uniquely defined geometric coordinate. That is, the geometric coordinates are not repeated in the source point cloud. For such point cloud reconstruction at the decoder (and at the encoder), duplicate points can be removed by simply comparing the geometric values ​​of the points during reconstruction of the geometric points, as described above with respect to Figure 5-7 discussed.

[0169] For multi-attribute encoding, the point cloud geometry is projected from 3D space to 2D space as before. Essentially, the depth (the coordinate of the point along the projection direction) is stored in multiple depth layers in multiple geometry video frames. The other two coordinates of the point are implicitly encoded as the 2D position of the point in the multiple geometry video frames. In addition, each attribute uses a separate set of attribute video frames. For example, the patch generation block 504 and frame packing block 512 in the encoder generate multiple groups (n groups) of attribute frames 8180 to 818N-1, one group for each different type of attribute associated with the geometry point in the geometry frame group 514. Compression and multiplexing are related to Figure 5 The discussion occurs similarly. Figure 5 , generating an encoded bitstream 825 for multi-attribute 3D point cloud data.

[0170] Using these encoding techniques, there is a correspondence between the 2D position of the geometry and the attributes in the multiple geometry and attribute frames for each projected point. As a result, the geometry associated with each point in the point cloud is unique, based on the assumption that there are no duplicate points in the source (uncompressed) point cloud. Therefore, the decoder 600 can handle duplicate points using the geometry information in the geometry frame 514 alone to detect the duplication of points during reconstruction of the point cloud from the uncompressed and decoded bitstream. Therefore, the decoder 600 can use similar techniques to handle the duplication of points, as described with respect to Figure 6 and Figure 7 As described.

[0171] Fig. 9 FIG. 9 is a flowchart of a process 900 for removing duplicate points based on geometry and attributes according to various embodiments of the present disclosure. For example, Fig. 9 The processes described in the above can be respectively represented by Figure 2 Server 200 or Figure 3 The electronic device 300 or Figure 5 and Figure 6 The geometry reconstruction blocks 530 and / or 614 in are performed, generally referred to herein as a system.

[0172] For example, in various embodiments, in applications where duplicate points are defined as having identical geometry and attributes, comparing the geometry alone is not sufficient to determine that the points are duplicates. In these embodiments, both the geometry and attributes are compared to determine duplicate points. Figure 8 In , color is used as an example, but any other attribute or attributes can be used in conjunction with geometry to determine if a point is a duplicate. Also, as mentioned above, any number of layers can be used instead of two.

[0173] The system starts at the first pixel (i=0, j=0) in the geometry frame (step 902). If occupied (step 904) (e.g., based on an occupancy map), the system looks up the values ​​in both the geometry frame and both the color frames (step 906), creates a new point (step 908), and adds the point to the reconstructed point cloud of the first depth layer (D0) (step 910).

[0174] If the flag "absolute D1" == 1 is set (at step 912), the depth delta is calculated as the value of depth1-deep0 (step 916), and if not greater than zero and the color values ​​are the same (step 918) (or within some set zero / similarity threshold), the system identifies the point as a geometry duplicate and a color duplicate and removes the point from the point cloud (or ignores / does not include / creates) when the system looks at the next width pixel (i.e., j=j+1) (step 924). However, if the point is not a geometry and color duplicate, the system creates a new point and adds the new point to the reconstructed point cloud at the second depth (d1) layer (steps 920 and 922), similar to what was done above for the points at the first depth (d0) layer.

[0175] For the unset flag "absolute D1" == 1 (at step 912), the delta is the value stored for the pixel, and if not greater than zero and the color value is the same (step 918) (or within a set zero / similarity threshold in some embodiments), the system identifies the point as a geometry duplicate and a color duplicate and removes the point from the point cloud (or ignores / does not include / creates) when the system looks for the next width pixel (i.e., j=j+1) (step 724). However, if the point is not a geometry and color duplicate, the system creates a new point and adds the new point to the reconstructed point cloud for the second depth (d1) layer (steps 920 and 922), similar to what was done above for the points in the first depth (d0) layer. Thereafter, the system continues processing for all frame width coordinates j (steps 924 and 926), followed by processing for all frame height coordinates i (steps 928 and 930) until the entire set of frames has been processed.

[0176] In another embodiment, the projected points are grouped in patches before being stored in the video frame. Therefore, in the process of reconstructing the point cloud from the decoded geometry frame, points are retrieved from the geometry and attribute frames on a patch-by-patch basis. Duplicate checking also needs to be done on a patch-by-patch basis. Pseudocode for this is provided below, and Table 2 lists the variables used in the pseudocode, which itself is shown below.

[0177] Example pseudo code of one embodiment for comparing geometry and attribute information to prevent point duplication when grouping 2D points into related patches before storing them in corresponding frames:

[0178]

[0179]

[0180]

[0181] Table 2: Description of variables in the example pseudo code provided above for the embodiment to compare geometry and attribute information to prevent point duplication when grouping 2D points into related patches before storing them in corresponding frames.

[0182]

[0183] In another embodiment, the geometric coordinates are repeated as many times as there are attributes in the source (multi-attribute) point cloud. In other words, there is an inherent repetition of geometric coordinates in the source (uncompressed) point cloud that represent the same location in 3D space. For this type of point cloud, it cannot be determined whether a given reconstructed point is a duplicate based solely on comparing geometry information. Instead, the geometry and associated color set (and other attributes, if present) are used to determine whether a point is a duplicate of a previous point.

[0184] In one embodiment, similar to the previous illustrative example, the number of depth layers ( Figure 8 The "d" in ) can be limited to two. In this case, the same 3D point (with different attributes) can be encoded in separate patches. In another embodiment, the number of depth layers ( Figure 8 The "d" in ) can be greater than or equal to the number of attributes (i.e., the number of geometric repetitions in the source point cloud). Even for such an embodiment, the same 3D point (with different attributes) can be encoded in separate patches. Therefore, in both embodiments, it is very likely that all points that potentially have the same geometric coordinates and / or attribute values ​​cannot be used for comparison (for calculating the "delta") at the same time. As a result, a mapping data structure (such as a dictionary) is first used to store the geometric coordinate values ​​and the geometric frame information. For example, a unique hash is constructed based on the reconstructed coordinate values ​​and used as a keyword for the dictionary, and the frame number from which the point is retrieved is stored in a list corresponding to the keyword in the dictionary. The size of the list corresponding to each key in the dictionary (which can be used to uniquely identify the 3D geometric coordinates) indicates the number of points that share the same geometric coordinates. After reconstructing all geometric points, only those points are obtained (using the hash key) whose corresponding list size is greater than the number of attributes (indicating the generation of duplicate points). Then, only the "n" most unique points are retained using the attribute information.

[0185] In yet another embodiment, regardless of how the geometry and attributes of the multi-attribute point cloud are defined, the reconstructed point cloud contains multiple repetitions of geometry with different attributes (derived from the multi-attribute nature of the point cloud). For such a reconstructed point cloud, it is again determined whether a given reconstructed point is a duplicate by comparing the geometry and attributes.

[0186] As discussed above and reiterated herein, it is not necessary to compare exact similarities between geometric values ​​and / or attribute values ​​to determine whether points in the reconstructed point cloud are duplicates. In some embodiments, points will be duplicates if they are close to each other using an appropriate metric. For example, a point can be defined as a duplicate of another point if the absolute delta (as described above) is less than some small threshold and / or the Euclidean distance calculated for the attributes between the two points is less than a small threshold. This strategy is particularly useful when some floating point attributes (e.g., reflections, normal vectors, etc.) are also associated with each point in the point cloud.

[0187] In yet another embodiment, a single geometry video frame and a corresponding single attribute video frame consist of two depth layers for each point cloud frame. Several interpolation techniques can be used to convert the two depth layers into a single video geometry frame. In one embodiment, the decoded geometry frame includes a single frame for each point cloud, and the decoded attribute frame contains a single frame for each point cloud. The single geometry layer contains interlaced pixels from the near and far layers of the geometry, and the single attribute layer contains interlaced pixels from the near and far layers of the attribute. An interlaced (geometry or attribute) frame is formed by taking a pixel from the near layer and then taking a pixel from the far layer. For example, at pixel (x, y), if (x+y) is even, the value from the near layer is used, and when (x+y) is odd, the value from the far layer is used.

[0188] If the occupancy value of a pixel is 1, two points will be generated for each pixel position, one from the near layer and one from the far layer. Since the interleaved frames only contain information from one of the two layers, the lost layer value can be predicted by interpolating the neighbors. When (x+y) is an even number, the value of the near layer can be used and the value of the far layer can be interpolated. Similarly, when (x+y) is an odd number, the value of the far layer can be used and the value of the near layer can be interpolated. In such an embodiment, duplication is avoided at the encoder by copying the depth value of the 3D point to multiple depth layers. However, as a result of both lossy encoding and / or pixel interpolation and extrapolation (prediction), duplication of geometric points and / or attribute values ​​may result. The handling of duplication of points in the final reconstructed point cloud follows the above description of Figure 5-7 process discussed. More specifically, after the geometry and / or attribute values ​​of the first or second layer have been derived, the predicted geometry and / or attribute values ​​are compared with the corresponding geometry and / or attribute values ​​of the same 2D pixel position obtained directly from the decoded geometry and / or attribute video frame. If a possible duplication is detected, that is, if the geometry and / or attribute value of the predicted (extrapolated) point is exactly equal to or within a certain threshold of the corresponding geometry and / or attribute value of the directly obtained point, the predicted point is not added to the reconstructed point cloud.

[0189] Various embodiments of the present disclosure may be applied to the encoding and decoding of polygonal meshes (or "meshes"). Like point clouds, meshes may be used to represent virtual 3D objects. Like point clouds, most mesh files also contain a list of vertices and attributes. In addition, most mesh files also contain a list of faces and edges used to generate piecewise continuous surfaces. Embodiments of the present disclosure may be used in conjunction with any mesh compression algorithm that employs depth layer-based techniques (e.g., MPEG V-PCC) to handle repeated vertices in order to take advantage of mature 2D image / video compression techniques.

[0190] Fig.10 1 is a flowchart of a process 1000 for decoding 3D point cloud data according to various embodiments of the present disclosure. For example, Fig.10 The processes described in the above can be respectively represented by Figure 2 Server 200 or Figure 3 The electronic device 300 or Figure 5 and Figure 6 The geometry reconstruction blocks 530 and / or 614 in are performed. Generally referred to herein as a system.

[0191] The process begins with the system receiving an encoded bitstream including 3D point cloud data (step 1005). For example, in step 1005, the 3D point cloud data can be compressed into 2D frames and metadata, wherein the frames are encoded to contain the 3D point cloud data. Thereafter, the system decodes the bitstream to identify a plurality of 2D frames (step 1010). For example, in step 1010, the system can decompress and separate the compressed frames and metadata as described above.

[0192] The system then identifies corresponding pixels in the two or more geometry frames (step 1015). For example, in step 1015, the system can identify corresponding pixels based on the positions of corresponding pixels in two or more 2D geometry frames from multiple 2D frames. For example, for different depths or layers, corresponding pixels can have the same i, j pixel coordinate values ​​between frames. In some embodiments, the system can use an occupancy map to identify corresponding pixels. Thereafter, the system identifies depth values ​​for the corresponding pixels (step 1020). For example, in step 1020, the system can read the values ​​encoded at the i, j coordinate positions in the two or more geometry frames.

[0193] Then, the system identifies the duplicate points based on the identified depth values ​​(step 1025). For example, in step 1025, the system can identify duplicate points based on the depth values ​​of the same or similar corresponding pixels in the identified two or more geometry frames. In one example, if the actual (or absolute) depth value is encoded, the system can identify the first depth value of the first corresponding pixel in the first of the two or more geometry frames, and then compare the value with the second second depth value of the second corresponding pixel in the two or more geometry frames. Then, if the compared depth values ​​are the same or within a threshold of each other, the system determines that the point represented by the second pixel in the second geometry frame is a duplicate point. Here, the depth value indicates the depth of the point in the 3D point cloud from the plane to which the point is projected, and is encoded as the actual depth value or the actual depth value minus the offset of the corresponding pixel position in the geometry frame. In another example, if the delta value is encoded, the system can determine whether the depth value of the second corresponding pixel in the second geometry frame of the two or more geometry frames is zero or below a threshold. If so, the system determines the point represented by the second pixel in the second geometry frame as a duplicate point. Here, the depth values ​​indicate the depth of the point in the 3D point cloud from the plane onto which the point is projected, and for the second geometry frame these depths are encoded as delta values ​​(possibly with an offset) relative to the first geometry frame.

[0194] In addition, in various embodiments, in addition to the geometry value, the system may also use texture or attribute values ​​to determine whether a point is repeated. The system may determine that a pixel in a second geometry frame of two or more geometry frames is geometrically repeated based on the depth value of the corresponding pixel, as described above. In addition, the system may determine whether one or more second texture values ​​corresponding to one or more geometrically repeatable second pixels are texture repetitions of one or more first texture values ​​corresponding to one or more first pixels. If so, the system determines that the points are texture repeats; if both the geometry and the texture are repeated, the point is determined to be a repeated point. For example, each identified repeated point has a geometry represented by two or more geometry frames and one or more attributes indicated by one or more attribute frame sets, and the system identifies repeated points based on the two or more geometry frames and the one or more attribute frame sets.

[0195] Thereafter, the system removes or ignores the identified duplicate points while reconstructing the 3D point cloud data (step 1030). For example, in step 1030, when generating one or more 3D objects, the system may not create, remove or add duplicate points to the point cloud.

[0196] In various embodiments, whether or not duplicate points are not added may depend on the type of application used to reconstruct the 3D point cloud. For example, when reconstructing 3D point cloud data for an application type that is not view-dependent but uses at least some of the identified duplicate points, the system may remove or ignore the identified duplicate points when reconstructing 3D point cloud data for a view-dependent application type. For example, view-dependent applications (e.g., 6DoF applications) allow objects to be seen from various viewpoints (e.g., AR / VR applications where users can "walk around" in a scene). In this case, other properties may be required to view points from different angles (e.g., color or reflection depending on the view, etc.). In these embodiments, the system can identify the type of application used to reconstruct the 3D point cloud (i.e., whether it is view-dependent) and remove or ignore points that are geometrically duplicated but not texture (or attribute) duplicated for view-dependent applications. For view-independent applications, the system removes or ignores any points that are geometrically duplicated when reconstructing the 3D point cloud data, including those that are not texture (or attribute) duplicated.

[0197] Although Figure 7 , 9 10 show an example of a process for removing duplicate points or decoding 3D point cloud data, but may be used for Figure 7 , 9 Various changes may be made to 10. For example, although shown as a series of steps, the various steps in each figure may overlap, occur in parallel, occur in a different order, or occur multiple times.

[0198] Although the drawings show different examples of user equipment, various changes may be made to the drawings. For example, the user equipment may include any number of each component in any suitable arrangement. Generally, the drawings do not limit the scope of the present disclosure to any particular configuration. In addition, although the drawings show operating environments in which various user equipment features disclosed in this patent document may be used, these features may be used in any other suitable system.

[0199] Although the present disclosure has been described with exemplary embodiments, various changes and modifications may be suggested to one skilled in the art. The present disclosure is intended to encompass such changes and modifications as fall within the scope of the appended claims.

Claims

1. A decoder for decoding three-dimensional 3D point cloud data, the decoder comprising: Communication interface; as well as At least one processor is coupled to the communication interface and is configured to: receiving an encoded bitstream including 3D point cloud data from an encoder, Decode the bitstream to identify multiple 2D frames, identifying corresponding pixels in two or more of the plurality of 2D frames based on locations of corresponding pixels in the two or more of the plurality of 2D frames, identifying depth values ​​and attribute values ​​for corresponding pixels in the two or more geometry frames, wherein the attribute values ​​are indicated by one or more attribute frame sets, identifying duplicate points in at least one of the two or more geometry frames based on the identified depth values ​​and the identified attribute values ​​of corresponding pixels in the two or more geometry frames, wherein the identified duplicate points include geometry duplicates and attribute duplicates, and When reconstructing 3D point cloud data, remove or ignore identified duplicate points. The removed duplicate points or the ignored identified duplicate points are not used to reconstruct the 3D point cloud.

2. The decoder according to claim 1, wherein: To identify duplicate points in at least one of the two or more geometry frames based on the identified depth value and the identified attribute value, the processor is further configured to: identifying, in a first geometry frame of the two or more geometry frames, a first depth value for a first pixel of the corresponding pixels, identifying, in a second geometry frame of the two or more geometry frames, a second depth value of a second pixel of the corresponding pixels, wherein the first pixel and the second pixel have corresponding positions in their respective geometry frames, comparing the identified first depth value and the second depth value to determine whether the identified first depth value and the second depth value are the same or within a threshold of each other, and determining that the point represented by the second pixel in the second geometry frame is one of the duplicate points based on determining that the first depth value and the second depth value are the same or within a threshold of each other, Wherein the depth value indicates the depth of a point in the 3D point cloud from a plane onto which the point is projected and is encoded in a corresponding pixel position in the two or more geometry frames as an actual depth value or an actual depth value minus an offset.

3. The decoder according to claim 1, wherein: To identify duplicate points in at least one of the two or more geometry frames based on the identified depth value and the identified attribute value, the processor is further configured to: identifying, in a first geometry frame of the two or more geometry frames, a first depth value for a first pixel of the corresponding pixels, determining whether a second depth value of a second pixel of corresponding pixels in a second geometry frame of the two or more geometry frames is zero or below a threshold, the first pixel and the second pixel having corresponding positions in their respective geometry frames, and determining that the point represented by the second pixel in the second geometry frame is one of the duplicate points based on determining that the second depth value is zero or below a threshold, The depth value indicates the depth of a point in the 3D point cloud from a plane onto which the point is projected, and for the second geometry frame is encoded as a delta value relative to the first geometry frame.

4. The decoder according to claim 1, wherein: To identify duplicate points in at least one of the two or more geometry frames based on the identified depth value and the identified attribute value, the processor is further configured to: determining, based on depth values ​​of corresponding pixels in the two or more geometry frames, that one or more second pixels of corresponding pixels in a second geometry frame of the two or more geometry frames are geometric repeats of one or more first pixels of corresponding pixels in a first geometry frame of the two or more geometry frames, Based on determining that the one or more second pixels are geometrically repeated, determining whether one or more second property values ​​of the second pixels respectively corresponding to the one or more geometrically repeated pixels are property repeats of one or more first property values ​​respectively corresponding to the one or more first pixels, and determining that a point represented by at least one of the one or more second pixels is one of the repeated points based on determining that at least one of the one or more second pixels is both a geometric repeat and an attribute repeat of at least one of the one or more first pixels, The first attribute value and the second attribute value are respectively identified from a first attribute frame and a second attribute frame in the plurality of 2D frames, and the first attribute frame and the second attribute frame correspond to a first geometry frame and a second geometry frame, respectively.

5. The decoder according to claim 1, wherein: The duplicate points removed or the identified duplicate points ignored are based on the type of application used to reconstruct the 3D point cloud, The type of application includes whether it depends on a view.

6. An encoder for encoding three-dimensional 3D point cloud data, the encoder comprising: Communication interface; as well as At least one processor is coupled to the communication interface and is configured to: encoding a first set of two-dimensional 2D frames of the 3D point cloud data into a coded bitstream, performing prediction on the first set of 2D frames to encode a second set of 2D frames of 3D point cloud data into an encoded bitstream, wherein, to perform the prediction, the processor is further configured to: Decoding the bitstream to generate multiple 2D frames; identifying corresponding pixels in two or more geometry frames based on locations of corresponding pixels in the two or more geometry frames in the plurality of 2D frames, identifying depth values ​​and attribute values ​​for corresponding pixels in the two or more geometry frames, wherein the attribute values ​​are indicated by one or more attribute frame sets, identifying duplicate points in at least one of the two or more geometry frames based on the identified depth values ​​and the identified attribute values ​​of corresponding pixels in the two or more geometry frames, wherein the identified duplicate points include geometry duplicates and attribute duplicates, Determine whether the identified duplicate points are removed or ignored when reconstructing the 3D point cloud data, Sending an encoded bitstream including 3D point cloud data to a decoder, The removed duplicate points or ignored duplicate points are not used to reconstruct the 3D point cloud.

7. The encoder according to claim 6, wherein To identify duplicate points in at least one of the two or more geometry frames based on the identified depth value and the identified attribute value, the processor is further configured to: identifying, in a first geometry frame of the two or more geometry frames, a first depth value for a first pixel of the corresponding pixels, identifying, in a second geometry frame of the two or more geometry frames, a second depth value of a second pixel of the corresponding pixels, wherein the first pixel and the second pixel have corresponding positions in their respective geometry frames, comparing the identified first depth value and the second depth value to determine whether the identified first depth value and the second depth value are the same or within a threshold of each other, and determining that the point represented by the second pixel in the second geometry frame is one of the duplicate points based on determining that the first depth value and the second depth value are the same or within a threshold of each other, Wherein the depth value indicates the depth of a point in the 3D point cloud from a plane onto which the point is projected and is encoded in a corresponding pixel position in the two or more geometry frames as an actual depth value or an actual depth value minus an offset.

8. The encoder according to claim 6, wherein: To identify duplicate points in at least one of the two or more geometry frames based on the identified depth value and the identified attribute value, the processor is further configured to: identifying, in a first geometry frame of the two or more geometry frames, a first depth value for a first pixel of the corresponding pixels, determining whether a second depth value of a second pixel of corresponding pixels in a second geometry frame of the two or more geometry frames is zero or below a threshold, the first pixel and the second pixel having corresponding positions in their respective geometry frames, and determining that the point represented by the second pixel in the second geometry frame is one of the duplicate points based on determining that the second depth value is zero or below a threshold, The depth value indicates the depth of a point in the 3D point cloud from a plane onto which the point is projected, and for the second geometry frame is encoded as a delta value relative to the first geometry frame.

9. The encoder according to claim 6, wherein: To identify duplicate points in at least one of the two or more geometry frames based on the identified depth value and the identified attribute value, the processor is further configured to: determining, based on depth values ​​of corresponding pixels in the two or more geometry frames, that one or more second pixels of corresponding pixels in a second geometry frame of the two or more geometry frames are geometric repeats of one or more first pixels of corresponding pixels in a first geometry frame of the two or more geometry frames, Based on determining that the one or more second pixels are geometrically repeated, determining whether one or more second property values ​​of the second pixels respectively corresponding to the one or more geometrically repeated pixels are property repeats of one or more first property values ​​respectively corresponding to the one or more first pixels, and determining that a point represented by at least one of the one or more second pixels is one of the repeated points based on determining that at least one of the one or more second pixels is both a geometric repeat and an attribute repeat of at least one of the one or more first pixels, The first attribute value and the second attribute value are respectively identified from a first attribute frame and a second attribute frame in the plurality of 2D frames, and the first attribute frame and the second attribute frame correspond to a first geometry frame and a second geometry frame, respectively.

10. The encoder of claim 6, wherein the duplicate points removed or the identified duplicate points ignored are based on a type of application used to reconstruct the 3D point cloud, and in, The type of an application includes whether it depends on a view.

11. A method performed by a decoder, the method comprising: Decode the bitstream to identify multiple 2D frames, identifying corresponding pixels in two or more of the plurality of 2D frames based on locations of corresponding pixels in the two or more of the plurality of 2D frames; identifying depth values ​​and attribute values ​​for corresponding pixels in the two or more geometry frames, wherein the attribute values ​​are indicated by one or more attribute frame sets; identifying repeated points in at least one of the two or more geometry frames based on the identified depth values ​​and the identified attribute values ​​of corresponding pixels in the two or more geometry frames, wherein the identified repeated points include geometric repetitions and attribute repetitions; and When reconstructing 3D point cloud data, remove or ignore identified duplicate points. The removed duplicate points or the ignored identified duplicate points are not used to reconstruct the 3D point cloud.

12. A method performed by an encoder, the method comprising: encoding a first set of two-dimensional 2D frames of the 3D point cloud data into a coded bitstream, Performing prediction on the first set of 2D frames to encode a second set of 2D frames of 3D point cloud data into a coded bitstream, wherein performing prediction comprises: Decoding the bitstream to generate multiple 2D frames; identifying corresponding pixels in two or more geometry frames of the plurality of 2D frames based on locations of the corresponding pixels in the two or more geometry frames; identifying depth values ​​and attribute values ​​for corresponding pixels in the two or more geometry frames, wherein the attribute values ​​are indicated by one or more attribute frame sets; identifying repeated points in at least one of the two or more geometry frames based on the identified depth values ​​and the identified attribute values ​​of corresponding pixels in the two or more geometry frames, wherein the identified repeated points include geometry repetitions and attribute repetitions; determining whether the identified duplicate points are to be removed or ignored when reconstructing the 3D point cloud data; and Sending an encoded bitstream including 3D point cloud data to a decoder, The removed duplicate points or ignored duplicate points are not used to reconstruct the 3D point cloud.

Images