Motion estimation in geometry point cloud compression

Abstract

1. A device for encoding point cloud data, comprising: a memory for storing the point cloud data; and one or more processors coupled to the memory and implemented in circuitry, the one or more processors configured to identify a first set of global motion parameters from global positioning system information, the one or more processors further configured to determine, based on the first set of global motion parameters, a second set of global motion parameters to be used for global motion estimation for a current frame, and apply motion compensation to a reference frame based on the second set of global motion parameters to generate a global motion compensated frame for the current frame.

Description

[Technical Field]

[0001] This application claims priority to U.S. Patent Application No. 17 / 495,428, filed October 6, 2021, and U.S. Provisional Patent Application No. 63 / 088,936, filed October 7, 2020, and U.S. Provisional Patent Application No. 63 / 090,627, filed October 12, 2020, and U.S. Provisional Patent Application No. 63 / 090,657, filed October 12, 2020, the entire contents of each of which are incorporated herein by reference. U.S. Patent Application No. 17 / 495428, filed October 6, 2021, claims priority to U.S. Provisional Patent Application No. 63 / 088936, filed October 7, 2020, and U.S. Provisional Patent Application No. 63 / 090627, filed October 12, 2020, and U.S. Provisional Patent Application No. 63 / 090657, filed October 12, 2020.

[0002] The present disclosure relates to point cloud encoding and decoding. [Background technology]

[0003] A point cloud is a collection of points in three-dimensional space. The points may correspond to points on an object in three-dimensional space. Thus, a point cloud may be used to represent the physical content of a three-dimensional space. Point clouds may have utility in a wide variety of situations. For example, a point cloud may be used in the context of autonomous vehicles to represent the location of objects on a road. In another example, a point cloud may be used in the context of representing the physical content of an environment for purposes of positioning virtual objects in an augmented reality (AR) or mixed reality (MR) application. Point cloud compression is the process for encoding and decoding a point cloud. Encoding a point cloud can reduce the amount of data required to store and transmit the point cloud. Summary of the Invention [Means for solving the problem]

[0004] Generally, this disclosure describes techniques for improving visualization of point cloud frames that may use the geometry-based point cloud compression (G-PCC) codec under development within MPEG by the 3D Graphics Coding (3DG) group. A G-PCC coder (e.g., a G-PCC encoder or a G-PCC decoder) may be configured to apply motion compensation to a reference frame to generate a motion-compensated frame. For example, the G-PCC encoder may apply “global” motion compensation to a reference frame (e.g., a predicted frame) to accommodate rotation of the entire reference frame and / or translation of the entire reference frame. In this example, the G-PCC encoder may apply “local” motion compensation to accommodate rotation and / or translation at a finer scale than global motion compensation. For example, the G-PCC encoder may apply local node motion compensation to one or more nodes (e.g., portions of a frame) of the global motion-compensated frame.

[0005] According to the techniques of this disclosure, a G-PCC coder (e.g., a G-PCC encoder or a G-PCC decoder) may be configured to apply global motion compensation based on global positioning system information (e.g., information from any satellite system, such as the Global Positioning System (GPS) implemented in the United States). For example, the G-PCC encoder may identify a first set of global motion parameters from the global positioning system information. The first set of global motion parameters may include orientation parameters (e.g., roll, pitch, yaw, or angular velocity) and / or position parameters (e.g., displacement or velocity along the x-, y-, or z-dimension). In this example, the G-PCC encoder may determine a second set of global motion parameters based on the first set of global motion parameters. For example, the G-PCC encoder may convert the orientation parameters and / or position parameters into a rotation matrix and a translation vector for the current frame. In this way, a G-PCC coder (e.g., a G-PCC encoder or a G-PCC decoder) may apply global motion compensation using satellite information, which may be more accurate than estimating a rotation matrix and a translation vector for the current frame based on feature points between a reference frame (e.g., a predicted frame) and the current frame. Increasing the accuracy of the motion compensation may increase the accuracy of the motion-compensated predicted frame, which may reduce the residual coded for the current frame and increase coding efficiency.

[0006] In one example, this disclosure describes a device for encoding point cloud data, the device comprising: a memory for storing the point cloud data; and one or more processors coupled to the memory and implemented in circuitry. The one or more processors are configured to identify a first set of global motion parameters from global positioning system information. The one or more processors are further configured to determine, based on the first set of global motion parameters, a second set of global motion parameters to be used for global motion estimation for a current frame, and apply motion compensation to a reference frame based on the second set of global motion parameters to generate a global motion-compensated frame for the current frame.

[0007] In another example, this disclosure describes a method for encoding point cloud data that includes identifying, by one or more processors, a first set of global motion parameters from global positioning system information, and determining, by the one or more processors, a second set of global motion parameters to be used for global motion estimation for a current frame based on the first set of global motion parameters, The method further includes applying, by the one or more processors, motion compensation to a reference frame based on the second set of global motion parameters to generate a global motion compensated frame for the current frame.

[0008] In another example, this disclosure describes a computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to identify a first set of global motion parameters from global positioning system information and determine, based on the first set of global motion parameters, a second set of global motion parameters to be used for global motion estimation for a current frame. The instructions further cause the one or more processors to apply motion compensation to a reference frame based on the second set of global motion parameters to generate a global motion-compensated frame for the current frame.

[0009] In another example, this disclosure describes a device for processing point cloud data, the device comprising: at least one means for identifying a first set of global motion parameters from global positioning system information; and means for determining a second set of global motion parameters to be used for global motion estimation for a current frame based on the first set of global motion parameters. The device further comprises means for applying motion compensation to a reference frame based on the second set of global motion parameters to generate a global motion compensated frame for the current frame.

[0010] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will become apparent from the description, drawings, and claims. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram illustrating an example encoding and decoding system that may implement techniques of this disclosure. [Figure 2] FIG. 2 is a block diagram illustrating an example geometry point cloud compression (G-PCC) encoder in accordance with techniques of this disclosure. [Figure 3]FIG. 2 is a block diagram illustrating an example G-PCC decoder in accordance with techniques of this disclosure. [Figure 4] FIG. 1 is a block diagram illustrating an example motion estimation flowchart in accordance with techniques of this disclosure. [Figure 5] FIG. 2 is a block diagram illustrating an example algorithm for estimating global motion, in accordance with techniques of this disclosure. [Figure 6] FIG. 10 is a block diagram illustrating an example algorithm for estimating local node motion vectors, in accordance with techniques of this disclosure. [Figure 7] FIG. 1 is a conceptual diagram showing the geodetic latitude and longitude of a point measured on an approximating ellipsoid of the Earth. [Figure 8] 1 is a conceptual diagram showing the ECEF (Earth-Centered, Earth-Fixed) coordinate system (X, Y, Z axes) relative to the equator and prime meridian (0° latitude and longitude). [Figure 9] 1 is a flowchart illustrating an exemplary encoding process in accordance with techniques of this disclosure. [Figure 10] 10 is a flowchart illustrating an example decoding process in accordance with techniques of this disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0012] A geometry-based point cloud compression (G-PCC) coder (e.g., a G-PCC encoder or a G-PCC decoder) may be configured to apply motion compensation to a reference frame to generate a motion-compensated frame. For example, the G-PCC encoder may apply "global" motion compensation to the reference frame to accommodate rotation of the entire reference frame and / or translation of the entire reference frame. In this example, the G-PCC encoder may apply "local" motion compensation to accommodate rotation and / or translation at a finer scale than global motion compensation. For example, the G-PCC encoder may apply local node motion compensation to one or more nodes (e.g., portions of a frame) of the global motion-compensated frame.

[0013] Some systems may estimate a rotation matrix and a translation vector for a current frame based on feature points between a reference frame (e.g., a predicted frame) and the current frame. For example, a G-PCC encoder may estimate a motion matrix and a translation vector to "match" feature points between the reference frame and the current frame. However, the detected feature points may be unreliable, which may result in an incorrect estimation of the rotation matrix and / or translation vector, and may introduce additional distortion instead of compensating for global motion. This additional distortion may reduce the coding efficiency of G-PCC encoders and decoders that use rotation matrices and translation vectors.

[0014] According to the techniques of this disclosure, a G-PCC encoder may be configured to apply global motion compensation based on global positioning system information. For example, the G-PCC encoder may identify a first set of global motion parameters. The first set of global motion parameters may include orientation parameters (e.g., roll, pitch, yaw, or angular velocity) and / or position parameters (e.g., displacement or velocity along the x-, y-, or z-dimensions). In this example, the G-PCC encoder may determine a second set of global motion parameters based on the first set of global motion parameters. For example, the G-PCC encoder may convert the orientation parameters and / or position parameters into a rotation matrix and a translation vector for the current frame. In this manner, the G-PCC coder (e.g., a G-PCC encoder or a G-PCC decoder) may apply global motion compensation using satellite information, which may be more accurate than estimating a rotation matrix and a translation vector for the current frame based on feature points between a reference frame (e.g., a predicted frame) and the current frame. Increasing the accuracy of motion compensation may increase the accuracy of the motion compensated predicted frame, which may increase coding efficiency by reducing the residual coded for the current frame.

[0015] 1 is a block diagram illustrating an example encoding and decoding system 100 that may implement the techniques of this disclosure. The techniques of this disclosure are generally directed to coding (encoding and / or decoding) point cloud data, i.e., supporting point cloud compression. In general, point cloud data includes any data for processing a point cloud. Coding may be effective to compress and / or decompress the point cloud data.

[0016] 1, the system 100 includes a source device 102 and a destination device 116. The source device 102 provides encoded point cloud data to be decoded by the destination device 116. In particular, in the example of FIG. 1, the source device 102 provides the point cloud data to the destination device 116 via a computer-readable medium 110. The source device 102 and the destination device 116 may comprise any of a wide range of devices, including a desktop computer, a notebook (i.e., laptop) computer, a tablet computer, a set-top box, a telephone handset such as a smartphone, a television, a camera, a display device, a digital media player, a video gaming console, a video streaming device, a land or sea vehicle, a spacecraft, an aircraft, a robot, a LIDAR device, a satellite, etc. In some cases, the source device 102 and the destination device 116 may be capable of wireless communication.

[0017] In the example of FIG. 1 , source device 102 includes a data source 104, memory 106, a G-PCC encoder 200, and an output interface 108. Destination device 116 includes an input interface 122, a G-PCC decoder 300, memory 120, and a data consumer 118. According to this disclosure, G-PCC encoder 200 of source device 102 and G-PCC decoder 300 of destination device 116 may be configured to apply techniques of this disclosure related to techniques for improving visualization of point cloud frames. Thus, source device 102 represents an example of an encoding device, and destination device 116 represents an example of a decoding device. In other examples, source device 102 and destination device 116 may include other components or arrangements. For example, source device 102 may receive data (e.g., point cloud data) from an internal or external source. Similarly, destination device 116 may interface with an external data consumer rather than including the data consumer within the same device.

[0018] The system 100 as shown in FIG. 1 is merely an example. Generally, other digital encoding and / or decoding devices may perform techniques for improving visualization of point cloud frames. The source device 102 and the destination device 116 are merely examples of devices in which the source device 102 generates coded data for transmission to the destination device 116. This disclosure refers to devices that perform coding (encoding and / or decoding) of data as “coding” devices. Accordingly, the G-PCC encoder 200 and the G-PCC decoder 300 represent examples of coding devices, specifically, encoders and decoders, respectively. In some examples, the source device 102 and the destination device 116 may operate in a substantially symmetrical manner, such that each of the source device 102 and the destination device 116 includes encoding and decoding components. Thus, the system 100 may support unidirectional or bidirectional transmission between the source device 102 and the destination device 116, for example, for streaming, playback, broadcasting, telephony, navigation, and other applications.

[0019] Generally, the data source 104 represents a source of data (i.e., raw, unencoded point cloud data) and may provide a sequential series of “frames” of data to the G-PCC encoder 200, which encodes the data for the frames. The data source 104 of the source device 102 may include a point cloud capture device, such as any of a variety of cameras or sensors, e.g., a 3D scanner or light detection and ranging (LIDAR) device, one or more video cameras, an archive containing previously captured data, and / or a data feed interface for receiving data from a data content provider. Alternatively or additionally, the point cloud data may be computer-generated from scanners, cameras, sensors, or other data. For example, the data source 104 may generate computer-graphics-based data as source data or may result in a combination of live data, archived data, and computer-generated data. That is, the data source 104 may generate the point cloud data. In each case, the G-PCC encoder 200 encodes the captured, pre-captured, or computer-generated data. The G-PCC encoder 200 may reorder frames from the order in which they were received (sometimes referred to as "display order") into a coding order for coding. The G-PCC encoder 200 may generate one or more bitstreams including the encoded data. The source device 102 may then output the encoded data onto a computer-readable medium 110 via an output interface 108, for receipt and / or retrieval by, for example, an input interface 122 of a destination device 116.

[0020] The memory 106 of the source device 102 and the memory 120 of the destination device 116 may represent general-purpose memory. In some examples, the memory 106 and the memory 120 may store raw data, e.g., raw data from the data source 104 and raw decoded data from the G-PCC decoder 300. Additionally or alternatively, the memory 106 and the memory 120 may store software instructions executable by the G-PCC encoder 200 and the G-PCC decoder 300, respectively. While the memory 106 and the memory 120 are shown separate from the G-PCC encoder 200 and the G-PCC decoder 300 in this example, it should be understood that the G-PCC encoder 200 and the G-PCC decoder 300 may also include internal memory for functionally similar or equivalent purposes. Additionally, the memory 106 and the memory 120 may store encoded data, e.g., output from the G-PCC encoder 200 and input to the G-PCC decoder 300. In some examples, portions of memory 106 and memory 120 may be allocated as one or more buffers, e.g., for storing raw decoded and / or encoded data. For example, memory 106 and memory 120 may store data representing point clouds.

[0021] The computer-readable medium 110 may represent any type of medium or device capable of transporting encoded data from the source device 102 to the destination device 116. In one example, the computer-readable medium 110 represents a communication medium that enables the source device 102 to transmit encoded data directly to the destination device 116 in real time, for example, via a radio frequency network or a computer-based network. The output interface 108 may modulate a transmission signal containing the encoded data, and the input interface 122 may demodulate a received transmission signal, in accordance with a communication standard such as a wireless communication protocol. The communication medium may comprise any wireless or wired communication medium, such as the radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful for facilitating communication from the source device 102 to the destination device 116.

[0022] In some examples, source device 102 may output the encoded data from output interface 108 to storage device 112. Similarly, destination device 116 may access the encoded data from storage device 112 via input interface 122. Storage device 112 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, Blu-ray disc, DVD, CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded data.

[0023] In some examples, source device 102 may output the encoded data to file server 114 or another intermediate storage device, which may store the encoded data generated by source device 102. Destination device 116 may access the stored data from file server 114 via streaming or download. File server 114 may be any type of server device capable of storing encoded data and transmitting the encoded data to destination device 116. File server 114 may represent a web server (e.g., for a website), a file transfer protocol (FTP) server, a content delivery network device, or a network-attached storage (NAS) device. Destination device 116 may access the encoded data from file server 114 through any standard data connection, including an Internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both suitable for accessing encoded data stored on file server 114. File server 114 and input interface 122 may be configured to operate according to a streaming transmission protocol, a download transmission protocol, or a combination thereof.

[0024] Output interface 108 and input interface 122 may represent a wireless transmitter / receiver, a modem, a wired networking component (e.g., an Ethernet card), a wireless communication component operating according to any of the various IEEE 802.11 standards, or other physical components. In examples in which output interface 108 and input interface 122 comprise wireless components, output interface 108 and input interface 122 may be configured to transfer data, such as encoded data, according to a cellular communication standard, such as 4G, 4G-LTE (Long Term Evolution), LTE-Advanced, 5G, etc. In some examples in which output interface 108 comprises a wireless transmitter, output interface 108 and input interface 122 may be configured to transfer data, such as encoded data, according to other wireless standards, such as the IEEE 802.11 specification, the IEEE 802.15 specification (e.g., ZigBee™), the Bluetooth™ standard, etc. In some examples, source device 102 and / or destination device 116 may include respective system-on-chip (SoC) devices. For example, the source device 102 may include an SoC device for implementing functionality attributed to the G-PCC encoder 200 and / or the output interface 108, and the destination device 116 may include an SoC device for implementing functionality attributed to the G-PCC decoder 300 and / or the input interface 122.

[0025] The techniques of this disclosure may be applied to encoding and decoding in support of any of a variety of applications, such as communication between autonomous vehicles, communication between processing devices such as scanners, cameras, sensors and local or remote servers, geographic mapping, or other applications.

[0026] The input interface 122 of the destination device 116 receives the encoded bitstream from the computer-readable medium 110 (e.g., a communication medium, a storage device 112, a file server 114, etc.). The encoded bitstream may include signaling information defined by the G-PCC encoder 200 that is also used by the G-PCC decoder 300, such as syntax elements having values ​​that describe the characteristics and / or processing of a coded unit (e.g., a slice, a picture, a group of pictures, a sequence, etc.). The data consumer 118 uses the decoded data. For example, the data consumer 118 may use the decoded data to determine the location of a physical object. In some examples, the data consumer 118 may include a display for presenting imagery based on the point cloud.

[0027] The G-PCC encoder 200 and the G-PCC decoder 300 may each be implemented as any of a variety of suitable encoder and / or decoder circuit configurations, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the techniques are implemented partially in software, a device may store instructions for the software on a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to implement the techniques of this disclosure. The G-PCC encoder 200 and the G-PCC decoder 300 may each be included in one or more encoders or decoders, any of which may be integrated as part of a combined encoder / decoder (codec) within the respective device. A device including the G-PCC encoder 200 and / or the G-PCC decoder 300 may comprise one or more integrated circuits, microprocessors, and / or other types of devices.

[0028] The G-PCC encoder 200 and the G-PCC decoder 300 may operate according to a coding standard such as the Video Point Cloud Compression (V-PCC) standard or the Geometry Point Cloud Compression (G-PCC) standard. This disclosure may generally refer to coding (e.g., encoding and decoding) of pictures to include the processes of encoding or decoding data. An encoded bitstream generally includes a series of values ​​for syntax elements that represent coding decisions (e.g., coding modes).

[0029] This disclosure generally refers to “signaling” certain information, such as syntax elements. The term “signaling” may generally refer to communication of values ​​for syntax elements and / or other data used to decode encoded data. That is, G-PCC encoder 200 may signal values ​​for syntax elements in a bitstream. Generally, signaling refers to generating values ​​in a bitstream. As mentioned above, source device 102 may transport the bitstream to destination device 116 substantially in real time or not in real time, such as may occur when storing syntax elements to storage device 112 for later retrieval by destination device 116.

[0030] ISO / IEC MPEG (JTC1 / SC29 / WG11) is investigating the potential need for, and aims to develop, a standard for, point cloud coding techniques with compression capabilities significantly beyond those of current methods. The group is working together on this research in a collaborative effort known as the 3-Dimensional Graphics Team (3DG) to evaluate compression design proposals by experts in the field.

[0031] Point cloud compression activities are categorized into two different approaches. The first approach is "video point cloud compression" (V-PCC), which segments a 3D object and projects the segments into multiple 2D planes (represented as "patches" in a 2D frame), which are further coded by a legacy 2D video codec such as the High Efficiency Video Coding (HEVC) (ITU-T H.265) codec. The second approach is "geometry-based point cloud compression" (G-PCC), which directly compresses the 3D geometry, i.e., the locations of a set of points in 3D space and the associated attribute values ​​(for each point associated with the 3D geometry). G-PCC addresses the compression of point clouds in both category 1 (static point clouds) and category 3 (dynamically acquired point clouds). The latest draft of the G-PCC standard is available at G-PCC DIS, ISO / IEC JTC1 / SC29 / WG11 w19088, Brussels, Belgium, January 2020, and the codec description is available at G-PCC Codec Description v6, ISO / IEC JTC1 / SC29 / WG11 w19091, Brussels, Belgium, January 2020.

[0032] A point cloud includes a set of points in 3D space and may have attributes associated with the points. The attributes may be color information such as R, G, B or Y, Cb, Cr, or reflectance information, or other attributes. Point clouds may be captured by various cameras or sensors, such as LIDAR sensors and 3D scanners, or may be computer-generated. That is, source device 102 may generate point cloud data based on signals from a LIDAR device (e.g., a LIDAR sensor and / or LIDAR device). Point cloud data is used in various applications, including, but not limited to, construction (modeling), graphics (3D models for visualization and animation), and the automotive industry (LIDAR sensors used to aid navigation).

[0033] The 3D space occupied by the point cloud data may be enclosed by a virtual bounding box. The positions of points within the bounding box may be represented with a certain precision, and therefore, the positions of one or more points may be quantized based on the precision. At the smallest level, the bounding box is divided into voxels, which are the smallest units of space, represented by a unit cube. A voxel in a bounding box may be associated with zero, one, or multiple points. The bounding box may be divided into multiple cubic / rectangular regions, which may be called tiles. Each tile may be coded into one or more slices. The division of the bounding box into slices and tiles may be based on the number of points in each division or other considerations (e.g., a particular region may be coded as a tile). The slice regions may be further divided using division decisions similar to those in video codecs.

[0034] According to the techniques of this disclosure, the G-PCC encoder 200 may be configured to apply global motion compensation based on global positioning system information. For example, the G-PCC encoder 200 may identify a first set of global motion parameters. The first set of global motion parameters may include orientation parameters (e.g., roll, pitch, yaw, or angular velocity) and / or position parameters (e.g., displacement or velocity along the x-, y-, or z-dimensions). In this example, the G-PCC encoder 200 may determine a second set of global motion parameters based on the first set of global motion parameters. For example, the G-PCC encoder 200 may convert the orientation parameters and / or position parameters into a rotation matrix and a translation vector for the current frame. In this manner, the G-PCC encoder 200 may apply global motion compensation using satellite information, which may be more accurate than estimating a rotation matrix and a translation vector for the current frame based on feature points between a reference frame (e.g., a predicted frame) and the current frame.

[0035] The G-PCC encoder 200 may signal a first set of global motion parameters (e.g., orientation parameters and / or position parameters). In some examples, the G-PCC encoder 200 may signal a second set of global motion parameters (e.g., a rotation matrix and a translation vector for the current frame). While examples describe signaling a rotation matrix and a translation vector for the current frame to signal the second set of global motion parameters, in some examples, the G-PCC encoder 200 may signal portions and / or estimates of the rotation matrix and translation vector for the current frame. For example, the G-PCC encoder 200 may signal a roll difference between the current frame and a reference frame, a pitch difference between the current frame and a reference frame, a yaw difference between the current frame and a reference frame, and a translation vector indicating an average velocity of the current frame. In some examples, the G-PCC encoder 200 may signal a second set of global motion parameters that includes a translation vector indicating only an average velocity of the current frame. In some examples, the G-PCC encoder 200 may signal only a second set of global motion parameters that includes only a translation vector with a magnitude indicating the average velocity of the current frame. In this way, the G-PCC encoder 200 may apply global motion compensation using satellite information, which may be more accurate than estimating a rotation matrix and translation vector for the current frame based on feature points between a reference frame (e.g., a predicted frame) and the current frame. Increasing the accuracy of the motion compensation may increase the accuracy of the motion-compensated predicted frame, which may reduce the residual coded for the current frame and increase coding efficiency.

[0036] Similarly, the G-PCC decoder 300 may apply global motion compensation based on global positioning system information. For example, the G-PCC decoder 300 may decode global motion information (e.g., a first set of global motion parameters and / or a second set of global motion parameters) from the bitstream. In this example, the G-PCC decoder 300 may determine global motion for the current frame based on the global motion information. For example, the G-PCC decoder 300 may apply global motion compensation based on a rotation matrix and a translation vector for the current frame.

[0037] The G-PCC decoder 300 may decode the rotation matrix and translation vectors directly from the bitstream, or may estimate and / or derive the rotation matrix and translation vectors from global motion information signaled in the bitstream. For example, the G-PCC encoder 300 may decode from the bitstream a roll difference between the current frame and a reference frame, a pitch difference between the current frame and a reference frame, a yaw difference between the current frame and a reference frame, and a translation vector indicating an average velocity of the current frame. In this example, the G-PCC decoder 300 may determine the rotation matrix and translation vectors based on the roll difference, pitch difference, yaw difference, and translation vector. In some examples, the G-PCC decoder 300 may decode the translation vectors from the bitstream and approximate the rotation matrix as an identity matrix. In some examples, the G-PCC decoder 300 may decode from the bitstream a translation vector of magnitude indicating an average velocity of the current frame, approximate the translation vector based on the received magnitude, and approximate the rotation matrix as an identity matrix. In this way, the G-PCC decoder 300 may apply global motion compensation using satellite information, which may be more accurate than estimating a rotation matrix and translation vector for the current frame based on feature points between a reference frame (e.g., a predicted frame) and the current frame. Increasing the accuracy of the motion compensation may increase the accuracy of the motion-compensated predicted frame, which may reduce the residual coded for the current frame and increase coding efficiency.

[0038] Figure 2 provides an overview of the G-PCC encoder 200. Figure 3 provides an overview of the G-PCC decoder 300. The modules shown are logical and do not necessarily correspond one-to-one to the code implemented in the reference implementation of the G-PCC codec, i.e., the TMC13 test model software studied by ISO / IEC MPEG (JTC1 / SC29 / WG11).

[0039] In both the G-PCC encoder 200 and the G-PCC decoder 300, the point cloud location is coded first. The attribute coding depends on the decoded geometry. In Figures 2 and 3, the gray shaded module is the commonly used option for Category 1 data. The diagonal crosshatched module is the commonly used option for Category 3 data. All other modules are common between Categories 1 and 3.

[0040] For Category 3 data, the compressed geometry is typically represented as an octree spanning from the root to the leaf level of individual voxels. For Category 1 data, the compressed geometry is typically represented by a pruned octree (i.e., an octree from the root to the leaf level of blocks larger than a voxel) plus a model that approximates the surface within each leaf of the pruned octree. In this way, both Category 1 and 3 data share the octree coding mechanism, and Category 1 data may further approximate the voxels within each leaf with a surface model. The surface model used is a triangulation involving 1 to 10 triangles per block, resulting in a triangle soup. Category 1 geometry codecs are therefore known as Trisoup geometry codecs, and Category 3 geometry codecs are known as Octree geometry codecs.

[0041] At each node in the octree, occupancy is signaled (when not inferred) for one or more of its child nodes (up to eight nodes). Multiple neighborhoods are specified, including (a) nodes that share a face with the current octree node, and (b) nodes that share a face, edge, or vertex with the current octree node. Within each neighborhood, the occupancy of the node and / or its children may be used to predict the occupancy of the current node or its children. For sparse points in some nodes of the octree, the codec also supports a direct coding mode, in which the 3D position of the point is directly coded. A flag may be signaled to indicate that direct mode is signaled. At the lowest level, the number of points associated with an octree node / leaf node may also be coded.

[0042] When geometry is coded, attributes corresponding to the geometry points are coded. When there are multiple attribute points corresponding to one reconstructed / decoded geometry point, an attribute value representing the reconstruction point may be derived.

[0043] G-PCC has three attribute coding methods: region-adaptive hierarchical transform (RAHT) coding, interpolation-based hierarchical nearest neighbor prediction (prediction transform), and interpolation-based hierarchical nearest neighbor prediction with update / lifting steps (lifting transform). RAHT and lifting are typically used for category 1 data, and prediction is typically used for category 3 data. However, either method can be used for any data; just like with geometry codecs in G-PCC, the attribute coding method used to code the point cloud is specified in the bitstream.

[0044] The coding of attributes may be performed at a level of detail (LOD), with each level of detail being used to obtain a more precise representation of the point cloud attributes, which may be specified based on a distance metric from neighboring nodes or based on a sampling distance.

[0045] In the G-PCC encoder 200, the residual obtained as the output of the attribute-directed coding method is quantized. The residual may be obtained by subtracting the attribute value from a prediction derived based on points in the neighborhood of the current point and based on the attribute values ​​of points already coded. The quantized residual may be coded using context-adaptive arithmetic coding.

[0046] In the example of FIG. 2, the G-PCC encoder 200 may include a coordinate transformation unit 202, a color transformation unit 204, a voxelization unit 206, an attribute transfer unit 208, an octree analysis unit 210, a surface approximation analysis unit 212, an arithmetic coding unit 214, a geometry reconstruction unit 216, a RAHT unit 218, an LOD generation unit 220, a lifting unit 222, a coefficient quantization unit 224, and an arithmetic coding unit 226.

[0047] As shown in the example of FIG. 2, the G-PCC encoder 200 may obtain a set of locations and a set of attributes for points in a point cloud. The G-PCC encoder 200 may obtain the set of locations and the set of attributes for points in the point cloud from the data source 104 (FIG. 1). The locations may include coordinates of the points in the point cloud. The attributes may include information about the points in the point cloud, such as a color associated with the points in the point cloud. The G-PCC encoder 200 may generate a geometry bitstream 203 that includes an encoded representation of the locations of the points in the point cloud. The G-PCC encoder 200 may also generate an attribute bitstream 205 that includes an encoded representation of the set of attributes.

[0048] The coordinate transformation unit 202 may apply a transform to the coordinates of the points to convert the coordinates from an initial domain to a transformation domain. In this disclosure, the transformed coordinates may be referred to as transformed coordinates. The color transformation unit 204 may apply a transform to convert color information of the attributes to a different domain. For example, the color transformation unit 204 may convert color information from an RGB color space to a YCbCr color space.

[0049] 2, voxelization unit 206 may voxelize the transformed coordinates. Voxelizing the transformed coordinates may include quantizing and removing some points of the point cloud. In other words, multiple points of the point cloud may be contained within a single "voxel," which may then be treated as one point in some respects.

[0050] The octree analysis unit 210 may generate an octree based on the voxelized transformation coordinates. According to techniques of this disclosure, the octree analysis unit encoder 210 may be configured to apply global motion compensation based on global positioning system information. For example, the octree analysis unit 210 may identify a first set of global motion parameters. The first set of global motion parameters may include orientation parameters (e.g., roll, pitch, yaw, or angular velocity) and / or position parameters (e.g., displacement or velocity along the x-, y-, or z-dimension). In this example, the octree analysis unit 210 may determine a second set of global motion parameters based on the first set of global motion parameters. For example, the octree analysis unit 210 may convert the orientation parameters and / or position parameters into a rotation matrix and a translation vector for the current frame. In this way, the octree analysis unit 210 may apply global motion compensation using satellite information, which may be more accurate than estimating a rotation matrix and translation vector for the current frame based on feature points between a reference frame (e.g., a predicted frame) and the current frame.

[0051] 2, surface approximation analysis unit 212 may analyze the points to determine a surface representation of the set of points. Arithmetic coding unit 214 may entropy code syntax elements representing the octree and / or surface information determined by surface approximation analysis unit 212. G-PCC encoder 200 may output these syntax elements in geometry bitstream 203. Geometry bitstream 203 may also include other syntax elements, including syntax elements that are not arithmetically coded.

[0052] The geometry reconstruction unit 216 may reconstruct transformation coordinates of points in the point cloud based on the octree, the data indicative of the surface determined by the surface approximation analysis unit 212, and / or other information. The number of transformation coordinates reconstructed by the geometry reconstruction unit 216 may differ from the original number of points in the point cloud due to voxelization and surface approximation. In this disclosure, the resulting points may be referred to as reconstructed points. The attribute transfer unit 208 may transfer attributes of the original points of the point cloud to the reconstructed points of the point cloud.

[0053] Further, the RAHT unit 218 may apply RAHT coding to the attributes of the reconstruction points. In some examples, under RAHT, the attributes of a block at a 2x2x2 point position are obtained and transformed along one direction to obtain four low-frequency nodes (L) and four high-frequency nodes (H). Then, the four low-frequency nodes (L) are transformed in a second direction to obtain two low-frequency nodes (LL) and two high-frequency nodes (LH). The two low-frequency nodes (LL) are transformed in a third direction to obtain one low-frequency node (LLL) and one high-frequency node (LLH). The low-frequency node LLL corresponds to the DC coefficient, and the high-frequency nodes H, LH, and LLH correspond to the AC coefficients. The transformation in each direction may be a 1-D transformation with two coefficient weights. The low-frequency coefficients are considered to be coefficients of a 2x2x2 block for the next higher level of the RAHT transformation, and the AC coefficients are coded without modification, and such transformation continues up to the top-level root node. The tree traversal for encoding is a top-to-bottom traversal used to calculate the weights used for the coefficients, and the transform order is bottom-to-top. The coefficients may then be quantized and coded.

[0054] Alternatively or additionally, the LOD generation unit 220 and the lifting unit 222 may apply LOD processing and lifting, respectively, to the attributes of the reconstruction points. The lifting unit 222 may be configured to perform interpretation-based hierarchical nearest neighbor prediction with update / lifting steps (lifting transformations). In some examples, the lifting unit 222 may be configured to perform global motion compensation.

[0055] The LOD generation unit 220 may be used to divide attributes into different refinement levels. Each refinement level refines the attributes of the point cloud. The first refinement level provides a coarse approximation and includes a small number of points. Subsequent refinement levels generally include more points, and so on. The refinement levels may be constructed using a distance-based metric or may use one or more other classification criteria (e.g., subsampling from a particular rank). Thus, a refinement level may include all reconstructed points. Each level of detail is generated by taking the union of all points up to a particular refinement level; for example, LOD1 is obtained based on refinement level RL1, LOD2 is obtained based on RL1 and RL2, and LOD1 is obtained by the union of RL1, RL2, ..., RLN. In some cases, a prediction scheme (e.g., predictive transformation) is performed after LOD generation, where the attributes associated with each point in the LOD are predicted from a weighted average of previous points, and the residual is quantized and entropy coded. A lifting scheme is built on the predictive transformation mechanism, where coefficients are updated using an update operator and adaptive quantization of the coefficients is performed.

[0056] The RAHT unit 218 and the lifting unit 222 may generate coefficients based on the attributes. The coefficient quantization unit 224 may quantize the coefficients generated by the RAHT unit 218 or the lifting unit 222. The arithmetic coding unit 226 may apply arithmetic coding to syntax elements representing the quantized coefficients. The G-PCC encoder 200 may output these syntax elements in the attribute bitstream 205. The attribute bitstream 205 may include other syntax elements, including non-arithmetically coded syntax elements.

[0057] The arithmetic coding unit 214 may signal a first set of global motion parameters (e.g., orientation parameters and / or position parameters). In some examples, the arithmetic coding unit 214 may signal a second set of global motion parameters (e.g., a rotation matrix and a translation vector for the current frame) in the geometry bitstream 203. Although examples describe signaling a rotation matrix and a translation vector for the current frame to signal the second set of global motion parameters, in some examples, the arithmetic coding unit 214 may signal portions and / or estimates of the rotation matrix and translation vector for the current frame in the geometry bitstream 203. For example, the arithmetic coding unit 214 may signal a roll difference between the current frame and a reference frame, a pitch difference between the current frame and a reference frame, a yaw difference between the current frame and a reference frame, and a translation vector indicating an average velocity of the current frame. In some examples, the arithmetic coding unit 214 may signal a second set of global motion parameters that includes a translation vector indicating only an average velocity of the current frame. In some examples, the arithmetic coding unit 214 may signal only the second set of global motion parameters, which includes only a translation vector with a magnitude indicating the average velocity of the current frame. In this way, the arithmetic coding unit 214 may apply global motion compensation using satellite information, which may be more accurate than estimating a rotation matrix and translation vector for the current frame based on feature points between a reference frame (e.g., a predicted frame) and the current frame.

[0058] In the example of FIG. 3, the G-PCC decoder 300 may include a geometry arithmetic decoding unit 302, an attribute arithmetic decoding unit 304, an octree synthesis unit 306, an inverse quantization unit 308, a surface approximation synthesis unit 310, a geometry reconstruction unit 312, a RAHT unit 314, an LOD generation unit 316, an inverse lifting unit 318, an inverse transform coordinate unit 320, and an inverse transform color unit 322.

[0059] The G-PCC decoder 300 may obtain the geometry bitstream 203 and the attribute bitstream 205. The geometry arithmetic decoding unit 302 of the decoder 300 may apply arithmetic decoding (e.g., context-adaptive binary arithmetic coding (CABAC) or other types of arithmetic decoding) to the syntax elements in the geometry bitstream 203. Similarly, the attribute arithmetic decoding unit 304 may apply arithmetic decoding to the syntax elements in the attribute bitstream 205.

[0060] For example, the geometry arithmetic decoding unit 302 may receive global motion information (e.g., a first set of global motion parameters and / or a second set of global motion parameters) from the geometry bitstream 203. For example, the geometry arithmetic decoding unit 302 may receive the second set of global motion parameters (e.g., a rotation matrix and a translation vector for the current frame). Although examples describe receiving a rotation matrix and a translation vector for the current frame to signal the second set of global motion parameters, in some examples, the geometry arithmetic decoding unit 302 may receive portions and / or estimates of the rotation matrix and the translation vector for the current frame. For example, the geometry arithmetic decoding unit 302 may receive a roll difference between the current frame and a reference frame, a pitch difference between the current frame and a reference frame, a yaw difference between the current frame and a reference frame, and a translation vector indicating an average velocity of the current frame. In some examples, the geometry arithmetic decoding unit 302 may receive the second set of global motion parameters including only a translation vector indicating an average velocity of the current frame. In some examples, the geometry arithmetic coding unit 302 may receive a second set of global motion parameters that includes only a translation vector with a magnitude that indicates the average velocity of the current frame.

[0061] The octree synthesis unit 306 may synthesize an octree based on syntax elements parsed from the geometry bitstream 203. The occupancy of each of the eight child nodes at each octree level is signaled in the bitstream, starting from the root node of the octree. When the signaling indicates that a child node at a particular octree level is occupied, the occupancy of the child node's children is signaled. Signaling of nodes at each octree level occurs before proceeding to subsequent octree levels. At the final level of the octree, each node corresponds to a voxel position, and an occupied leaf node may specify that one or more points are occupied at the voxel position. In some instances, due to quantization, some branches of the octree may terminate before the final level. In such cases, the leaf node is considered an occupied node with no child nodes. In cases where surface approximations are used in the geometry bitstream 203, the surface approximation synthesis unit 310 may determine the surface model based on syntax elements parsed from the geometry bitstream 203 and based on the octree.

[0062] The octree synthesis unit 306 may determine global motion for the current frame based on the global motion information. For example, the octree synthesis unit 306 may apply global motion compensation based on a rotation matrix and a translation vector for the current frame. Again, the geometry arithmetic decoding unit 302 may receive the rotation matrix and the translation vector from the geometry bitstream 203. The octree synthesis unit 306 may determine or estimate a second set of global motion parameters (e.g., a rotation matrix and a translation vector) based on a portion of the first set of global motion parameters and / or the second global motion parameters.

[0063] For example, the octree synthesis unit 306 may decode a roll difference between the current frame and the reference frame, a pitch difference between the current frame and the reference frame, a yaw difference between the current frame and the reference frame, and a translation vector indicating the average velocity of the current frame from the geometry bitstream 203. In this example, the octree synthesis unit 306 may determine a rotation matrix and a translation vector based on the roll difference, the pitch difference, the yaw difference, and the translation vector.

[0064] In some examples, the octree synthesis unit 306 may decode translation vectors from the geometry bitstream 203 and approximate rotation matrices as identity matrices. In some examples, the octree synthesis unit 306 may decode translation vectors of magnitude indicating the average velocity of the current frame from the geometry bitstream 203, approximate translation vectors based on the received magnitudes, and approximate rotation matrices as identity matrices. In this way, the octree synthesis unit 306 may apply global motion compensation using satellite information, which may be more accurate than estimating rotation matrices and translation vectors for the current frame based on feature points between a reference frame (e.g., a predicted frame) and the current frame.

[0065] The octree synthesis unit 306 may convert the orientation parameters and / or position parameters into a rotation matrix and a translation vector for the current frame. In this way, the octree synthesis unit 306 may apply global motion compensation using satellite information, which may be more accurate than estimating a rotation matrix and a translation vector for the current frame based on feature points between a reference frame (e.g., a predicted frame) and the current frame.

[0066] Furthermore, the geometry reconstruction unit 312 may perform reconstruction to determine the coordinates of points in the point cloud. For each position in a leaf node of the octree, the geometry reconstruction unit 312 may reconstruct the node position by using the binary representation of the leaf node in the octree. At each respective leaf node, the number of points at the respective leaf node is signaled, which indicates the number of overlapping points at the same voxel position. Using geometry quantization, the point positions are scaled to determine the reconstructed point position value.

[0067] The inverse coordinate transformation unit 320 may apply an inverse transform to the reconstructed coordinates (positions) of points in the point cloud to transform them from the transformed domain back to the original domain. Point positions in the point cloud may be in the floating-point domain, whereas point positions in the G-PCC codec are coded in the integer domain. An inverse transform may be used to transform the positions back to the original domain.

[0068] 3, the inverse quantization unit 308 may inverse quantize the attribute values, which may be based on syntax elements obtained from the attribute bitstream 205 (e.g., including syntax elements decoded by the attribute arithmetic decoding unit 304).

[0069] Depending on how the attribute values ​​are encoded, the RAHT unit 314 may perform RAHT coding to determine color values ​​for points of the point cloud based on the dequantized attribute values. RAHT decoding is performed from the top to the bottom of the tree. At each level, a configuration value is derived using the low-frequency and high-frequency coefficients derived from the dequantization process. At leaf nodes, the derived value corresponds to the attribute value of the coefficient. The weight derivation process for a point is similar to the process used in the G-PCC encoder 200. Alternatively, the LOD generation unit 316 and the inverse lifting unit 318 may use a level-of-detail-based technique to determine color values ​​for points of the point cloud. The LOD generation unit 316 decodes each LOD, which represents an increasingly finer representation of the point's attributes. When using predictive transformation, the LOD generation unit 316 may derive a predicted value for a point from a weighted sum of points in the previous LOD or previously reconstructed within the same LOD. The LOD generation unit 316 may add the predicted value to the residual (obtained after inverse quantization) to obtain a reconstructed value of the attribute. When using a lifting scheme, the LOD generation unit 316 may include an update operator for updating the coefficients used to derive the attribute value. The LOD generation unit 316 may also apply inverse adaptive quantization in this case.

[0070] 3, the inverse color transform unit 322 may apply an inverse color transform to the color values. The inverse color transform may be the inverse of the color transform applied by the color transform unit 204 of the G-PCC encoder 200. For example, the color transform unit 204 may convert the color information from the RGB color space to the YCbCr color space. Thus, the inverse color transform unit 322 may convert the color information from the YCbCr color space to the RGB color space.

[0071] The various units in FIGS. 2 and 3 are shown to aid in understanding the operations performed by G-PCC encoder 200 and G-PCC decoder 300. The units may be implemented as fixed-function circuits, programmable circuits, or a combination thereof. A fixed-function circuit refers to a circuit that provides specific functionality, and the operations that may be performed are pre-defined. A programmable circuit refers to a circuit that may be programmed to perform various tasks, and provides flexible functionality in the operations that may be performed. For example, a programmable circuit may execute software or firmware that causes the programmable circuit to operate in a manner defined by the software or firmware instructions. A fixed-function circuit may execute software instructions (e.g., to receive or output parameters), but the types of operations that the fixed-function circuit performs are generally invariant. In some examples, one or more of the units may be different circuit blocks (fixed function or programmable), and in some examples, one or more units may be integrated circuits.

[0072] In accordance with techniques of this disclosure, G-PCC encoder 200 may represent an example of a device including a memory for storing point cloud data and one or more processors coupled to the memory and implemented in circuitry. The one or more processors are configured to identify a first set of global motion parameters from global positioning system information and determine, based on the first set of global motion parameters, a second set of global motion parameters to be used for global motion estimation for the current frame. The one or more processors are further configured to apply motion compensation to the reference frame using the second set of global motion parameters to generate a global motion-compensated frame for the current frame.

[0073] The G-PCC decoder 300 may represent an example of a device comprising a memory for storing point cloud data and one or more processors implemented in circuitry coupled to the memory. The one or more processors may be configured to decode symbols of the bitstream indicating global motion information and, based on the global motion information, determine a second set of global motion parameters to be used for global motion estimation for the current frame. The one or more processors may be further configured to apply motion compensation to the reference frame using the set of global motion parameters to generate a global motion-compensated frame for the current frame.

[0074] There are two types of motion involved in the G-PCC technique: a global motion matrix and local nodal motion vectors. Global motion parameters may include a rotation matrix and a translation vector. A G-PCC coder (e.g., G-PCC encoder 200 or G-PCC decoder 300) may apply global motion compensation to all points in a reference frame (e.g., a predicted frame). A local nodal motion vector of a node in an octree is a motion vector that applies only to points within the node of a predicted (reference) frame. For example, a G-PCC coder may apply local nodal motion compensation to only a portion of a reference frame (e.g., a set of nodes).

[0075] FIG. 4 is a block diagram illustrating an example motion estimation flowchart in accordance with techniques of this disclosure. Given a predicted frame 402 and a current frame 410, the G-PCC encoder 200 may first estimate global motion at a global scale (404). For example, the G-PCC encoder 200 may generate a rotation matrix and a translation vector according to techniques described herein. After applying global motion to the predicted frame 402 (406), the G-PCC encoder 200 may apply local node motion estimation (408) to estimate local motion at finer scales and at the node level in the octree. The G-PCC encoder 200 may apply the estimated local node motion in motion compensation. For example, the G-PCC encoder 200 may apply motion estimation to encode motion vector information and point information (412).

[0076] Figure 5 is a block diagram showing an exemplary algorithm for estimating global motion. The process shown in Figure 5 may be an example of step 404 in Figure 4. The G-PCC encoder 200 may be configured to define a global motion matrix for matching feature points between a predicted (reference) frame and a current frame. The overall global motion estimation algorithm can be divided into three steps: a feature point finding step (502), a feature point pair sampling step (504), and a motion estimation step using a least mean squares (LMS) algorithm (506).

[0077] The G-PCC encoder 200 may perform an LMS algorithm (506) to define points that have large position changes between the predicted frame and the current frame as feature points. For each point in the current frame, the G-PCC encoder 200 may find the closest point in the predicted frame and construct point pairs between the current frame and the predicted frame (502). If the distance between the paired points is greater than a threshold, the G-PCC encoder 200 may consider the paired points to be feature points.

[0078] After finding the feature points, the G-PCC coder may perform sampling on the feature points to reduce the scale of the problem (504) (e.g., by selecting a subset of feature points to reduce the complexity of motion estimation). The G-PCC encoder 200 may then apply a least mean squares (LMS) algorithm to derive motion parameters by attempting to reduce the error between each feature point in the predicted frame and the current frame. This process may loop for each of the feature point pairs of step 502.

[0079] Figure 6 is a block diagram illustrating an exemplary algorithm for estimating local node motion vectors. The process illustrated in Figure 6 may be an example of step 408 in Figure 4. In the example of Figure 6, the G-PCC encoder 200 may recursively estimate motion vectors. The G-PCC encoder 200 may select an optimal motion vector based on a rate-distortion cost using a cost function.

[0080] If the current node is not split into eight children, the G-PCC encoder 200 may determine the motion vector that can minimize the cost between the current node 602 and the prediction node. If the current node is split into eight children (610), the G-PCC encoder 200 may apply a motion estimation algorithm to find the motion for each child node (612) and obtain the total cost under the split condition by adding the estimated cost values ​​of each child node (614). The G-PCC encoder 200 may reach a decision on whether to split by comparing the costs between splitting and not splitting. If the current node is split, the G-PCC encoder 200 may assign a respective motion vector to each subnode (or further split into children of the subnode). If the current node is not split, the G-PCC encoder 200 may find the motion vector that achieves the minimum cost (604) and assign the motion vector to the current node.

[0081] Two parameters that affect the performance of motion vector estimation are block size (BlockSize) and minimum prediction unit size (MinPUSize). BlockSize defines the upper limit of the node size for applying motion vector estimation, and MinPUSize defines the lower limit.

[0082] The G-PCC encoder 200 may generate motion matrices and translation vectors that "match" each feature point between the predicted (reference) frame and the current frame. The detected feature points may be unreliable, in which case the global motion parameter estimation will be incorrect, thus introducing additional distortion instead of compensating for global motion. This additional distortion may result in lower coding efficiency than when global motion is properly compensated.

[0083] In some systems, detected feature points in successive frames may not match. For example, the relative rotation and / or translation between each feature point in successive frames may be different. If a G-PCC coder (e.g., G-PCC encoder 200 or G-PCC decoder 300) matches each feature point in successive frames with each other, distortion may occur.

[0084] A problem with current motion estimation architectures is execution efficiency. The total execution time can be 25 times longer than the anchor version used as a reference. Of the additional total execution time, approximately half the time is typically spent on global motion estimation and approximately half is typically spent on local node motion estimation. Such long execution times can be impractical for some applications (e.g., real-time coding of point cloud compensation) and thus hinder motion estimation in such applications.

[0085] According to the techniques of this disclosure, a G-PCC coder (G-PCC encoder 200 or G-PCC decoder 300) may be configured to apply global motion compensation based on global positioning system information. According to the techniques of this disclosure, a G-PCC coder may be configured to apply motion compensation using one or more of the following techniques: 1) The G-PCC encoder 200 may identify a first set of global motion parameters from GPS (Global Positioning System) information. Global Positioning System, also referred to herein simply as "GPS," may refer to any satellite system, such as, for example, the Global Positioning System (GPS) implemented in the United States, the Global Navigation Satellite System (GLONASS), the Beidou Navigation Satellite System (BDS), Galileo, the Quasi-Zenith Satellite System (QZSS), the International Relations Navigation Satellite System (IRNSS), or another satellite system. a. The first set of global motion parameters may include a set of orientation parameters, such as roll, pitch, yaw, angular velocity, etc. That is, the G-PCC encoder 200 may identify a first set of global motion parameters that includes the set of orientation parameters. In some examples, the set of orientation parameters may include one or more of roll, pitch, yaw, or angular velocity for the current frame. b. In some examples, the three parameters of orientation may be specified based on a reference frame. Thus, the three parameters may be included in a first set of global motion parameters. For example, given the roll, pitch, and yaw for the last frame, the orientation parameters may include the differences in roll, pitch, and yaw between the current frame and the last frame. That is, G-PCC encoder 200 may identify a set of orientation parameters that includes one or more of the difference in roll between the current frame and the reference frame, the difference in pitch between the current frame and the reference frame, the difference in yaw between the current frame and the reference frame, or the difference in angular velocity between the current frame and the reference frame. c. The first set of global motion parameters may include a set of position parameters, e.g., displacement in the x, y, and z dimensions, or velocity (e.g., velocity x, velocity y, velocity z). That is, the G-PCC encoder 200 may identify a set of orientation parameters that includes the set of position parameters. In some examples, the set of position parameters may include one or more of a displacement for the current frame or an average velocity for the current frame. d. In some examples, the three position parameters may be specified based on a reference frame. That is, the G-PCC encoder 200 may identify a set of orientation parameters that includes the three position parameters based on the reference frame. Thus, the three parameters may be included in a first set of global motion parameters. For example, the position parameters may be calculated using an East-North-Up coordinate system. That is, the set of position parameters may include one or more of an east velocity, a north velocity, or an up velocity in the East-North-Up coordinate system. 2) The G-PCC coder (e.g., the G-PCC encoder 200 or the G-PCC decoder 300) may derive a second set of global motion parameters from the first set of global motion parameters used for global motion estimation. That is, the G-PCC coder may determine the second set of global motion parameters used for global motion estimation for the current frame based on the first set of global motion parameters. a. The second set of global motion parameters may include elements of a global motion matrix that may represent (e.g., describe) a rotation matrix and a translation vector. That is, the G-PCC coder (e.g., the G-PCC encoder 200 or the G-PCC decoder 300) may derive the second set of global motion parameters from the first set of global motion parameters, including a rotation matrix that indicates the yaw, pitch, and roll for the current frame, and a translation vector that indicates the average velocity for the current frame. b. The second set of global motion parameters can be calculated exactly or approximated from the first set of global motion parameters. 3) The G-PCC coder (e.g., the G-PCC encoder 200 or the G-PCC decoder 300) may apply motion compensation to the reference frame using the second set of global motion parameters to obtain a compensated frame, i.e., the G-PCC coder applies motion compensation to the reference frame using the second set of global motion parameters to generate a global motion compensated frame for the current frame. The aG-PCC coder may use the compensated frame as a reference for motion compensation for the current frame. b. In some examples, a G-PCC coder (eg, G-PCC encoder 200 or G-PCC decoder 300) may apply compensation by first applying a rotation and then a translation. c. In some examples, the G-PCC coder may apply compensation by first applying a translation and then a rotation. 4) The G-PCC encoder 200 may signal a second set of global motion parameters in the bitstream. The G-PCC encoder 200 may use the second set of global motion parameters to enable the G-PCC decoder 300 to estimate global motion and apply prediction or motion compensation. That is, the G-PCC encoder 200 may signal the second set of global motion parameters in the bitstream. For example, the G-PCC encoder 200 may signal a rotation matrix indicating the yaw, pitch, and roll for the current frame and a translation vector indicating the average velocity for the current frame. Similarly, the G-PCC decoder 300 may decode from the bitstream the rotation matrix indicating the yaw, pitch, and roll for the current frame and the translation vector indicating the average velocity for the current frame. a. In some examples, the G-PCC encoder 200 may signal a first set of global motion parameters in the bitstream. For example, the G-PCC decoder 300 may derive a second set of global motion parameters. That is, the G-PCC encoder 200 may signal the first set of global motion parameters in the bitstream. For example, the G-PCC encoder 200 may signal one or more of the set of orientation parameters or the set of translation parameters identified from the global positioning system information. Similarly, the G-PCC decoder 300 may decode one or more of the set of orientation parameters or the set of translation parameters identified from the global positioning system information from the bitstream. As described below, the G-PCC encoder 200 may encode portions and / or estimates of the first set of global motion parameters and / or the second set of global motion parameters. 5) The G-PCC coder (e.g., the G-PCC encoder 200 or the G-PCC decoder 300) may try to find an appropriate combination of global motion parameters for local node motion vector estimation to find a balanced trade-off between execution time and performance. That is, the G-PCC coder may restrict the block size for local motion to be equal to the minimum prediction unit size. A G-PCC coder (e.g., the G-PCC encoder 200 or the G-PCC decoder 300) may restrict the blockSize for local motion to be equal to minPuSize, which may help ensure that the execution time for local motion vector estimation is minimized to limit the impact on coding efficiency. 6) The G-PCC coder (eg, the G-PCC encoder 200 or the G-PCC decoder 300) may improve the global motion estimation algorithm by one or more of the following steps: a. First, the G-PCC coder (e.g., the G-PCC encoder 200 or the G-PCC decoder 300) may estimate an initial translation vector by minimizing the mean square error between the current frame and the reference frame. That is, the G-PCC coder (e.g., the G-PCC encoder 200 or the G-PCC decoder 300) may estimate an initial translation vector by minimizing the mean square error between the current frame and the reference frame. When estimating the initial translation vector, the G-PCC coder may take into account the label of whether the point is on the ground. That is, the G-PCC coder (e.g., the G-PCC encoder 200 or the G-PCC decoder 300) may determine whether the point is on the ground. In this example, the G-PCC coder may estimate a rotation matrix for the second current frame based on whether the point is on the ground. The bG-PCC coder (e.g., G-PCC encoder 200 or G-PCC decoder 300) may combine the initial translation vector with an identity matrix and feed the combined initial translation vector and identity matrix into an iterative closest point method or similar method to estimate the rotation matrix and translation vector. b. In some examples, the G-PCC coder (e.g., the G-PCC encoder 200 or the G-PCC decoder 300) may first estimate a rotation matrix based on a label for whether a point is ground or not. That is, the G-PCC coder (e.g., the G-PCC encoder 200 or the G-PCC decoder 300) may determine whether a point is ground or not. In this example, the G-PCC coder may estimate a rotation matrix for the second current frame based on whether the point is ground or not. For example, the G-PCC encoder 200 may derive this label and signal the label to the G-PCC decoder 300. That is, the G-PCC encoder 200 may signal a set of labels indicating whether a point is ground or not. In some examples, each of the G-PCC encoder 200 and the G-PCC decoder 300 may derive the label. The G-PCC coder may derive labels based on a ground estimation algorithm, which may be based on the height of the point, the density of the point cloud in the point's neighborhood, the point's relative distance from the LIDAR origin / fixed point, etc. A bG-PCC coder (eg, G-PCC encoder 200 or G-PCC decoder 300) may send an initial rotation matrix containing a zero translation vector to an iterative nearest neighbor method or a similar method to estimate the rotation matrix and translation vector.

[0086] In this example, a G-PCC coder (e.g., G-PCC encoder 200 or G-PCC decoder 300) may derive global motion parameters from GPS information. The GPS information may include values / parameters (or estimated values ​​for an acquisition point or each acquisition point) that can be used in deriving roll-pitch-yaw information and velocity under an East-North-Up coordinate system for each time stamp. For example, the G-PCC coder may directly calculate a global motion matrix and a translation vector from the provided information items.

[0087] A rotation matrix may define the change of axes from a reference frame to the current frame: roll, pitch, and yaw (roll) with respect to the reference frame. ref ,pitch ref , Yo ref ) and the roll, pitch, and yaw (roll) for the current frame cur ,pitch cur , Yo cur ), a G-PCC coder (e.g., G-PCC encoder 200 or G-PCC decoder 300) may use the difference in roll, pitch, and yaw to derive. The G-PCC coder may then use this difference as Δ roll =Roll cur -roll ref , Δ pitch =Pitch cur -pitch ref , Δ yaw =Yaw cur -Yo ref It may be calculated as: The rotation matrix for roll is:

number

number

number

[0088] In some examples, a G-PCC coder (e.g., G-PCC encoder 200 or G-PCC decoder 300) may be configured to determine whether a positive direction of one or more of roll, pitch, and yaw occurs. Accordingly, the G-PCC coder may change the sign of the angle because the positive direction may be defined differently.

[0089] For translation vectors, the East-North-Up (ENU) coordinate system aligns well with the coordinate system for the point cloud frame, so a G-PCC coder (e.g., the G-PCC encoder 200 or the G-PCC decoder 300) may use the ENU velocity. The G-PCC coder may calculate the average velocity of the reference frame and the current frame as V = (v East , v North , v Up ) The G-PCC coder may decompose the velocity vector into coordinates in the coordinate system of the reference frame.

[0090] A G-PCC coder (eg, G-PCC encoder 200 or G-PCC decoder 300) may calculate a new rotation matrix for the velocity as follows: The rotation matrix of the velocity for roll is:

number

number

number

number

[0091] In this example, the G-PCC coder (e.g., G-PCC encoder 200 or G-PCC decoder 300) may signal only the roll-pitch-yaw and translation vectors instead of all 12 elements for global motion. This reduces the number of global motion parameters from 12 to 6. That is, the G-PCC encoder 200 may signal a rotation matrix indicating yaw, pitch, and roll for the current frame, including 9 elements, and a translation vector indicating an average velocity for the current frame, including 3 elements (e.g., a total of 12 elements). In some examples, the G-PCC encoder 200 may signal only the roll for the current frame, the pitch for the current frame, the yaw for the current frame, and a translation vector indicating an average velocity for the current frame (e.g., a total of 6 elements), without signaling the complete global rotation matrix.

[0092] A G-PCC coder (e.g., G-PCC encoder 200 or G-PCC decoder 300) may determine a rotation matrix and a translation vector. The G-PCC coder may calculate the difference between roll, pitch, and yaw (Δ roll , Δ pitch , Δ yaw) and the key parameters of the translation vector. Instead of compressing / signaling the entire rotation matrix, the G-PCC encoder 200 may signal only the delta and translation vector, thereby reducing the number of signaled parameters from 12 (9 for the rotation matrix and 3 for the translation vector) to 6 (3 for the delta values ​​and 3 for the translation vector). For example, the G-PCC encoder 200 may signal a translation vector indicating the roll difference between the current frame and the reference frame, the pitch difference between the current frame and the reference frame, the yaw difference between the current frame and the reference frame, and the average velocity of the current frame (e.g., six elements). Similarly, the G-PCC decoder 300 may decode a translation vector indicating the roll difference between the current frame and the reference frame, the pitch difference between the current frame and the reference frame, the yaw difference between the current frame and the reference frame, and the average velocity of the current frame (e.g., six elements).

[0093] Furthermore, the G-PCC coder (e.g., the G-PCC encoder 200 or the G-PCC decoder 300) may reduce the number of global motion parameters through approximation. The elements in the rotation matrix are close to 1 and 0, so the G-PCC coder may approximate the rotation matrix as an identity matrix. This allows the six parameters to be further reduced to only three for the translation vector. That is, the G-PCC encoder 200 may signal a translation vector (e.g., three elements) that indicates the average velocity of the current frame, and not signal a rotation matrix that indicates the yaw, pitch, and roll for the current frame. Similarly, the G-PCC decoder 300 may decode a translation vector (e.g., three elements) that indicates the average velocity of the current frame, and not decode a rotation matrix that indicates the yaw, pitch, and roll for the current frame.

[0094] Assuming that a vehicle (e.g., equipped with LIDAR) moves forward most of the time, a G-PCC coder (e.g., G-PCC encoder 200 or G-PCC decoder 300) may convert the translation vector into T appr =[0,abs(T),0], where abs() is an absolute function for calculating the magnitude of the translation vector T. In that case, the total number of global motion parameters can be reduced to only 1. In this example, the system is defined as a vehicle moving in the positive y direction, but a similar derivation can be applied to other systems. That is, the G-PCC encoder 200 may signal a translation vector whose magnitude indicates the average velocity of the current frame. Similarly, the G-PCC decoder 300 may decode a translation vector whose magnitude indicates the average velocity of the current frame.

[0095] Various combinations of the input parameters, BlockSize and MinPUSize, are tried to find a balanced trade-off point between execution time and performance.

[0096] A G-PCC coder (e.g., G-PCC encoder 200 or G-PCC decoder 300) may determine double BlockSize and double MinPUSize to achieve a balanced trade-off between performance gain and execution time.

[0097] In some examples, a G-PCC coder (e.g., G-PCC encoder 200 or G-PCC decoder 300) may set BlockSize equal to MinPUSize to achieve a balanced trade-off between performance gain and execution time. That is, the G-PCC coder may perform motion vector estimation for the current frame based on a global motion compensated frame, and the G-PCC coder is configured to restrict the block size for local motion to be equal to the minimum prediction unit size to perform motion vector estimation.

[0098] The global motion estimation algorithm may be improved by configuring a G-PCC coder (e.g., the G-PCC encoder 200 or the G-PCC decoder 300) to first estimate a translation vector by minimizing the mean squared error (MSE) between a current frame and a predicted (reference) frame. That is, the G-PCC coder may estimate an initial translation vector by minimizing the mean squared error between a second current frame and a second reference frame. Then, after applying the estimated translation vector, the G-PCC coder (e.g., the G-PCC encoder 200 or the G-PCC decoder 300) may determine a rotation matrix.

[0099] The new global motion estimation algorithm may have two steps. The first step is to configure a G-PCC coder (e.g., G-PCC encoder 200 or G-PCC decoder 300) to calculate an initial translation vector T'. The second step is to have the G-PCC coder send T' and the identity matrix to an iterative nearest neighbor algorithm (e.g., provided by the open3d library) or a similar alternative algorithm.

[0100] In one step, we configure the G-PCC coder to estimate T'. Assume T' = [a, b, c]. The translation vector is assumed to minimize the MSE between the current frame and the predicted frame. Here, G-PCC may represent the MSE from the following loss function:

number

[0101]

number

number

number

number

number

number

[0102] However, since the motion between frames is always large, calculating a, b, and c once may not be accurate enough. A G-PCC coder (e.g., G-PCC encoder 200 or G-PCC decoder 300) may define a loop number l. In the first iteration, given a reference frame f0 and the current frame, the G-PCC coder calculates

number

number

number

number

[0103] Another modification is to the weighting function. Ground points may "confuse" the algorithm executed by a G-PCC coder (e.g., G-PCC encoder 200 or G-PCC decoder 300) to estimate global motion, and therefore the G-PCC coder may "turn off" ground points. The weighting function may be modified by the following formula:

number

[0104] The G-PCC coder (e.g., G-PCC encoder 200 or G-PCC decoder 300) may determine whether a point is a ground point or not based on one or more characteristics of the point, such as height, distance from the center of the point cloud frame, distance from a point in the frame along a plane (e.g., the xy plane).

[0105] In addition to the position of a point of an object / scene / point cloud relative to a local reference, some applications also use its position relative to a fixed location on the Earth. There are several coordinate systems used to describe the geographic location of a point. Below we briefly describe some coordinate systems used in some applications.

[0106] One such coordinate system is the geodetic coordinate system, which is described in more detail at https: / / en.wikipedia.org / wiki / Geodetic_datum. A geodetic coordinate system uses three sets of values: latitude, longitude, and altitude.

[0107] Figure 7 is a conceptual diagram showing the geodetic latitude and longitude of a point measured on the Earth's ellipsoid approximation. Figure 7 shows how the geodetic latitude of a point is obtained as the angle formed by the normal to the point on the Earth's surface (ellipsoid approximation) and the equatorial plane. The angle φ indicates the point's north-south position relative to the Earth. Longitude is measured as the angle λ (in degrees) between the prime meridian (chosen as the point at Greenwich, England), with positive angles measured east of the prime meridian and negative angles measured west. Altitude is defined as the distance measured in units above the ellipsoid surface in a direction perpendicular to the ellipsoid. More information can be found at https: / / en.wikipedia.org / wiki / Geodetic_datum

[0108] Another coordinate system is the geocentric, earth-fixed (ECEF) coordinate system, which is described below. In this coordinate system, the center of the Earth is chosen as the reference point, and the location of a point on the Earth (or more generally, anywhere in space near the Earth's surface) is expressed as a displacement in the x, y, and z dimensions relative to this origin. The positive z-axis is chosen as the line from the origin to the North Pole. The positive x-axis is chosen as the line joining the center to the point on the Earth's surface that is 0 degrees latitude and 0 degrees longitude.

[0109] Figure 8 is a conceptual diagram showing the ECEF (Earth Centered Earth Fixed) coordinate system (X, Y, Z axes) relative to the equator and prime meridian (0° latitude and longitude). A diagram of the ECEF coordinate system relative to latitude and longitude is shown in Figure 8. More details can be found at https: / / en.wikipedia.org / wiki / ECEF

[0110] Next, we will discuss local horizontal coordinate systems (ENU, NED). A local horizontal coordinate system specifies a local horizontal plane on the Earth's surface and represents point locations on that plane with displacements to the east, north, and up (ENU). We may discuss equivalent coordinate systems that use displacements to the north, east, and down (NED). Different coordinate systems may be used in different applications. In some examples, displacements may be described in meters.

[0111] This coordinate system uses ENU / NED displacements as well as a reference against which the displacements are measured. The ENU / NED reference may be described in terms of the ECEF or geodetic coordinate system, or another coordinate system. One advantage of the ENU / NED coordinate system may be that relative displacement values ​​are generally small compared to absolute positions as ECEF coordinates.

[0112] Next, we will discuss the orientation of the point cloud. The orientation of objects in a scene can also be important for applications where one may want to visualize the point cloud with information from the surrounding scene. To visualize the point cloud, one must know the orientation of one or more of the x, y, and z axes of the point cloud frame relative to a fixed reference. This may be expressed as a rotation of the fixed reference xyz axes onto the axes used by the point cloud frame. The rotation may be expressed as a matrix or a roll-pitch-yaw angle triplet.

[0113] In some examples of G-PCC, the G-PCC encoder 200 and the G-PCC decoder 300 may code the x, y, and z positions of points in the point cloud relative to a slice origin, which may be coded relative to an origin specified in a sequence parameter set (SPS), which may be signaled by the G-PCC encoder 200. This (called the SPS origin) is the actual origin of the point cloud frame. The current signaling also includes a scale flag indicating whether the resulting coordinate values ​​after applying a non-standard scaling operation at the decoder side have metric units.

[0114] However, the absolute location of the SPS origin is not specified in the current example of G-PCC. In some applications (e.g., geospatial data visualization), it may be useful to convey the actual location of the frame origin to indicate where the point cloud was acquired or the relationship of the point cloud to a geospatial object. For example, an application may want to render point cloud information in addition to other attributes acquired simultaneously at a particular location. Without the location of the SPS origin, the G-PCC decoder 300 or the application may need to perform expensive registration and separation algorithms to identify the location of the SPS origin.

[0115] In some coordinate systems, orientation information may be important in addition to the location of the point cloud origin. To enable the G-PCC decoder 300 to properly render the point cloud, the G-PCC encoder 200 informs the G-PCC decoder 300 of the x-, y-, and z-axis orientation. Such reporting of x-, y-, and z-axis orientation is not currently supported in G-PCC. In a more specific example, a LIDAR system on a car captures a point cloud relative to the car's driving direction (e.g., the vehicle may assume that the vehicle's driving direction is the positive y-direction), and the car's driving direction may change from frame to frame. The orientation information (as well as the SPS origin) may be frame-specific information.

[0116] One or more of the following techniques may be applied independently or in combination with others.

[0117] Next, we will discuss geolocation / GIS projection. In some examples, the G-PCC encoder 200 may signal the location of the SPS origin relative to a fixed reference. The G-PCC decoder 300 may analyze the signaled location. More generally, the location of the origin associated with a point cloud frame may be signaled.

[0118] In some examples, the G-PCC encoder 200 may specify the location of the SPS origin relative to a fixed origin and coordinate axes using offsets in each dimension. The G-PCC decoder 300 may analyze the specified location. In one example, the SPS origin may be represented using the ECEF coordinate system. In one example, the SPS origin may be represented using the ENU coordinate system. In one example, the SPS origin may be represented using a geodetic coordinate system. More generally, any positioning system may be used.

[0119] In some examples, the G-PCC encoder 200 may signal a syntax element used to indicate the coordinate system used to represent the position of the SPS origin, and the G-PCC decoder 300 may parse the syntax element to determine the coordinate system.

[0120] In some examples, the G-PCC encoder 200 may signal one or more syntax elements that indicate the number of bits used to code the position of the SPS origin in the indicated coordinate system, and the G-PCC decoder 300 may parse the syntax elements to determine the number of bits.

[0121] Next, we will discuss orientation. In some examples, the G-PCC encoder 200 may use parameters that represent orientation of the x, y, and z axes. For example, the parameters may represent rotation from a fixed axis coordinate system (e.g., the ECEF XYZ coordinate system). In some examples, the parameters may represent a rotation matrix. In some examples, the parameters may represent roll, pitch, and yaw.

[0122] In some examples, the G-PCC encoder 200 may signal a syntax element used to indicate the coordinate system used to represent the orientation of the point cloud frame axes, and the G-PCC decoder 300 may parse the syntax element to determine the coordinate system.

[0123] In some examples, the G-PCC encoder 200 may signal one or more syntax elements that indicate the number of bits used to code parameters to represent the orientation of the point cloud frame, and the G-PCC decoder 300 may parse the syntax elements to determine the number of bits.

[0124] Next, restrictions on position and orientation will be described. The G-PCC encoder 200 may apply certain restrictions based on applicable constraints due to the nature of the application system used. For example, for LIDAR data captured by a vehicle, the G-PCC encoder 200 may limit the position information by providing only latitude and longitude and not specifying altitude. In another example, for data captured by LIDAR, the G-PCC encoder 200 may provide only the vehicle's velocity in the ENU direction to enable the G-PCC decoder 300 to derive position, and may also provide angular velocity to specify orientation over multiple frames.

[0125] In some examples, other parameters associated with the point cloud may be signaled, such as angular velocity, angular acceleration, linear velocity, linear acceleration, time associated with the capture of the point cloud frame, etc.

[0126] Next, we will discuss the relationship between frame-based signaling and sequence-based signaling. In some examples, the position and / or orientation of the point cloud origin and the axis orientation are fixed for the entire sequence or bitstream. In such cases, for example, the G-PCC encoder 200 may signal the position and axis orientation of the point cloud origin only once per sequence (e.g., in a parameter set such as an SPS). The G-PCC decoder 300 may parse syntax elements in the parameter set to determine the position and orientation.

[0127] In some examples, the position and / or orientation of the point cloud origin may vary from frame to frame, and the G-PCC encoder 200 may signal values ​​for one or more frames. The G-PCC decoder 300 may analyze the signaled values ​​to determine the position and orientation.

[0128] The following example shows how the position and coordinates of the point cloud origin are represented for each frame.

[0129] [Table 1]

[0130] When pcoo_update_flag is equal to 0, the syntax structure contains at least one of the SPS origin position and axis orientations for the current frame relative to some fixed coordinate system as indicated by pcoo_origin_coordinate_system_id and pcoo_origin_orientation_system_id, respectively. When pcoo_update_flag is equal to 1, the syntax structure contains the SPS origin position and axis orientations for the current frame relative to some fixed coordinate system as indicated by pcoo_origin_coordinate_system_id and pcoo_origin_orientation_system_id, respectively, where pcoo_origin_coordinate_val[] and pcoo_origin_orientation_val[] indicate delta-coded values ​​from the previous frame when pcoo_update_flag was equal to 0.

[0131] In some examples, a restriction may be added so that pcoo_update_flag can only be 0 for pictures that cannot be removed from the bitstream (e.g., frames associated with the lowest frame rate, or the first frame of a sequence).

[0132] In one example, an ID is specified for each syntax structure, and when pcoo_update_flag is equal to 1, the reference ID to the reference position and heading is signaled.

[0133] In some examples, the reference position and orientation is selected as the previous point cloud frame with a temporary ID equal to or lower than the current frame.

[0134] Alternatively, when pcoo_update_flag is equal to 1, a frame index is signaled and the delta-coded position and orientation are measured relative to the position and orientation of that frame.

[0135] In one example, delta coding is applied only to position values ​​and not to heading values.

[0136] If pcoo_origin_info_present_flag is equal to 1, it specifies that the absolute position information of the SPS origin is signaled. If pcoo_origin_info_present_flag is equal to 0, it specifies that the absolute position information of the point cloud frame is not signaled.

[0137] In such cases, default absolute position information may be selected or sent by external means.

[0138] pcoo_origin_coordinate_system_id specifies the coordinate system used to represent the absolute position of the SPS origin with respect to the current frame. Values ​​of pcoo_origin_coordinate_system_id shall be in the range 0 to 2 for bitstreams conforming to this version of this specification. Other values ​​are reserved for future use by ISO / IEC.

[0139] pcoo_origin_coordinate_num_params specifies the number of parameters to be signaled for the position in the indicated coordinate system.

[0140] In some alternatives, pcoo_origin_coordinate_num_params is fixed and predetermined based on the value of pcoo_origin_coordinate_system_id. In some alternatives, this syntax element is coded as _minusN, and values ​​smaller than N are not allowed to be signaled.

[0141] pcoo_origin_coordinate_num_bits is used to specify the number of bits used to signal pcoo_origin_coordinate_val[i].

[0142] pcoo_origin_coordinate_val[i], for i=0..pcoo_origin_coordinate_num_params-1, is used to derive the position of the SPS origin for the current frame.

[0143] The interpretation of pcoo_origin_coordinate_val[] can be shown by the following table:

[0144] [Table 2]

[0145] It should be noted that the precisions above, and in the remainder of this disclosure, are examples only, and that the techniques of this disclosure apply to any precision in meters, degrees, or other units.

[0146] In one example, for some coordinate systems, the number of parameters for the same coordinate system may differ based on the value of update_flag. For example, when update_flag is equal to 0, the ENU coordinate system may have six points, and when update_flag is equal to 1, the ENU coordinate system may have only three points corresponding to local displacements.

[0147] In another example, the option to signal a local reference may be allowed. A local reference may be signaled when a syntax element indicating the presence of a local reference (e.g., local reference_present_flag) is signaled.

[0148] If pcoo_orientation_info_present_flag is equal to 1, it specifies that the orientation of the XYZ axes of the point cloud frame is signaled. If pcoo_orientation_info_present_flag is equal to 0, it specifies that the orientation of the XYZ axes of the point cloud frame is not signaled.

[0149] pcoo_orientation_system_id specifies the orientation coordinate system used to represent the orientation of the XYZ axes of the point cloud frame. Values ​​of pcoo_orientation_system_id shall be in the range 0 to 1 for bitstreams conforming to this version of this specification. Other values ​​are reserved for future use by ISO / IEC.

[0150] pcoo_orientation_coordinate_num_params specifies the number of parameters to be signaled for the orientation in the indicated coordinate system.

[0151] In some examples, the value of pcoo_orientation_coordinate_num_params may be fixed and predetermined based on the value of pcoo_orientation_coordinate_system_id. In some examples, this syntax element may be coded as _minusN, where values ​​less than N are not allowed to be signaled.

[0152] pcoo_orientation_coordinate_num_bits is used to specify the number of bits used to signal pcoo_origin_coordinate_val[i].

[0153] pcoo_orientation_coordinate_val[i] for i = 1. pcoo_orientation_coordinate_num_params is used to determine the orientation XYZ axes of the point cloud frame.

[0154] The interpretation of pcoo_orientation_coordinate_val may be shown by the following table:

[0155] [Table 3]

[0156] In one example, at least one of pcoo_orientation_info_present_flag and pcoo_origin_info_present_flag is constrained to be equal to 1 in the syntax structure.

[0157] In some examples, the indexed parameters 0-8 correspond to row-wise scanned elements of the rotation matrix (i.e., the i-th index corresponds to the (i / 3)-th row and the (i%2)-th column). More generally, any scanning pattern of rotation matrix elements may be chosen to obtain the pcoo_orientation_coordinate_val[i] parameters.

[0158] 9 is a flowchart illustrating an example encoding process according to the techniques of this disclosure. The G-PCC encoder 200 (e.g., the octree analysis unit 210) may identify a first set of global motion parameters from the global positioning system information (902). For example, the G-PCC encoder 200 may identify a set of orientation parameters and / or a set of position parameters for the current frame from the global positioning system information.

[0159] The G-PCC encoder 200 (e.g., the octree analysis unit 210) may determine a second set of global motion parameters to be used for global motion estimation for the current frame based on the first set of global motion parameters (904). For example, the G-PCC encoder 200 may determine a rotation matrix indicating the yaw, pitch, and roll for the current frame and a translation vector indicating the average velocity for the current frame.

[0160] The G-PCC encoder 200 (e.g., the octree analysis unit 210) may apply motion compensation to the reference frame based on the second set of global motion parameters to generate a global motion-compensated frame for the current frame (906). For example, the G-PCC encoder 200 (e.g., the octree analysis unit 210) may apply global motion compensation to all points in the reference frame (e.g., the predicted frame) based on the second set of global motion parameters (e.g., a rotation matrix and a translation vector). In this way, the G-PCC encoder 200 may apply global motion compensation using satellite information, which may be more accurate than estimating a rotation matrix and a translation vector for the current frame based on feature points between the reference frame (e.g., the predicted frame) and the current frame.

[0161] The G-PCC encoder 200 (e.g., the octree analysis unit 210) may apply local node motion estimation based on the global motion compensation frame to generate motion vector information and point information for the current frame (908). For example, the G-PCC encoder 200 (e.g., the octree analysis unit 210) may apply local node motion estimation. For example, the G-PCC encoder 200 (e.g., the octree analysis unit 210) may apply a brute force search within the current node. In this example, the G-PCC encoder 200 (e.g., the octree analysis unit 210) may sample a portion of the points within the current node. Given an initial motion vector, the G-PCC encoder 200 (e.g., the octree analysis unit 210) may generate compensation points and calculate costs between pairs of those points and compensation points in the reference node. The G-PCC encoder 200 (e.g., the octree analysis unit 210) may select the motion vector that minimizes the cost as the final estimated local node motion vector.

[0162] The G-PCC encoder 200 (e.g., the arithmetic coding unit 214) may signal global motion information, motion vector information, and point information for the current frame (910). For example, the G-PCC encoder 200 (e.g., the arithmetic coding unit 214) may signal a first set of global motion parameters (e.g., a set of orientation parameters and / or a set of position parameters). In some examples, the G-PCC encoder 200 (e.g., the arithmetic coding unit 214) may signal a second set of global motion parameters (e.g., a rotation matrix and a translation vector). The G-PCC encoder 200 (e.g., the arithmetic coding unit 214) may signal a roll difference between the current frame and the reference frame, a pitch difference between the current frame and the reference frame, a yaw difference between the current frame and the reference frame, and a translation vector indicating the average velocity of the current frame (e.g., six elements). In some examples, the G-PCC encoder 200 (e.g., the arithmetic coding unit 214) may signal a translation vector that indicates the average velocity of the current frame (e.g., three elements). The G-PCC encoder 200 (e.g., the arithmetic coding unit 214) may then signal an approximation of the translation vector as T appr =[0,abs(T),0], where abs() is the absolute function for calculating the magnitude of the translation vector T.

[0163] 10 is a flowchart illustrating an example decoding process according to the techniques of this disclosure. G-PCC encoder 200 (e.g., geometry arithmetic decoding unit 302) may decode symbols of a bitstream that indicate global motion information, motion vector information, and point information for a current frame (1002).

[0164] For example, the G-PCC decoder 300 (e.g., the geometry arithmetic decoding unit 302) may decode 203 a first set of global motion parameters and / or a second set of global motion parameters from the geometry bitstream. The G-PCC decoder 300 (e.g., the geometry arithmetic decoding unit 302) may decode a roll difference between the current frame and the reference frame, a pitch difference between the current frame and the reference frame, a yaw difference between the current frame and the reference frame, and a translation vector indicating an average velocity of the current frame. In some examples, the G-PCC decoder 300 (e.g., the geometry arithmetic decoding unit 302) may decode a translation vector indicating an average velocity of the current frame. In some examples, the G-PCC decoder 300 (e.g., the geometry arithmetic coding unit 302) may decode a magnitude translation vector indicating an average velocity of the current frame.

[0165] The G-PCC decoder 300 (e.g., the octree analysis unit 306) may determine a set of global motion parameters to be used for global motion estimation for the current frame based on the global motion information (1004). For example, the G-PCC decoder 300 (e.g., the octree synthesis unit 306) may determine a rotation matrix and a translation vector based on a roll difference, a pitch difference, a yaw difference, and a translation vector. The G-PCC decoder 300 (e.g., the octree synthesis unit 306) may decode the translation vector from the geometry bitstream 203 and approximate the rotation matrix as an identity matrix. In some examples, the G-PCC decoder 300 (e.g., the octree synthesis unit 306) may decode a translation vector of a magnitude indicating an average velocity of the current frame from the geometry bitstream 203. In this example, the G-PCC decoder 300 (e.g., the octree synthesis unit 306) may approximate the translation vector based on the received magnitude and approximate the rotation matrix as an identity matrix.

[0166] The G-PCC decoder 300 (e.g., the octree synthesis unit 306) may apply global motion compensation to the reference frame based on the set of global motion parameters to generate a global motion-compensated frame for the current frame (1006). In this way, the G-PCC decoder 300 (e.g., the octree synthesis unit 306) may apply global motion compensation using satellite information, which may be more accurate than estimating a rotation matrix and a translation vector for the current frame based on feature points between the reference frame (e.g., the predicted frame) and the current frame.

[0167] The G-PCC decoder 300 (e.g., the octree analysis unit 306) may apply local node motion estimation based on the global motion compensation frame to generate a current frame (1008). The G-PCC decoder 300 may output the current frame (1012). For example, the G-PCC decoder 300 may cause a display to output the current frame.

[0168] The examples in the various aspects of the present disclosure may be used individually or in any combination.

[0169] Clause A1 A device for encoding point cloud data, comprising: a memory for storing the point cloud data; and one or more processors coupled to the memory and implemented in circuitry, wherein the one or more processors are configured to: identify a first set of global motion parameters from global positioning system information; determine a second set of global motion parameters to be used for global motion estimation for a current frame based on the first set of global motion parameters; and apply motion compensation to a reference frame based on the second set of global motion parameters to generate a global motion compensated frame for the current frame.

[0170] Clause A2. The device of clause A1, wherein the one or more processors are configured to signal a second set of global motion parameters in the bitstream.

[0171] Clause A3. The device of clause A1, wherein the one or more processors are configured to signal the first set of global motion parameters in the bitstream.

[0172] Clause A4. The device of clause A1, wherein the first set of global motion parameters includes a set of orientation parameters.

[0173] Clause A5. The device of clause A4, wherein the set of orientation parameters includes one or more of roll, pitch, yaw, or angular velocity for a current frame.

[0174] Clause A6. The device of clause A4, wherein the set of orientation parameters includes one or more of a roll difference between the current frame and the reference frame, a pitch difference between the current frame and the reference frame, a yaw difference between the current frame and the reference frame, or an angular velocity difference between the current frame and the reference frame.

[0175] Clause A7. The device of clause A1, wherein the first set of global motion parameters includes a set of position parameters.

[0176] Clause A8. The device of clause A7, wherein the set of position parameters includes one or more of a displacement for a current frame or an average velocity for a current frame.

[0177] Clause A9. The device of clause A7, wherein the set of location parameters includes one or more of an east velocity, a north velocity, or an up velocity of an East-North-Up coordinate system.

[0178] Clause A10. The device of clause A1, wherein the second set of global motion parameters includes a rotation matrix indicating yaw, pitch, and roll for the current frame and a translation vector indicating an average velocity for the current frame.

[0179] Clause A11 The device of clause A1, wherein the one or more processors are configured to signal a translation vector indicating the global motion parameters: roll for the current frame, pitch for the current frame, yaw for the current frame, and average velocity for the current frame.

[0180] Clause A12. The device of clause A1, wherein the one or more processors are configured to signal a translation vector indicating a roll difference between the current frame and the reference frame, a pitch difference between the current frame and the reference frame, a yaw difference between the current frame and the reference frame, and an average velocity of the current frame.

[0181] Clause A13. The device of clause A1, wherein the one or more processors are configured to signal a translation vector indicating an average velocity for the current frame and not signal a rotation matrix indicating yaw, pitch, and roll for the current frame.

[0182] Clause A14. The device of clause A1, wherein the one or more processors are configured to signal a translation vector having a magnitude indicative of an average velocity for a current frame.

[0183] Clause A15 The device of clause A1, wherein the one or more processors are configured to perform motion vector estimation for the current frame based on a global motion compensated frame, and the one or more processors are configured to limit a block size for local motion to be equal to a minimum prediction unit size for performing motion vector estimation.

[0184] Clause A16. The device of clause A1, wherein the reference frame is a first reference frame, the current frame is a first current frame, and the one or more processors are configured to estimate the initial translation vector by minimizing a mean squared error between the second current frame and the second reference frame.

[0185] Clause A17. The device of clause A1, wherein the current frame is a first current frame, and the one or more processors are configured to determine whether the point is on the ground and estimate a rotation matrix for a second current frame based on whether the point is on the ground.

[0186] Clause A18. The device of clause A17, wherein the one or more processors are configured to signal a set of labels indicating whether the point is on the ground or not.

[0187] Clause A19. The device of clause A1, wherein the one or more processors are further configured to generate point cloud data.

[0188] Clause A20 The device of clause A19, wherein the one or more processors are configured to generate the point cloud data based on signals from the LIDAR device as part of generating the point cloud data.

[0189] Clause A21 The device of clause A1, wherein the device is one of a mobile phone, a tablet computer, a vehicle, or an augmented reality device.

[0190] Clause A22. The device of clause A1, wherein the device comprises an interface configured to transmit the encoded point cloud data.

[0191] Clause A23 A method for encoding point cloud data, the method comprising: identifying, by one or more processors, a first set of global motion parameters from global positioning system information; determining, by the one or more processors, based on the first set of global motion parameters, a second set of global motion parameters to be used for global motion estimation for a current frame; and applying, by the one or more processors, based on the second set of global motion parameters, motion compensation to a reference frame to generate a global motion compensated frame for the current frame.

[0192] Clause A24 The method of clause A23, further comprising signaling, by one or more processors, the second set of global motion parameters in the bitstream.

[0193] Clause A25 The method of clause A23, further comprising signaling, by one or more processors, the first set of global motion parameters in the bitstream.

[0194] Clause A26 The method of clause A23, wherein the first set of global motion parameters includes a set of orientation parameters.

[0195] Clause A27 The method of clause A26, wherein the set of orientation parameters includes one or more of roll, pitch, yaw, or angular velocity for the current frame.

[0196] Clause A28 The method of clause A26, wherein the set of orientation parameters includes one or more of a roll difference between the current frame and the reference frame, a pitch difference between the current frame and the reference frame, a yaw difference between the current frame and the reference frame, or an angular velocity difference between the current frame and the reference frame.

[0197] Clause A29 The method of clause A23, wherein the first set of global motion parameters includes a set of position parameters.

[0198] Clause A30 The method of clause A29, wherein the set of position parameters includes one or more of a displacement for the current frame or an angular velocity for the current frame.

[0199] Clause A31 The method of clause A29, wherein the set of position parameters includes one or more of an east velocity, a north velocity, or an up velocity of an East-North-Up coordinate system.

[0200] Clause A32 The method of clause A23, wherein the second set of global motion parameters includes a rotation matrix indicating yaw, pitch, and roll for the current frame and a translation vector indicating an average velocity for the current frame.

[0201] Clause A33 The method of clause A23, further comprising signaling a translation vector indicating a roll for the current frame, a pitch for the current frame, a yaw for the current frame, and an average velocity for the current frame.

[0202] Clause A34 The method of clause A23, further comprising signaling a translation vector indicating a roll difference between the current frame and the reference frame, a pitch difference between the current frame and the reference frame, a yaw difference between the current frame and the reference frame, and an average velocity of the current frame.

[0203] Clause A35 The method of clause A23, further comprising the step of signaling, by the one or more processors, a translation vector indicating an average velocity for the current frame, and not signaling a rotation matrix indicating yaw, pitch, and roll for the current frame.

[0204] Clause A36 The method of clause A23, further comprising signaling, by the one or more processors, a translation vector whose magnitude indicates an average velocity for the current frame.

[0205] Clause A37 The method of clause A23, further comprising performing, by one or more processors, a step of performing motion vector estimation for the current frame based on the global motion compensated frame, wherein the step of performing motion vector estimation comprises a step of restricting the block size for local motion to be equal to the minimum prediction unit size.

[0206] Clause A38 The method of clause A23, wherein the reference frame is a first reference frame and the current frame is a first current frame, and the method further comprises estimating the initial translation vector by minimizing the mean square error between the second current frame and the second reference frame.

[0207] Clause A39 The method of clause A23, wherein the current frame is a first current frame, and the method further includes the steps of determining, by one or more processors, whether the point is on the ground, and estimating, by one or more processors, a rotation matrix for a second current frame based on whether the point is on the ground.

[0208] Clause A40 The method of clause A39, further comprising signaling, by one or more processors, a set of labels indicating whether the point is on the ground or not.

[0209] Clause A41 The method of clause A23, further comprising generating, by one or more processors, point cloud data.

[0210] Clause A42 The method of clause A41, further comprising generating, by one or more processors, point cloud data based on signals from the LIDAR device.

[0211] Clause A43 A computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors to identify a first set of global motion parameters from global positioning system information; determine a second set of global motion parameters to be used for global motion estimation for a current frame based on the first set of global motion parameters; and apply motion compensation to a reference frame based on the second set of global motion parameters to generate a global motion compensated frame for the current frame.

[0212] Clause B1 A device for encoding point cloud data, comprising: a memory for storing the point cloud data; and one or more processors coupled to the memory and implemented in circuitry, wherein the one or more processors are configured to: identify a first set of global motion parameters from global positioning system information; determine a second set of global motion parameters to be used for global motion estimation for a current frame based on the first set of global motion parameters; and apply motion compensation to a reference frame based on the second set of global motion parameters to generate a global motion compensated frame for the current frame.

[0213] Clause B2. The device of clause B1, wherein the one or more processors are configured to signal a second set of global motion parameters in the bitstream.

[0214] Clause B3. The device of clause B1, wherein the one or more processors are configured to signal the first set of global motion parameters in the bitstream.

[0215] Clause B4. The device of any one of clauses B1 to B3, wherein the first set of global motion parameters includes a set of orientation parameters.

[0216] Clause B5. The device of clause B4, wherein the set of orientation parameters includes one or more of roll, pitch, yaw, or angular velocity for a current frame.

[0217] Clause B6. The device of clause B4, wherein the set of orientation parameters includes one or more of a roll difference between the current frame and the reference frame, a pitch difference between the current frame and the reference frame, a yaw difference between the current frame and the reference frame, or an angular velocity difference between the current frame and the reference frame.

[0218] Clause B7 The device of any one of clauses B1 to B6, wherein the first set of global motion parameters includes a set of position parameters.

[0219] Clause B8. The device of clause B7, wherein the set of position parameters includes one or more of a displacement for a current frame or an average velocity for a current frame.

[0220] Clause B9. The device of clause B7, wherein the set of location parameters includes one or more of an east velocity, a north velocity, or an up velocity of an East-North-Up coordinate system.

[0221] Clause B10. The device of any one of clauses B1 to B9, wherein the second set of global motion parameters includes a rotation matrix indicating yaw, pitch, and roll for the current frame and a translation vector indicating average velocity for the current frame.

[0222] Clause B11 The device of any one of clauses B1, B4 to B10, wherein the one or more processors are configured to signal a translation vector indicating a roll for the current frame, a pitch for the current frame, a yaw for the current frame, and an average velocity for the current frame.

[0223] Clause B12. The device of any one of clauses B1, B4 to B10, wherein the one or more processors are configured to signal a translation vector indicating a roll difference between the current frame and the reference frame, a pitch difference between the current frame and the reference frame, a yaw difference between the current frame and the reference frame, and an average velocity of the current frame.

[0224] Clause B13 The device of any one of clauses B1, B4 to B10, wherein the one or more processors are configured to signal a translation vector indicating an average velocity for the current frame and not signal a rotation matrix indicating yaw, pitch, and roll for the current frame.

[0225] Clause B14 The device of any one of clauses B1, B4 to B10, wherein the one or more processors are configured to signal a translation vector having a magnitude indicative of an average velocity for the current frame.

[0226] Clause B15 A device described in any one of clauses B1 to B14, wherein the one or more processors are configured to perform motion vector estimation for a current frame based on a global motion compensated frame, and the one or more processors are configured to limit a block size for local motion to be equal to a minimum prediction unit size for performing motion vector estimation.

[0227] Clause B16 The device of any one of clauses B1 to B15, wherein the reference frame is a first reference frame, the current frame is a first current frame, and the one or more processors are configured to estimate the initial translation vector by minimizing a mean squared error between the second current frame and the second reference frame.

[0228] Clause B17. The device of any one of clauses B1 to B15, wherein the current frame is a first current frame, and the one or more processors are configured to determine whether the point is on the ground and estimate a rotation matrix for a second current frame based on whether the point is on the ground.

[0229] Clause B18. The device of clause B17, wherein the one or more processors are configured to signal a set of labels that indicate whether the point is on the ground or not.

[0230] Clause B19. The device of any one of clauses B1 to B18, wherein the one or more processors are further configured to generate point cloud data.

[0231] Clause B20. The device of clause B19, wherein the one or more processors are configured to generate the point cloud data based on signals from the LIDAR device as part of generating the point cloud data.

[0232] Clause B21 The device of any one of clauses B1 to B20, wherein the device is one of a mobile phone, a tablet computer, a vehicle, or an augmented reality device.

[0233] Clause B22. The device of any one of clauses B1 to B21, wherein the device comprises an interface configured to transmit the encoded point cloud data.

[0234] Clause B23 A method for encoding point cloud data, the method comprising: identifying, by one or more processors, a first set of global motion parameters from global positioning system information; determining, by the one or more processors, based on the first set of global motion parameters, a second set of global motion parameters to be used for global motion estimation for a current frame; and applying, by the one or more processors, based on the second set of global motion parameters, motion compensation to a reference frame to generate a global motion compensated frame for the current frame.

[0235] Clause B24 The method of clause B23, further comprising signaling, by the one or more processors, the second set of global motion parameters in the bitstream.

[0236] Clause B25 The method of clause B23, further comprising signaling, by one or more processors, the first set of global motion parameters in a bitstream.

[0237] Clause B26 The method of any one of clauses B23 to B25, wherein the first set of global motion parameters includes a set of orientation parameters.

[0238] Clause B27 The method of clause B26, wherein the set of orientation parameters includes one or more of roll, pitch, yaw, or angular velocity for the current frame.

[0239] Clause B28. The method of clause B26, wherein the set of orientation parameters includes one or more of a roll difference between the current frame and the reference frame, a pitch difference between the current frame and the reference frame, a yaw difference between the current frame and the reference frame, or an angular velocity difference between the current frame and the reference frame.

[0240] Clause B29 The method of any one of clauses B23 to B28, wherein the first set of global motion parameters includes a set of position parameters.

[0241] Clause B30 The method of clause B29, wherein the set of position parameters includes one or more of a displacement for the current frame or an angular velocity for the current frame.

[0242] Clause B31 The method of clause B29, wherein the set of position parameters includes one or more of an east velocity, a north velocity, or an up velocity of an East-North-Up coordinate system.

[0243] Clause B32. The device of any one of clauses B23 to B31, wherein the second set of global motion parameters includes a rotation matrix indicating yaw, pitch, and roll for the current frame and a translation vector indicating average velocity for the current frame.

[0244] Clause B33. The method of clause B23, further comprising signaling a translation vector indicating a roll for the current frame, a pitch for the current frame, a yaw for the current frame, and an average velocity for the current frame.

[0245] Clause B34. The method of any one of clauses B23, B26 to B33, further comprising signaling a translation vector indicating a roll difference between the current frame and the reference frame, a pitch difference between the current frame and the reference frame, a yaw difference between the current frame and the reference frame, and an average velocity of the current frame.

[0246] Clause B35 The method of any one of clauses B23, B26 to B33, further comprising the step of signaling, by one or more processors, a translation vector indicating an average velocity for the current frame, and not signaling a rotation matrix indicating yaw, pitch, and roll for the current frame.

[0247] Clause B36 The method of any one of clauses B23, B26 to B33, further comprising signaling, by one or more processors, a translation vector whose magnitude indicates the average velocity for the current frame.

[0248] Clause B37 The method of any one of clauses B23 to B36, further comprising performing, by one or more processors, a step of performing motion vector estimation for the current frame based on the global motion compensated frame, wherein performing motion vector estimation comprises a step of restricting the block size for local motion to be equal to the minimum prediction unit size.

[0249] Clause B38 The method of any one of clauses B23 to B37, wherein the reference frame is a first reference frame and the current frame is a first current frame, and the method further comprises estimating the initial translation vector by minimizing the mean square error between the second current frame and the second reference frame.

[0250] Clause B39 The method of any one of clauses B23 to B37, wherein the current frame is a first current frame, and the method further includes the steps of determining, by one or more processors, whether the point is on the ground, and estimating, by one or more processors, a rotation matrix for a second current frame based on whether the point is on the ground.

[0251] Clause B40 The method of clause B39, further comprising signaling, by one or more processors, a set of labels indicating whether the point is on the ground or not.

[0252] Clause B41 The method of any one of clauses B23 to B40, further comprising generating, by one or more processors, point cloud data.

[0253] Clause B42 The method of clause B41, further comprising generating, by one or more processors, point cloud data based on signals from the LIDAR device.

[0254] Clause B43 A computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors to identify a first set of global motion parameters from global positioning system information; determine a second set of global motion parameters to be used for global motion estimation for a current frame based on the first set of global motion parameters; and apply motion compensation to a reference frame based on the second set of global motion parameters to generate a global motion compensated frame for the current frame.

[0255] It should be appreciated that in some examples, some acts or events of any of the techniques described herein may be performed in a different sequence, added, combined, or entirely excluded (e.g., not all described acts or events may be necessary to practice the techniques). Moreover, in some examples, acts or events may be performed in parallel rather than sequentially, for example, through multithreaded processing, interrupt processing, or multiple processors.

[0256] In one or more examples, the functions described may be implemented as hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted via a computer-readable medium as one or more instructions or code and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which correspond to tangible media such as data storage media, or communication media, including any medium that facilitates transfer of a computer program from one place to another, for example, according to a communication protocol. As such, computer-readable media may generally correspond to (1) non-transitory tangible computer-readable storage media or (2) a communication medium such as a signal or carrier wave. Data storage media may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described in this disclosure. A computer program product may include computer-readable media.

[0257] By way of example, and not limitation, such computer-readable storage media may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included within the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but instead cover non-transitory tangible storage media. As used herein, disk and disc include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically and discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0258] The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the terms "processor" and "processing circuitry," as used herein, may refer to any of the above structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or may be incorporated into a combined codec. Also, the techniques may be implemented entirely in one or more circuits or logic elements.

[0259] The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units have been described in this disclosure to highlight functional aspects of devices configured to implement the disclosed techniques, but they do not necessarily require realization by different hardware units. Rather, as described above, the various units may be combined in a codec hardware unit or may be provided by a collection of interoperable hardware units, including one or more processors as described above, along with appropriate software and / or firmware.

[0260] Various examples have been described. These and other examples are within the scope of the following claims. [Explanation of symbols]

[0261] 100 systems 102 Source Devices 104 Data Sources 106 memory 108 Output Interface 110 Computer-Readable Medium 112 Storage Devices 114 File Server 116 Destination Device 118 Data Consumers 120 memory 122 input interface 200 G-PCC Encoder 202 Coordinate Transformation Unit 203 Geometry Bitstream 204 Color Conversion Unit 205 Attribute Bitstream 206 Voxelization Unit 208 Attribute Transfer Unit 210 Octree Analysis Unit 212 Surface Approximation Analysis Unit 214 Arithmetic Coding Unit 216 Geometry Reconstruction Unit 218 RAHT Unit 220 LOD generation units 222 Lifting Unit 224 Coefficient Quantization Unit 226 Arithmetic Coding Unit 300 G-PCC decoder 302 Geometry Arithmetic Decoding Unit 304 Attribute Arithmetic Decoding Unit 306 8th Tree Synthesis Unit 308 Inverse Quantization Unit 310 Surface Approximation Synthesis Unit 312 Geometry Reconstruction Unit 314 RAHT unit 316 LOD generation units 318 Reverse Lifting Unit 320 Inverse Transform Coordinate Units 322 Inverse Transformation Color Unit 402 predicted frames 602 Current Node

Claims

1. 1. A device for encoding point cloud data, comprising: a memory for storing the point cloud data; one or more processors implemented in circuitry coupled to the memory, the one or more processors: identifying a first set of global motion parameters from the global positioning system information; determining a second set of global motion parameters to be used in global motion estimation for a current frame based on the first set of global motion parameters; A device configured to apply motion compensation to a reference frame based on the second set of global motion parameters to generate a global motion compensated frame for the current frame.

2. The device of claim 1 , wherein the one or more processors are configured to signal the second set of global motion parameters in a bitstream.

3. The device of claim 1 , wherein the one or more processors are configured to signal the first set of global motion parameters in a bitstream.

4. The device of claim 1 , wherein the first set of global motion parameters includes a set of orientation parameters.

5. The device of claim 4 , wherein the set of orientation parameters includes one or more of roll, pitch, yaw, or angular velocity for the current frame.

6. 5. The device of claim 4, wherein the set of orientation parameters includes one or more of a roll difference between the current frame and a reference frame, a pitch difference between the current frame and the reference frame, a yaw difference between the current frame and the reference frame, or an angular velocity difference between the current frame and the reference frame.

7. The device of claim 1 , wherein the first set of global motion parameters includes a set of position parameters.

8. The device of claim 7 , wherein the set of position parameters includes one or more of a displacement for the current frame or an average velocity for the current frame.

9. The device of claim 7 , wherein the set of location parameters includes one or more of an east velocity, a north velocity, or an up velocity of an East-North-Up coordinate system.

10. 10. The device of claim 1, wherein the second set of global motion parameters includes a rotation matrix indicating yaw, pitch, and roll for the current frame and a translation vector indicating an average velocity for the current frame.

11. 10. The device of claim 1, wherein the one or more processors are configured to signal a translation vector indicating a roll for the current frame, a pitch for the current frame, a yaw for the current frame, and an average velocity for the current frame.

12. 10. The device of claim 1, wherein the one or more processors are configured to signal a translation vector indicating a difference in roll between the current frame and a reference frame, a difference in pitch between the current frame and the reference frame, a difference in yaw between the current frame and the reference frame, and an average velocity of the current frame.

13. 10. The device of claim 1, wherein the one or more processors are configured to signal a translation vector indicating an average velocity for the current frame and not signal a rotation matrix indicating yaw, pitch, and roll for the current frame.

14. 1. A method for encoding point cloud data, comprising: identifying, by one or more processors, a first set of global motion parameters from the global positioning system information; determining, by one or more processors, a second set of global motion parameters to be used for global motion estimation for the current frame based on the first set of global motion parameters; and applying, by one or more processors, motion compensation to a reference frame based on the second set of global motion parameters to generate a global motion compensated frame for the current frame.

15. A computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors to: identifying a first set of global motion parameters from the global positioning system information; determining a second set of global motion parameters to be used in global motion estimation for a current frame based on the first set of global motion parameters; A computer-readable storage medium that applies motion compensation to a reference frame based on the second set of global motion parameters to generate a global motion compensated frame for the current frame.

Images