Point cloud data transmission device, point cloud data transmission method, point cloud data reception device, and point cloud data reception method

By encoding and decoding point cloud data, combined with inter-frame prediction technology, the reference road frame is generated, and the problems of point cloud data processing efficiency and quality in the existing technology are solved, and efficient point cloud data processing and service provision are achieved.

CN118923117APending Publication Date: 2024-11-08LG ELECTRONICS INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380029113.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-03-21
Filing Date
2023-03-21
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The prior art is difficult to efficiently process large amounts of point cloud data, resulting in high waiting time and encoding/decoding complexity, affecting the efficiency and quality of point cloud data processing.

Method used

A method is adopted, including encoding and decoding point cloud data, sending and receiving bitstreams of point cloud data, using point cloud video encoder and decoder for geometric and attribute encoding, and combining inter-frame prediction technology to generate reference paths Frames to support efficient geometric compression and attribute compression.

Benefits of technology

It realizes efficient processing of point cloud data and provides high-quality point cloud services, which are suitable for applications such as virtual reality, augmented reality and self-driving, reducing the problem of memory burden and reduced prediction accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118923117A_ABST
    Figure CN118923117A_ABST
Patent Text Reader

Abstract

A point cloud data transmission method according to an embodiment may comprise the steps of: encoding point cloud data; and transmitting the bit stream including the point cloud data. A point cloud data receiving method according to an embodiment may comprise the steps of: receiving a bitstream including point cloud data; and decoding the point cloud data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Embodiments relate to a method and apparatus for processing point cloud content. Background Art

[0002] Point cloud content is content represented by a point cloud, which is a collection of points belonging to a coordinate system representing a three-dimensional space. Point cloud content can express media configured in three dimensions and is used to provide various services such as virtual reality (VR), augmented reality (AR), mixed reality (MR), and self-driving services. However, tens of thousands to hundreds of thousands of point data are required to represent point cloud content. Therefore, a method of efficiently processing a large amount of point data is needed. Summary of the invention

[0003] Technical issues

[0004] The embodiment provides an apparatus and method for efficiently processing point cloud data. The embodiment provides a method and apparatus for processing point cloud data to solve waiting time and encoding / decoding complexity.

[0005] The technical scope of the embodiments is not limited to the above-mentioned technical objectives, and can be extended to other technical objectives that can be inferred by those skilled in the art based on the entire content disclosed in this document.

[0006] Technical Solution

[0007] In one aspect of the present disclosure, a method for transmitting point cloud data may include: encoding the point cloud data; and transmitting a bit stream containing the point cloud data. In another aspect of the present disclosure, a method for receiving point cloud data may include: receiving a bit stream containing the point cloud data; and decoding the point cloud data.

[0008] Beneficial Effects

[0009] The apparatus and method according to the embodiments can efficiently process point cloud data.

[0010] The apparatus and method according to the embodiments can provide high-quality point cloud services.

[0011] The device and method according to the embodiment may provide point cloud content for providing general services such as VR service and self-driving service. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated into and constitute a part of this application, illustrate embodiments of the present disclosure, and together with this description, are used to explain the principles of the present disclosure. In order to better understand the various embodiments described below, reference should be made to the following description of the embodiments in conjunction with the accompanying drawings. The same reference numerals will be used throughout the drawings to refer to the same or similar parts.

[0013] Figure 1 An exemplary point cloud content providing system according to an embodiment is shown;

[0014] Figure 2 is a block diagram illustrating point cloud content providing operations according to an embodiment;

[0015] Figure 3 An exemplary point cloud encoder according to an embodiment is illustrated;

[0016] Figure 4 shows examples of octrees and occupancy codes according to an embodiment;

[0017] Figure 5 illustrates an example of point configuration in each LOD according to an embodiment;

[0018] Figure 6 illustrates an example of point configuration in each LOD according to an embodiment;

[0019] Figure 7 A point cloud decoder according to an embodiment is illustrated;

[0020] Figure 8 A transmitting device according to an embodiment is illustrated;

[0021] Figure 9 A receiving device according to an embodiment is illustrated;

[0022] Figure 10 An exemplary structure that can be operated in conjunction with a point cloud data transmitting / receiving method / apparatus according to an embodiment is illustrated;

[0023] Figure 11 An example of a road point cloud captured by a LiDAR device on a moving vehicle according to an embodiment is illustrated;

[0024] Figure 12 illustrates an example of a separated road point cloud according to an embodiment;

[0025] Figure 13 illustrates an example of road points captured by LiDAR according to an embodiment;

[0026] Figure 14 An example is illustrated in which a shape partially changes when a vehicle equipped with a LiDAR apparatus according to an embodiment does not move but an object around the vehicle moves;

[0027] Figure 15 An example of a point cloud acquired when a vehicle provided with a LiDAR device according to an embodiment turns right is illustrated;

[0028] Figure 16illustrates an example of a point cloud captured by a LiDAR device according to an embodiment;

[0029] Figure 17 An example in which a waypoint is changing according to a moving object according to an embodiment is illustrated;

[0030] Figure 18 illustrates an example of a waypoint being changed when there is only translational motion according to an embodiment;

[0031] Figure 19 illustrates an example where a road point is being changed when there is translational motion and rotational motion according to an embodiment;

[0032] Figure 20 A point cloud data transmitting device according to an embodiment is illustrated;

[0033] Figure 21 A point cloud data receiving device according to an embodiment is illustrated;

[0034] Figure 22 A bitstream including point cloud data and parameters according to an embodiment is illustrated;

[0035] Figure 23 illustrates a sequence parameter set (SPS) according to an embodiment;

[0036] Figure 24 illustrates a geometry parameter set (GPS) according to an embodiment;

[0037] Figure 25 illustrates the syntax of a tile parameter set (TPS) according to an embodiment;

[0038] Figure 26 illustrates the syntax of a geometry tile header (geometry data header) according to an embodiment;

[0039] Figure 27 shows the syntax of the Geometry PU header according to an embodiment;

[0040] Figure 28 A point cloud data transmission method according to an embodiment is shown; and

[0041] Figure 29 A point cloud data receiving method according to an embodiment is shown. DETAILED DESCRIPTION

[0042] Reference will now be made in detail to the preferred embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. The detailed description given below with reference to the accompanying drawings is intended to illustrate exemplary embodiments of the present disclosure, rather than to show the only embodiments that can be implemented according to the present disclosure. The following detailed description includes specific details in order to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure can be practiced without these specific details.

[0043] Although most of the terms used in the present disclosure are selected from common terms widely used in the art, some terms are arbitrarily selected by the applicant and their meanings are explained in detail in the following description as needed. Therefore, the present disclosure should be understood based on the intended meanings of the terms rather than their simple names or meanings.

[0044] Figure 1 An exemplary point cloud content providing system according to an embodiment is shown.

[0045] Figure 1 The point cloud content providing system shown may include a transmitting device 10000 and a receiving device 10004. The transmitting device 10000 and the receiving device 10004 can transmit and receive point cloud data via wired or wireless communication.

[0046] According to an embodiment, the point cloud data sending device 10000 can obtain and process point cloud video (or point cloud content) and send it. According to an embodiment, the sending device 10000 may include a fixed station, a base transceiver system (BTS), a network, an artificial intelligence (AI) device and / or system, a robot, an AR / VR / XR device and / or a server. According to an embodiment, the sending device 10000 may include a device configured to communicate with a base station and / or other wireless devices using a radio access technology (e.g., 5G new RAT (NR), long term evolution (LTE)), a robot, a vehicle, an AR / VR / XR device, a portable device, a home appliance, an Internet of Things (IoT) device, and an AI device / server.

[0047] According to the implementation manner, the sending device 10000 includes a point cloud video acquirer 10001, a point cloud video encoder 10002 and / or a transmitter (or communication module) 10003.

[0048] The point cloud video acquirer 10001 according to the embodiment acquires the point cloud video through a processing process such as capture, synthesis or generation. The point cloud video is a point cloud content represented by a point cloud, which is a set of points located in a 3D space and can be referred to as point cloud video data, point cloud data, etc. The point cloud video according to the embodiment may include one or more frames. A frame represents a still image / picture. Therefore, the point cloud video may include a point cloud image / frame / picture and may be referred to as a point cloud image, frame or picture.

[0049] The point cloud video encoder 10002 according to the embodiment encodes the acquired point cloud video data. The point cloud video encoder 10002 may encode the point cloud video data based on point cloud compression coding. The point cloud compression coding according to the embodiment may include geometry-based point cloud compression (G-PCC) coding and / or video-based point cloud compression (V-PCC) coding or next generation coding. The point cloud compression coding according to the embodiment is not limited to the above-mentioned embodiment. The point cloud video encoder 10002 may output a bit stream containing the encoded point cloud video data. The bit stream may include not only the encoded point cloud video data, but also signaling information related to the encoding of the point cloud video data.

[0050] The transmitter 10003 according to the embodiment transmits a bitstream containing encoded point cloud video data. The bitstream according to the embodiment is encapsulated in a file or a fragment (e.g., a stream fragment) and is transmitted via various networks such as a broadcast network and / or a broadband network. Although not shown in the figure, the transmitting device 10000 may include a packager (or a package module) configured to perform a packaging operation. According to an embodiment, the packager may be included in the transmitter 10003. According to an embodiment, the file or fragment may be sent to the receiving device 10004 via a network, or stored in a digital storage medium (e.g., USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.). The transmitter 10003 according to the embodiment can communicate with the receiving device 10004 (or the receiver 10005) via a 4G, 5G, 6G or other network by wire / wireless communication. In addition, the transmitter may perform necessary data processing operations according to the network system (e.g., a 4G, 5G or 6G communication network system). The transmitting device 10000 may transmit the encapsulated data in an on-demand manner.

[0051] The receiving device 10004 according to an embodiment includes a receiver 10005, a point cloud video decoder 10006 and / or a renderer 10007. According to an embodiment, the receiving device 10004 may include a device configured to communicate with a base station and / or other wireless devices using a radio access technology (e.g., 5G New RAT (NR), Long Term Evolution (LTE)), a robot, a vehicle, an AR / VR / XR device, a portable device, a home appliance, an Internet of Things (IoT) device, and an AI device / server.

[0052] The receiver 10005 according to the embodiment receives a bitstream containing point cloud video data or a file / segment encapsulated with a bitstream from a network or a storage medium. The receiver 10005 may perform necessary data processing according to a network system (e.g., a communication network system of 4G, 5G, 6G, etc.). The receiver 10005 according to the embodiment may decapsulate the received file / segment and output a bitstream. According to the embodiment, the receiver 10005 may include a decapsulator (or decapsulation module) configured to perform a decapsulation operation. The decapsulator may be implemented as an element (or component) separated from the receiver 10005.

[0053] The point cloud video decoder 10006 decodes the bitstream containing the point cloud video data. The point cloud video decoder 10006 can decode the point cloud video data according to the method in which the point cloud video data is encoded (for example, in the reverse process of the operation of the point cloud video encoder 10002). Therefore, the point cloud video decoder 10006 can decode the point cloud video data by performing point cloud decompression encoding (the reverse process of point cloud compression). The point cloud decompression encoding includes G-PCC encoding.

[0054] The renderer 10007 renders the decoded point cloud video data. The renderer 10007 may output the point cloud content by rendering not only the point cloud video data but also the audio data. According to an embodiment, the renderer 10007 may include a display configured to display the point cloud content. According to an embodiment, the display may be implemented as a separate device or component rather than being included in the renderer 10007.

[0055] The arrow indicated by the dotted line in the figure represents the transmission path of the feedback information obtained by the receiving device 10004. Feedback information is information that reflects the interactivity with the user who consumes the point cloud content, and includes information about the user (e.g., head orientation information, viewport information, etc.). Specifically, when the point cloud content is content for a service that requires interaction with the user (e.g., self-driving service, etc.), the feedback information may be provided to the content sender (e.g., sending device 10000) and / or the service provider. Depending on the embodiment, the feedback information may be used in the receiving device 10004 and the sending device 10000, or may not be provided.

[0056] The head orientation information according to the embodiment is information about the user's head position, orientation, angle, movement, etc. The receiving device 10004 according to the embodiment may calculate the viewport information based on the head orientation information. The viewport information may be information about the area of ​​the point cloud video that the user is watching. The viewpoint is the point through which the user watches the point cloud video, and may refer to the center point of the viewport area. That is, the viewport is an area centered on the viewpoint, and the size and shape of the area may be determined by the field of view (FOV). Therefore, in addition to the head orientation information, the receiving device 10004 may also extract the viewport information based on the vertical or horizontal FOV supported by the device. In addition, the receiving device 10004 performs gaze analysis, etc. to check the way the user consumes the point cloud, the area where the user gazes in the point cloud video, the gaze time, etc. According to the embodiment, the receiving device 10004 may send feedback information including the gaze analysis result to the sending device 10000. The feedback information according to the embodiment may be obtained during the rendering and / or display process. The feedback information according to the embodiment may be obtained by one or more sensors included in the receiving device 10004. According to an embodiment, the feedback information may be obtained by the renderer 10007 or a separate external element (or device, component, etc.). Figure 1 The dotted line in represents the process of sending the feedback information obtained by the renderer 10007. The point cloud content providing system can process (encode / decode) the point cloud data based on the feedback information. Therefore, the point cloud video data decoder 10006 can perform a decoding operation based on the feedback information. The receiving device 10004 can send the feedback information to the sending device 10000. The sending device 10000 (or the point cloud video data encoder 10002) can perform an encoding operation based on the feedback information. Therefore, the point cloud content providing system can efficiently process necessary data (for example, point cloud data corresponding to the user's head position) based on the feedback information instead of processing (encoding / decoding) the entire point cloud data, and provide the point cloud content to the user.

[0057] According to an embodiment, the transmitting device 10000 may be referred to as an encoder, a transmitting device, a transmitter, etc., and the receiving device 10004 may be referred to as a decoder, a receiving device, a receiver, etc.

[0058] According to the implementation Figure 1 The point cloud data processed in the point cloud content providing system (through a series of processes of acquisition / encoding / transmission / decoding / rendering) may be referred to as point cloud content data or point cloud video data. According to an embodiment, point cloud content data may be used as a concept covering metadata or signaling information related to point cloud data.

[0059] Figure 1 The elements of the point cloud content providing system shown may be implemented by hardware, software, a processor and / or a combination thereof.

[0060] Figure 2 is a block diagram illustrating point cloud content providing operations according to an embodiment.

[0061] Figure 2 The block diagram shows Figure 1 The operation of the point cloud content providing system described in . As described above, the point cloud content providing system can process point cloud data based on point cloud compression coding (e.g., G-PCC).

[0062] According to an embodiment, a point cloud content providing system (e.g., a point cloud sending device 10000 or a point cloud video acquirer 10001) can acquire a point cloud video (20000). The point cloud video is represented by a point cloud belonging to a coordinate system for expressing a 3D space. According to an embodiment, a point cloud video may include a Ply (Polygon file format or Stanford Triangle format) file. When a point cloud video has one or more frames, the acquired point cloud video may include one or more Ply files. A Ply file contains point cloud data such as point geometry and / or attributes. Geometry includes the position of a point. The position of each point may be represented by a parameter (e.g., the value of the X, Y, and Z axes) representing a three-dimensional coordinate system (e.g., a coordinate system consisting of X, Y, and Z axes). Attributes include attributes of the point (e.g., information about the texture, color (YCbCr or RGB), reflectivity r, transparency, etc. of each point). A point has one or more attributes. For example, a point may have a color attribute or two attributes, color and reflectivity. According to an embodiment, geometry may be referred to as position, geometry information, geometry data, position information, position data, etc., and attributes may be referred to as attributes, attribute information, attribute data, etc. The point cloud content providing system (e.g., the point cloud transmitting device 10000 or the point cloud video acquirer 10001) may obtain point cloud data from information related to a point cloud video acquisition process (e.g., depth information, color information, etc.).

[0063] According to an embodiment, a point cloud content providing system (e.g., a transmitting device 10000 or a point cloud video encoder 10002) may encode point cloud data (20001). The point cloud content providing system may encode point cloud data based on point cloud compression coding. As described above, point cloud data may include geometric information and attribute information about points. Therefore, the point cloud content providing system may perform geometric coding for encoding geometry and output a geometric bitstream. The point cloud content providing system may perform attribute coding for encoding attributes and output an attribute bitstream. According to an embodiment, the point cloud content providing system may perform attribute coding based on geometric coding. The geometric bitstream and the attribute bitstream according to the embodiment may be multiplexed and output as one bitstream. The bitstream according to the embodiment may also include signaling information related to geometric coding and attribute coding.

[0064] According to the embodiment, the point cloud content providing system (eg, the transmitting device 10000 or the transmitter 10003) may transmit the encoded point cloud data (20002). Figure 1 As shown, the encoded point cloud data can be represented by a geometry bitstream and an attribute bitstream. In addition, the encoded point cloud data can be sent in the form of a bitstream together with signaling information related to the encoding of the point cloud data (e.g., signaling information related to geometry encoding and attribute encoding). The point cloud content providing system can encapsulate the bitstream carrying the encoded point cloud data and send it in the form of a file or a fragment.

[0065] The point cloud content providing system (eg, receiving device 10004 or receiver 10005) according to the embodiment may receive a bit stream containing encoded point cloud data. In addition, the point cloud content providing system (eg, receiving device 10004 or receiver 10005) may demultiplex the bit stream.

[0066] The point cloud content providing system (e.g., the receiving device 10004 or the point cloud video decoder 10005) may decode the encoded point cloud data (e.g., geometry bitstream, attribute bitstream) sent in the bitstream. The point cloud content providing system (e.g., the receiving device 10004 or the point cloud video decoder 10005) may decode the point cloud video data based on the signaling information related to the encoding of the point cloud video data contained in the bitstream. The point cloud content providing system (e.g., the receiving device 10004 or the point cloud video decoder 10005) may decode the geometry bitstream to reconstruct the position (geometry) of the point. The point cloud content providing system may reconstruct the attributes of the point by decoding the attribute bitstream based on the reconstructed geometry. The point cloud content providing system (e.g., the receiving device 10004 or the point cloud video decoder 10005) may reconstruct the point cloud video based on the position according to the reconstructed geometry and the decoded attributes.

[0067] According to an embodiment, a point cloud content providing system (e.g., receiving device 10004 or renderer 10007) may render decoded point cloud data (20004). The point cloud content providing system (e.g., receiving device 10004 or renderer 10007) may use various rendering methods to render the geometry and attributes decoded by the decoding process. Points in the point cloud content may be rendered as vertices with a specific thickness, cubes with a specific minimum size centered at the corresponding vertex position, or circles centered at the corresponding vertex position. All or part of the rendered point cloud content is provided to the user through a display (e.g., a VR / AR display, a general display, etc.).

[0068] The point cloud content providing system (e.g., receiving device 10004) according to the embodiment may obtain feedback information (20005). The point cloud content providing system may encode and / or decode the point cloud data based on the feedback information. Feedback Information and Operations of the Point Cloud Content Providing System According to the Embodiment and Reference Figure 1 The feedback information and operations described are the same, so their detailed description is omitted.

[0069] Figure 3 An exemplary point cloud encoder according to an implementation is shown.

[0070] Figure 3 Show Figure 1 An example of a point cloud video encoder 10002. The point cloud encoder reconstructs and encodes point cloud data (e.g., the location and / or attributes of the points) to adjust the quality of the point cloud content (e.g., lossless, lossy, or near-lossless) according to network conditions or applications. When the total size of the point cloud content is large (e.g., 60 Gbps of point cloud content given for 30 fps), the point cloud content providing system may not be able to stream the content in real time. Therefore, the point cloud content providing system may reconstruct the point cloud content based on the maximum target bit rate to provide the point cloud content according to the network environment, etc.

[0071] As reference Figure 1 and Figure 2 As described, the point cloud encoder can perform geometry encoding and attribute encoding. Geometry encoding is performed before attribute encoding.

[0072] According to an embodiment, the point cloud encoder includes a coordinate transformer (transforming coordinates) 30000, a quantizer (quantizing and removing points (voxelization)) 30001, an octree analyzer (analyzing octree) 30002 and a surface approximation analyzer (analyzing surface approximation) 30003, an arithmetic encoder (arithmetic coding) 30004, a geometry reconstructor (reconstructing geometry) 30005, a color transformer (transforming color) 30006, an attribute transformer (transforming attributes) 30007, a RAHT transformer (RAHT) 30008, an LOD generator (generating LOD) 30009, a lifting transformer (lifting) 30010, a coefficient quantizer (quantizing coefficients) 30011 and / or an arithmetic encoder (arithmetic coding) 30012.

[0073] The coordinate transformer 30000, the quantizer 30001, the octree analyzer 30002, the surface approximation analyzer 30003, the arithmetic encoder 30004 and the geometry reconstructor 30005 may perform geometry coding. The geometry coding according to the embodiment may include octree geometry coding, prediction tree geometry coding, direct coding, triplet geometry coding and entropy coding. Direct coding and triplet geometry coding are applied selectively or in combination. The geometry coding is not limited to the above examples.

[0074] As shown in the figure, the coordinate transformer 30000 according to the embodiment receives the position and transforms it into coordinates. For example, the position can be transformed into position information in a three-dimensional space (e.g., a three-dimensional space represented by an XYZ coordinate system). The position information in the three-dimensional space according to the embodiment can be referred to as geometric information.

[0075] The quantizer 30001 according to the embodiment quantizes the geometry. For example, the quantizer 30001 may quantize the points based on the minimum position value of all points (e.g., the minimum value on each of the X, Y, and Z axes). The quantizer 30001 performs a quantization operation: the difference between the minimum position value and the position value of each point is multiplied by a preset quantization scaling value, and then the nearest integer value is found by rounding the value obtained by the multiplication. Therefore, one or more points may have the same quantized position (or position value). The quantizer 30001 according to the embodiment performs voxelization based on the quantized position to reconstruct the quantized point. As in the case of a pixel (the smallest unit containing 2D image / video information), a point of a point cloud content (or 3D point cloud video) according to the embodiment may be included in one or more voxels. As a composite of volume and pixel, the term voxel refers to a 3D cubic space generated when a 3D space is divided into units (unit = 1.0) based on an axis representing the 3D space (e.g., X-axis, Y-axis, and Z-axis). The quantizer 30001 may match a point group in a 3D space with a voxel. According to an embodiment, a voxel may include only one point. According to an embodiment, a voxel may include one or more points. In order to represent a voxel as a point, the position of the center of the voxel may be set based on the position of one or more points included in the voxel. In this case, the properties of all positions included in a voxel may be combined and assigned to the voxel.

[0076] The octree analyzer 30002 according to the embodiment performs octree geometry encoding (or octree encoding) to present voxels in an octree structure. The octree structure represents points matched to voxels based on an octal tree structure.

[0077] The surface approximation analyzer 30003 according to an embodiment may analyze and approximate an octree. The octree analysis and approximation according to an embodiment is a process of analyzing a region including a plurality of points to efficiently provide an octree and voxelization.

[0078] The arithmetic encoder 30004 according to the embodiment performs entropy coding on the octree and / or the approximate octree. For example, the coding scheme includes arithmetic coding. As a result of the coding, a geometry bitstream is generated.

[0079] The color converter 30006, the attribute converter 30007, the RAHT converter 30008, the LOD generator 30009, the lifting converter 30010, the coefficient quantizer 30011 and / or the arithmetic encoder 30012 perform attribute encoding. As described above, a point may have one or more attributes. The attribute encoding according to the embodiment is also applied to the attributes of a point. However, when the attribute (e.g., color) includes one or more elements, the attribute encoding is applied independently to each element. The attribute encoding according to the embodiment includes color transform encoding, attribute transform encoding, regional adaptive hierarchical transform (RAHT) encoding, interpolation-based hierarchical nearest neighbor prediction (prediction transform) encoding, and interpolation-based hierarchical nearest neighbor prediction (lifting transform) encoding with an update / lifting step. Depending on the point cloud content, the above-mentioned RAHT encoding, prediction transform encoding, and lifting transform encoding may be selectively used, or a combination of one or more encoding schemes may be used. The attribute encoding according to the embodiment is not limited to the above examples.

[0080] The color converter 30006 according to an embodiment performs color conversion encoding to convert the color value (or texture) included in the attribute. For example, the color converter 30006 can convert the format of the color information (e.g., from RGB to YCbCr). Optionally, the operation of the color converter 30006 according to an embodiment can be applied according to the color value included in the attribute.

[0081] The geometry reconstructor 30005 according to an embodiment reconstructs (decompresses) an octree and / or an approximate octree. The geometry reconstructor 30005 reconstructs an octree / voxel based on the result of analyzing the point distribution. The reconstructed octree / voxel may be referred to as a reconstructed geometry (restored geometry).

[0082] The attribute transformer 30007 according to the embodiment performs attribute transformation to transform the attribute based on the reconstructed geometry and / or the position where the geometry encoding is not performed. As described above, since the attribute depends on the geometry, the attribute transformer 30007 can transform the attribute based on the reconstructed geometry information. For example, based on the position value of the point included in the voxel, the attribute transformer 30007 can transform the attribute of the point at the position. As described above, when the center position of the voxel is set based on the position of one or more points included in the voxel, the attribute transformer 30007 transforms the attribute of one or more points. When triplet geometry encoding is performed, the attribute transformer 30007 can transform the attribute based on the triplet geometry encoding.

[0083] The attribute converter 30007 can perform attribute conversion by calculating the average of the attributes or attribute values ​​(e.g., the color or reflectivity of each point) of the neighboring points within a specific position / radius from the center position (or position value) of each voxel. The attribute converter 30007 can apply weights according to the distance from the center to each point when calculating the average. Therefore, each voxel has a position and a calculated attribute (or attribute value).

[0084] The attribute converter 30007 can search for neighboring points within a specific position / radius from the center position of each voxel based on a KD tree or a Morton code. The KD tree is a binary search tree and supports a data structure capable of managing points based on position, so that a nearest neighbor search (NNS) can be performed quickly. The Morton code is generated by presenting the coordinates (e.g., (x, y, z)) representing the 3D positions of all points as bit values ​​and mixing the bits. For example, when the coordinates representing the point position are (5, 9, 1), the bit values ​​of the coordinates are (0101, 1001, 0001). Mixing the bit values ​​in the order of z, y, and x according to the bit index produces 010001000111. This value is represented as a decimal number 1095. That is, the Morton code value of the point with coordinates (5, 9, 1) is 1095. The attribute converter 30007 can sort the points based on the Morton code value and perform NNS through a depth-first traversal process. After the attribute transform operation, KD tree or Morton code is used when NNS is needed in another transform process for attribute encoding.

[0085] As shown, the transformed attributes are input to the RAHT transformer 30008 and / or the LOD generator 30009.

[0086] The RAHT transformer 30008 according to an embodiment performs RAHT encoding for predicting attribute information based on the reconstructed geometric information. For example, the RAHT transformer 30008 may predict attribute information of a higher-level node in the octree based on attribute information associated with a lower-level node in the octree.

[0087] The LOD generator 30009 according to an embodiment generates a level of detail (LOD) to perform predictive transform coding. The LOD according to an embodiment is the degree of detail of the point cloud content. As the LOD value decreases, the details of the point cloud content are degraded. As the LOD value increases, the details of the point cloud content are enhanced. Points can be classified by LOD.

[0088] The lifting transformer 30010 according to an embodiment performs lifting transform coding that transforms point cloud attributes based on weights. As described above, lifting transform coding can be optionally applied.

[0089] The coefficient quantizer 30011 according to the embodiment quantizes the attribute of the attribute encoding based on the coefficient.

[0090] The arithmetic encoder 30012 according to the embodiment encodes quantized properties based on arithmetic coding.

[0091] Although not shown in the figure, Figure 3 The elements of the point cloud encoder may be implemented by hardware, software, firmware, or a combination thereof including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device. The one or more processors may execute the above Figure 3 At least one of the operations and / or functions of the elements of the point cloud encoder. In addition, one or more processors may be operable or executable to perform Figure 3 The software program and / or instruction set for the operation and / or function of the elements of the point cloud encoder. According to the embodiment, one or more memories may include high-speed random access memory, or include non-volatile memory (e.g., one or more disk storage devices, flash memory devices or other non-volatile solid-state memory devices).

[0092] Figure 4 Examples of octrees and occupancy codes are shown according to an embodiment.

[0093] As reference Figures 1 to 3 As described, the point cloud content providing system (point cloud video encoder 10002) or the point cloud encoder (e.g., octree analyzer 30002) performs octree geometry encoding (or octree encoding) based on the octree structure to efficiently manage the area and / or position of voxels.

[0094] Figure 4 The upper part of FIG. 1 shows the octree structure. The 3D space of the point cloud content according to the embodiment is represented by the axes (eg, X-axis, Y-axis, and Z-axis) of the coordinate system. d ,2 d ,2 d ) to create an octree structure. Here, 2d can be set to the value of the smallest bounding box that surrounds all points of the point cloud content (or point cloud video). Here, d represents the depth of the octree. The value of d is determined in the following formula. In the following formula, (x int n ,y int n ,z int n ) represents the position (or position value) of the quantized point.

[0095] d=Ceil(Log2(Max(x_n^int,y_n^int,z_n^in,n=1,…,N)+1))

[0096] like Figure 4 As shown in the middle of the upper part of , the entire 3D space can be divided into eight spaces according to the partition. Each divided space is represented by a cube with six faces. Figure 4 As shown in the upper right part of , each of the eight spaces is divided again based on the axis of the coordinate system (e.g., X-axis, Y-axis, and Z-axis). Therefore, each space is divided into eight smaller spaces. The divided smaller spaces are also represented by cubes with six faces. This division scheme is applied until the leaf nodes of the octree become voxels.

[0097] Figure 4 The lower part shows the octree occupancy code. The occupancy code of the octree is generated to indicate whether each of the eight divided spaces generated by dividing one space contains at least one point. Therefore, a single occupancy code is represented by eight child nodes. Each child node represents the occupancy of the divided space, and the child node has a 1-bit value. Therefore, the occupancy code is represented as an 8-bit code. That is, when the space corresponding to the child node contains at least one point, the node is assigned a value of 1. When the space corresponding to the child node does not contain a point (the space is empty), the node is assigned a value of 0. Since Figure 4 The occupancy code shown is 00100001, thus indicating that the spaces corresponding to the third and eighth subnodes among the eight subnodes each contain at least one point. As shown in the figure, each of the third and eighth subnodes has eight subnodes, and the subnodes are represented by 8-bit occupancy codes. The accompanying drawings show that the occupancy code of the third subnode is 10000111, and the occupancy code of the eighth subnode is 01001111. The point cloud encoder (e.g., arithmetic encoder 30004) according to an embodiment may perform entropy encoding on the occupancy code. In order to increase compression efficiency, the point cloud encoder may perform intra / inter encoding on the occupancy code. The receiving device (e.g., receiving device 10004 or point cloud video decoder 10006) according to an embodiment reconstructs the occupancy code based on the occupancy code.

[0098] According to an embodiment, a point cloud encoder (e.g., Figure 4 The point cloud encoder or octree analyzer 30002) can perform voxelization and octree encoding to store point locations. However, points are not always evenly distributed in the 3D space, so there may be specific areas where fewer points exist. Therefore, performing voxelization on the entire 3D space is inefficient. For example, when a specific area contains few points, voxelization does not need to be performed in the specific area.

[0099] Therefore, for the above-mentioned specific area (or nodes other than the leaf nodes of the octree), the point cloud encoder according to the embodiment may skip voxelization and perform direct encoding to directly encode the point positions included in the specific area. The coordinates of the directly encoded points according to the embodiment are called direct coding mode (DCM). The point cloud encoder according to the embodiment may also perform triplet geometry encoding based on the surface model, which is to reconstruct the point positions in the specific area (or node) based on voxels. Triplet geometry encoding is a geometry encoding that represents an object as a series of triangular meshes. Therefore, the point cloud decoder can generate a point cloud from the mesh surface. Direct encoding and triplet geometry encoding according to the embodiment can be performed selectively. In addition, direct encoding and triplet geometry encoding according to the embodiment can be performed in combination with octree geometry encoding (or octree encoding).

[0100] In order to perform direct encoding, the option of using direct mode to apply direct encoding should be enabled. The node to which direct encoding is to be applied is not a leaf node, and there should be points less than a threshold value within a specific node. In addition, the total number of points to which direct encoding is to be applied should not exceed a preset threshold. When the above conditions are met, the point cloud encoder (or arithmetic encoder 30004) according to the embodiment can perform entropy encoding on the point position (or position value).

[0101] A point cloud encoder (e.g., surface approximation analyzer 30003) according to an embodiment may determine a specific level of the octree (a level less than the depth d of the octree), and may use a surface model starting from this level to perform triplet geometry encoding to reconstruct point positions in the node area based on voxels (triplet mode). A point cloud encoder according to an embodiment may specify a level to which triplet geometry encoding is to be applied. For example, when a specific level is equal to the depth of the octree, the point cloud encoder does not operate in triplet mode. In other words, a point cloud encoder according to an embodiment may operate in triplet mode only when the specified level is less than the depth value of the octree. A 3D cubic area of ​​a node at a specified level according to an embodiment is referred to as a block. A block may include one or more voxels. A block or voxel may correspond to a block. Geometry is represented as a surface within each block. A surface according to an embodiment may intersect each edge of a block at most once.

[0102] A block has 12 edges, so there are at least 12 intersections in a block. Each intersection is called a vertex. When there is at least one occupied voxel adjacent to the edge among all blocks sharing the edge, the vertex existing along the edge is detected. An occupied voxel according to an embodiment refers to a voxel containing a point. The vertex position detected along the edge is the average position of the edge of all voxels adjacent to the edge among all blocks along the shared edge.

[0103] Once the vertex is detected, the point cloud encoder according to the embodiment can perform entropy encoding on the start point (x, y, z) of the edge, the direction vector (Δx, Δy, Δz) of the edge, and the vertex position value (relative position value within the edge). When triplet geometry encoding is applied, the point cloud encoder according to the embodiment (e.g., geometry reconstructor 30005) can generate restored geometry (reconstructed geometry) by performing triangle reconstruction, upsampling, and voxelization processes.

[0104] The vertices located at the edges of the block determine the surface passing through the block. The surface according to an embodiment is a non-planar polygon. In the triangle reconstruction process, the surface represented by the triangle is reconstructed based on the starting point of the edge, the direction vector of the edge and the position value of the vertex. The triangle reconstruction process is performed by i) calculating the centroid value of each vertex, ii) subtracting the center value from each vertex value, and iii) estimating the sum of the squares of the values ​​obtained by subtraction.

[0105]

[0106] The minimum value of the sum is estimated, and the projection process is performed according to the axis with the minimum value. For example, when the element x is the smallest, each vertex is projected on the x-axis relative to the center of the block and projected on the (y,z) plane. When the value obtained by the projection on the (y,z) plane is (ai,bi), the value of θ is estimated by atan2(bi,ai), and the vertices are sorted based on the value of θ. The following table shows the vertex combination for creating a triangle according to the number of vertices. The vertices are sorted from 1 to n. The following table shows that for four vertices, two triangles can be constructed according to the vertex combination. The first triangle can be composed of vertices 1, 2, and 3 among the sorted vertices, and the second triangle can be composed of vertices 3, 4, and 1 among the sorted vertices.

[0107] A triangle formed from vertices ordered 1,…,n

[0108] n triangle

[0109] 3(1,2,3)

[0110] 4(1,2,3),(3,4,1)

[0111] 5(1,2,3),(3,4,5),(5,1,3)

[0112] 6(1,2,3),(3,4,5),(5,6,1),(1,3,5)

[0113] 7(1,2,3),(3,4,5),(5,6,7),(7,1,3),(3,5,7)

[0114] 8(1,2,3),(3,4,5),(5,6,7),(7,8,1),(1,3,5),(5,7,1)

[0115] 9(1,2,3),(3,4,5),(5,6,7),(7,8,9),(9,1,3),(3,5,7),(7,9,3)

[0116] 10(1,2,3),(3,4,5),(5,6,7),(7,8,9),(9,10,1),(1,3,5),(5,7,9),(9,1,5)

[0117] 11(1,2,3),(3,4,5),(5,6,7),(7,8,9),(9,10,11),(11,1,3),(3,5,7),(7,9,11),(11,3,7)

[0118] 12(1,2,3),(3,4,5),(5,6,7),(7,8,9),(9,10,11),(11,12,1),(1,3,5),(5,7,9),(9,11,1),(1,5,9)

[0119] An upsampling process is performed to add points in the middle along the edges of the triangle, and voxelization is performed. The added points are generated based on the upsampling factor and the width of the block. The added points are called refined vertices. The point cloud encoder according to an embodiment may voxelize the refined vertices. In addition, the point cloud encoder may perform attribute encoding based on the voxelized position (or position value).

[0120] Figure 5 An example of point configuration in each LOD according to an embodiment is shown.

[0121] As reference Figures 1 to 4 As described, the encoded geometry is reconstructed (decompressed) before performing attribute encoding. When direct encoding is applied, the geometry reconstruction operation may include changing the placement of the directly encoded points (e.g., placing the directly encoded points in front of the point cloud data). When triplet geometry encoding is applied, the geometry reconstruction process is performed by triangle reconstruction, upsampling, and voxelization. Since the attributes depend on the geometry, attribute encoding is performed based on the reconstructed geometry.

[0122] The point cloud encoder (e.g., LOD generator 30009) can classify (or reorganize) the points by LOD. The figure shows the point cloud content corresponding to the LOD. The leftmost screen in the figure shows the original point cloud content. The second screen from the left in the figure shows the distribution of points in the lowest LOD, and the rightmost screen in the figure shows the distribution of points in the highest LOD. That is, the points in the lowest LOD are sparsely distributed, and the points in the highest LOD are densely distributed. That is, as the LOD increases in the direction indicated by the arrow indicated at the bottom of the figure, the space (or distance) between the points becomes narrower.

[0123] Figure 6 An example of point configuration for each LOD is shown according to an embodiment.

[0124] As reference Figures 1 to 5 As described, the point cloud content providing system or point cloud encoder (eg, point cloud video encoder 10002, Figure 3 The point cloud encoder or LOD generator 30009) can generate LOD. LOD is generated by reorganizing points into a set of refinement levels according to the set LOD distance value (or Euclidean distance set). The LOD generation process is performed not only by the point cloud encoder, but also by the point cloud decoder.

[0125] Figure 6 The upper part of shows examples of points (P0 to P9) of point cloud content distributed in 3D space. Figure 6 In , the original order represents the order of points P0 to P9 before LOD generation. Figure 6 In the LOD-based order, the order of points generated according to the LOD is shown. Points are reorganized by LOD. In addition, a high LOD contains points belonging to a lower LOD. Figure 6 As shown, LOD0 contains P0, P5, P4, and P2. LOD1 contains the points of LOD0, P1, P6, and P3. LOD2 contains the points of LOD0, the points of LOD1, P9, P8, and P7.

[0126] As reference Figure 3 As described, the point cloud encoder according to the embodiment may selectively or in combination perform prediction transform coding, lifting transform coding, and RAHT transform coding.

[0127] The point cloud encoder according to the embodiment may generate a predictor for a point to perform predictive transform coding for setting a prediction attribute (or a prediction attribute value) of each point. That is, N predictors may be generated for N points. The predictor according to the embodiment may calculate a weight (=1 / distance) based on the LOD value of each point, index information about neighboring points that exist within a set distance of each LOD, and the distance to the neighboring point.

[0128] The predicted attribute (or attribute value) according to the embodiment is set to the average of the values ​​obtained by multiplying the attribute (or attribute value) (e.g., color, reflectivity, etc.) of the neighboring points set in the predictor of each point by the weight (or weight value) calculated based on the distance to each neighboring point. The point cloud encoder (e.g., coefficient quantizer 30011) according to the embodiment may quantize and inverse quantize the residual (which may be referred to as residual attribute, residual attribute value or attribute prediction residual, attribute residual) obtained by subtracting the predicted attribute (attribute value) from the attribute (attribute value) of each point. The quantization process is configured as shown in the following table.

[0129] Table. Attribute prediction residual quantization pseudo code

[0130] int PCCQuantization(int value,int quantStep){

[0131] if(value>=0){

[0132] return floor(value / quantStep+1.0 / 3.0);

[0133] }else{

[0134] return-floor(-value / quantStep+1.0 / 3.0);

[0135] }

[0136] }

[0137] Table. Pseudocode for inverse quantization of attribute prediction residuals

[0138] int PCCInverseQuantization(int value,int quantStep){

[0139] if(quantStep==0){

[0140] return value;

[0141] }else{

[0142] return value*quantStep;

[0143] }

[0144] }

[0145] When the predictor of each point has a neighboring point, the point cloud encoder (e.g., the arithmetic encoder 30012) according to the embodiment may perform entropy encoding on the quantized and inverse quantized residual values ​​as described above. When the predictor of each point has no neighboring point, the point cloud encoder (e.g., the arithmetic encoder 30012) according to the embodiment may perform entropy encoding on the attributes of the corresponding point without performing the above operation.

[0146] The point cloud encoder (e.g., lifting transformer 30010) according to an embodiment may generate a predictor for each point, set the calculated LOD and register neighbor points in the predictor, and set weights according to the distance to the neighbor points to perform lifting transform encoding. The lifting transform encoding according to an embodiment is similar to the above-mentioned predictive transform encoding, but is different in that weights are cumulatively applied to attribute values. The process of cumulatively applying weights to attribute values ​​according to an embodiment is configured as follows.

[0147] 1) Create an array quantized weight (QW) for storing weight values ​​of each point. The initial value of all elements of QW is 1.0. Multiply the QW value of the predictor index of the neighbor node registered in the predictor by the weight of the predictor of the current point, and add the values ​​obtained by multiplication.

[0148] 2) Boosting prediction process: The value obtained by multiplying the attribute value of the point by the weight is subtracted from the existing attribute value to calculate the predicted attribute value.

[0149] 3) Create temporary arrays called updateweight and update, and initialize the temporary arrays to zero.

[0150] 4) Accumulate the weight calculated by multiplying the weight calculated for all predictors by the weight stored in the QW corresponding to the predictor index with the updateweight array as the index of the neighbor node. Accumulate the value obtained by multiplying the attribute value of the neighbor node index by the calculated weight with the update array.

[0151] 5) Boosting update process: Divide the attribute values ​​of the update array of all predictors by the weight value of the updateweight array of the predictor index, and add the existing attribute value to the value obtained by the division.

[0152] 6) For all predictors, the predicted attribute is calculated by multiplying the attribute value updated by the lifting update process by the weight (stored in QW) updated by the lifting prediction process. The point cloud encoder (e.g., coefficient quantizer 30011) according to the embodiment quantizes the predicted attribute value. In addition, the point cloud encoder (e.g., arithmetic encoder 30012) performs entropy encoding on the quantized attribute value.

[0153] A point cloud encoder according to an embodiment (e.g., RAHT transformer 30008) may perform RAHT transform coding, in which attributes associated with nodes at a lower level in an octree are used to predict attributes of nodes at a higher level. RAHT transform coding is an example of attribute intra-frame coding by scanning backward through an octree. A point cloud encoder according to an embodiment scans the entire area starting from voxels and repeats a merging process of merging voxels into larger blocks at each step until a root node is reached. The merging process according to an embodiment is performed only on occupied nodes. The merging process is not performed on empty nodes. The merging process is performed on the upper node directly above the empty node.

[0154] The following formula represents the RAHT transformation matrix. In this formula, Represents the average attribute value of voxels at level l. Can be based on and to calculate. and The weight is and

[0155]

[0156] here, Is the low-pass value and is used during the next highest level of merging. Denotes the high-pass coefficient. The high-pass coefficient of each step is quantized and subjected to entropy coding (eg, encoded by arithmetic encoder 300012). The weight is calculated as pass and Create the root node as follows.

[0157]

[0158] The values ​​of gDC are also quantized and subjected to entropy decoding like the high-pass coefficients.

[0159] Figure 7 A point cloud decoder according to an embodiment is shown.

[0160] Figure 7 The point cloud decoder shown is an example of a point cloud decoder and can perform a decoding operation, which is Figures 1 to 6 The inverse process of the encoding operation of the point cloud encoder shown.

[0161] As reference Figure 1 and Figure 6 As described, the point cloud decoder can perform geometry decoding and attribute decoding. Geometry decoding is performed before attribute decoding.

[0162] According to an embodiment, a point cloud decoder includes an arithmetic decoder (arithmetic decoding) 7000, an octree synthesizer (synthesized octree) 7001, a surface approximation synthesizer (synthesized surface approximation) 7002 and a geometry reconstructor (reconstructed geometry) 7003, an inverse coordinate transformer (inverse transformed coordinates) 7004, an arithmetic decoder (arithmetic decoding) 7005, an inverse quantizer (inverse quantization) 7006, a RAHT transformer 7007, an LOD generator (generate LOD) 7008, an inverse lifting (inverse lifting) 7009 and / or an inverse color transformer (inverse transformed color) 7010.

[0163] The arithmetic decoder 7000, the octree synthesizer 7001, the surface approximation synthesizer 7002, the geometry reconstructor 7003, and the coordinate inverse transformer 7004 may perform geometry decoding. The geometry decoding according to the embodiment may include direct decoding and triplet geometry decoding. Direct encoding and triplet geometry decoding are selectively applied. The geometry decoding is not limited to the above examples, and as a reference Figures 1 to 6 The inverse process of the geometric encoding described is performed.

[0164] The arithmetic decoder 7000 according to the embodiment decodes the received geometry bitstream based on arithmetic coding. The operation of the arithmetic decoder 7000 corresponds to the inverse process of the arithmetic encoder 30004.

[0165] The octree synthesizer 7001 according to the embodiment may generate an octree by acquiring an occupancy code (or information about the geometry obtained as a result of decoding) from the decoded geometry bitstream. Figures 1 to 6 Describe the configuration in detail.

[0166] When triplet geometry encoding is applied, the surface approximation synthesizer 7002 according to an embodiment may synthesize a surface based on the decoded geometry and / or the generated octree.

[0167] According to an embodiment, the geometry reconstructor 7003 may regenerate the geometry based on the surface and / or the decoded geometry. Figures 1 to 9 As described, direct coding and triplet geometry coding are selectively applied. Therefore, the geometry reconstructor 7003 directly imports the position information about the points to which direct coding is applied and adds them. When triplet geometry coding is applied, the geometry reconstructor 7003 can reconstruct the geometry by performing the reconstruction operations (e.g., triangle reconstruction, upsampling, and voxelization) of the geometry reconstructor 30005. Details and References Figure 6 Those described are the same, so their description is omitted. The reconstructed geometry may include a point cloud screen or frame that does not contain attributes.

[0168] The coordinate inverse transformer 7004 according to an embodiment may acquire a point position by transforming the coordinates based on the reconstructed geometry.

[0169] The arithmetic decoder 7005, the inverse quantizer 7006, the RAHT transformer 7007, the LOD generator 7008, the inverse lifting 7009 and / or the color inverse transformer 7010 may perform reference Figure 6 Attribute decoding described. Attribute decoding according to an embodiment includes regional adaptive hierarchical transform (RAHT) decoding, hierarchical nearest neighbor prediction (prediction transform) decoding based on interpolation, and hierarchical nearest neighbor prediction (lifting transform) decoding based on interpolation with an update / lifting step. The above three decoding schemes can be used selectively, or a combination of one or more decoding schemes can be used. Attribute decoding according to an embodiment is not limited to the above examples.

[0170] The arithmetic decoder 7005 according to the embodiment decodes the attribute bit stream through arithmetic coding.

[0171] The inverse quantizer 7006 according to the embodiment inversely quantizes information about the decoded attribute bitstream or the attribute obtained as a result of decoding, and outputs the inversely quantized attribute (or attribute value). The inverse quantization may be selectively applied based on the attribute encoding of the point cloud encoder.

[0172] According to an embodiment, the RAHT transformer 7007, the LOD generator 7008 and / or the inverse lifter 7009 may process the reconstructed geometry and inverse quantized attributes. As described above, the RAHT transformer 7007, the LOD generator 7008 and / or the inverse lifter 7009 may selectively perform a decoding operation corresponding to the encoding of the point cloud encoder.

[0173] The color inverse transformer 7010 according to an embodiment performs inverse transform encoding to inversely transform a color value (or texture) included in the decoded attribute. The operation of the color inverse transformer 7010 may be selectively performed based on the operation of the color transformer 30006 of the point cloud encoder.

[0174] Although not shown in the figure, Figure 7 The elements of the point cloud decoder may be implemented by hardware, software, firmware, or a combination thereof including one or more processors or integrated circuits configured to communicate with one or more memories included in the point cloud providing device. The one or more processors may execute the above Figure 7 At least one or more of the operations and / or functions of the elements of the point cloud decoder. In addition, one or more processors may be operable or executed to perform Figure 7 A software program and / or instruction set for the operation and / or functionality of the elements of a point cloud decoder.

[0175] Figure 8 A transmitting device according to an embodiment is shown.

[0176] Figure 8The sending device shown is Figure 1 The sending device 10000 (or Figure 3 Example of a point cloud encoder). Figure 8 The sending device shown can be executed with reference to Figures 1 to 6 The transmitting device according to the embodiment may include a data input unit 8000, a quantization processor 8001, a voxelization processor 8002, an octree occupancy code generator 8003, a surface model processor 8004, an intra-frame / inter-frame coding processor 8005, an arithmetic encoder 8006, a metadata processor 8007, a color transformation processor 8008, an attribute transformation processor 8009, a prediction / lifting / RAHT transformation processor 8010, an arithmetic encoder 8011 and / or a transmission processor 8012.

[0177] The data input unit 8000 according to the embodiment receives or acquires point cloud data. The data input unit 8000 may perform the same operation and / or acquisition method as the point cloud video acquirer 10001 (or refer to Figure 2 The acquisition process 20000) described above has the same or similar operations and / or acquisition methods.

[0178] The data input unit 8000, the quantization processor 8001, the voxelization processor 8002, the octree occupancy code generator 8003, the surface model processor 8004, the intra / inter coding processor 8005 and the arithmetic encoder 8006 perform geometric coding. Figures 1 to 9 The geometrical encodings described are the same or similar, so their detailed description is omitted.

[0179] The quantization processor 8001 according to an embodiment quantizes geometry (e.g., position values ​​of points). The operation and / or quantization of the quantization processor 8001 is similar to the reference Figure 3 The operation and / or quantization of the quantizer 30001 described above are the same or similar. Figures 1 to 9 Same as those described.

[0180] According to an embodiment, the voxelization processor 8002 voxelizes the quantized position value of the point. The voxelization processor 8002 may perform the same as the reference Figure 3 The operation and / or voxelization process of the quantizer 30001 described above is the same or similar to the operation and / or process described above. Figures 1 to 6 Same as those described.

[0181] The octree occupancy code generator 8003 according to the embodiment performs octree encoding on the voxelized position of the point based on the octree structure. The octree occupancy code generator 8003 may generate an occupancy code. The octree occupancy code generator 8003 may perform the same as the reference code. Figure 3 and Figure 4 The operations and / or methods of the point cloud encoder (or octree analyzer 30002) described herein are the same or similar to the operations and / or methods described herein. Figures 1 to 6 Same as those described.

[0182] According to an embodiment, the surface model processor 8004 may perform triplet geometry encoding based on the surface model to reconstruct the point position in a specific area (or node) based on voxels. Figure 3 The operations and / or methods of the point cloud encoder (e.g., surface approximation analyzer 30003) described above are the same or similar to the operations and / or methods. Figures 1 to 6 Same as those described.

[0183] According to an embodiment, the intra-frame / inter-frame encoding processor 8005 may perform intra-frame / inter-frame encoding on the point cloud data. The intra-frame / inter-frame encoding processor 8005 may perform the same as the reference. Figure 7 The intra / inter coding described is the same or similar to the coding. Details and references Figure 7 The intra / inter encoding processor 8005 may be included in the arithmetic encoder 8006 according to an embodiment.

[0184] According to an embodiment, the arithmetic encoder 8006 performs entropy coding on the octree and / or approximate octree of the point cloud data. For example, the coding scheme includes arithmetic coding. The arithmetic encoder 8006 performs the same or similar operations and / or methods as the operations and / or methods of the arithmetic encoder 30004.

[0185] The metadata processor 8007 according to an embodiment processes metadata (e.g., set values) about the point cloud data and provides it to necessary processing processes such as geometry coding and / or attribute coding. In addition, the metadata processor 8007 according to an embodiment may generate and / or process signaling information related to geometry coding and / or attribute coding. The signaling information according to an embodiment may be encoded separately from the geometry coding and / or attribute coding. The signaling information according to an embodiment may be interleaved.

[0186] The color conversion processor 8008, the attribute conversion processor 8009, the prediction / lifting / RAHT conversion processor 8010 and the arithmetic encoder 8011 perform attribute encoding. Figures 1 to 6 The attribute codes described are the same or similar, so their detailed description is omitted.

[0187] The color transform processor 8008 according to an embodiment performs color transform encoding to transform the color value included in the attribute. The color transform processor 8008 may perform color transform encoding based on the reconstructed geometry. The reconstructed geometry is compared with the reference Figures 1 to 9 In addition, its execution is the same as that of reference Figure 3 The operations and / or methods of the color converter 30006 described above are the same as or similar to the operations and / or methods described above, and detailed descriptions thereof are omitted.

[0188] The attribute transformation processor 8009 according to an embodiment performs attribute transformation to transform attributes based on reconstructed geometry and / or locations where geometry encoding is not performed. The attribute transformation processor 8009 performs the attribute transformation with reference to Figure 3 The operations and / or methods of the attribute converter 30007 described in the foregoing description are the same as or similar to those of the attribute converter 30007. Detailed description thereof is omitted. The prediction / lifting / RAHT transform processor 8010 according to the embodiment may encode the transformed attribute by any one or a combination of RAHT coding, prediction transform coding, and lifting transform coding. The prediction / lifting / RAHT transform processor 8010 performs the same as or similar to the reference. Figure 3 The operations of the RAHT transformer 30008, the LOD generator 30009 and the lifting transformer 30010 described above are the same or similar to at least one operation. In addition, the prediction transform coding, the lifting transform coding and the RAHT transform coding are the same as those of the reference Figures 1 to 9 Those described are the same, so their detailed description is omitted.

[0189] The arithmetic encoder 8011 according to the embodiment may encode the encoded attribute based on arithmetic coding. The arithmetic encoder 8011 performs the same or similar operation and / or method as that of the arithmetic encoder 300012.

[0190] The transmission processor 8012 according to the embodiment may send individual bitstreams containing encoded geometry and / or encoded attribute and metadata information, or send a bitstream configured with encoded geometry and / or encoded attribute and metadata information. When the encoded geometry and / or encoded attribute and metadata information according to the embodiment is configured into a bitstream, the bitstream may include one or more sub-bitstreams. The bitstream according to the embodiment may include signaling information and slice data, and the signaling information includes a sequence parameter set (SPS) for sequence level signaling, a geometry parameter set (GPS) for geometry information coding signaling, an attribute parameter set (APS) for attribute information coding signaling, and a patch parameter set (TPS) for patch level signaling. Slice data may include information about one or more slices. A slice according to the embodiment may include a geometry bitstream Geom00 and one or more attribute bitstreams Attr00 and Attr10.

[0191] A slice is a sequence of syntactic elements that represents the entire or a portion of a coded point cloud frame.

[0192] The TPS according to an embodiment may include information about each of the one or more tiles (e.g., coordinate information and height / size information about a bounding box). The geometry bitstream may include a header and a payload. The header of the geometry bitstream according to an embodiment may include a parameter set identifier (geom_parameter_set_id), a tile identifier (geom_tile_id), and a slice identifier (geom_slice_id) included in the GPS, as well as information about the data contained in the payload. As described above, the metadata processor 8007 according to an embodiment may generate and / or process signaling information and send it to the transmission processor 8012. According to an embodiment, an element that performs geometry encoding and an element that performs attribute encoding may share data / information with each other as indicated by the dotted lines. The transmission processor 8012 according to an embodiment may perform operations and / or transmission methods that are the same or similar to those of the transmitter 10003. Details and References Figure 1 and Figure 2 Those described are the same, so their description is omitted.

[0193] Figure 9 A receiving device according to an embodiment is shown.

[0194] Figure 9 The receiving device shown is Figure 1 The receiving device 10004 (or Figure 10 and Figure 11 Example of a point cloud decoder. Figure 9 The receiving device shown can execute and refer to Figures 1 to 11 One or more operations and methods that are the same or similar to those of the described point cloud decoder.

[0195] The receiving device according to the embodiment may include a receiver 9000, a receiving processor 9001, an arithmetic decoder 9002, an octtree reconstruction processor based on an occupancy code 9003, a surface model processor (triangle reconstruction, upsampling, voxelization) 9004, an inverse quantization processor 9005, a metadata parser 9006, an arithmetic decoder 9007, an inverse quantization processor 9008, a prediction / lifting / RAHT inverse transform processor 9009, a color inverse transform processor 9010 and / or a renderer 9011. Each decoding element according to the embodiment may perform an inverse process of the operation of the corresponding encoding element according to the embodiment.

[0196] The receiver 9000 according to the embodiment receives point cloud data. The receiver 9000 may perform the same Figure 1The operation and / or receiving method of the receiver 10005 is the same as or similar to the operation and / or receiving method of the receiver 10005. The detailed description thereof is omitted.

[0197] The reception processor 9001 according to an embodiment may acquire a geometry bitstream and / or an attribute bitstream from the received data. The reception processor 9001 may be included in the receiver 9000 .

[0198] The arithmetic decoder 9002, the octtree reconstruction processor 9003 based on the occupancy code, the surface model processor 9004, and the inverse quantization processor 9005 may perform geometric decoding. Figures 1 to 10 The geometric decoding described is the same or similar, so its detailed description is omitted.

[0199] The arithmetic decoder 9002 according to the embodiment may decode the geometry bitstream based on arithmetic coding. The arithmetic decoder 9002 performs the same or similar operations and / or encoding as those of the arithmetic decoder 7000.

[0200] The octree reconstruction processor 9003 based on the occupancy code according to the embodiment can reconstruct the octree by obtaining the occupancy code from the decoded geometry bitstream (or information about the geometry obtained as a result of decoding). The octree reconstruction processor 9003 based on the occupancy code performs the same or similar operations and / or methods as the operation of the octree synthesizer 7001 and / or the octree generation method. When triplet geometry coding is applied, the surface model processor 9004 according to the embodiment can perform triplet geometry decoding and related geometry reconstruction (e.g., triangle reconstruction, upsampling, voxelization) based on the surface model method. The surface model processor 9004 performs the same or similar operations as the surface approximation synthesizer 7002 and / or the geometry reconstructor 7003.

[0201] The inverse quantization processor 9005 according to an embodiment may inversely quantize the decoded geometry.

[0202] According to an embodiment, the metadata parser 9006 may parse metadata (eg, setting values) contained in the received point cloud data. The metadata parser 9006 may pass the metadata to the geometry decoding and / or attribute decoding. Figure 8 The metadata described is the same, so a detailed description thereof is omitted.

[0203] The arithmetic decoder 9007, the inverse quantization processor 9008, the prediction / lifting / RAHT inverse transform processor 9009 and the color inverse transform processor 9010 perform attribute decoding. Figures 1 to 10 The attribute decoding described is the same or similar, so the detailed description thereof is omitted.

[0204] The arithmetic decoder 9007 according to the embodiment may decode the attribute bitstream by arithmetic coding. The arithmetic decoder 9007 may decode the attribute bitstream based on the reconstructed geometry. The arithmetic decoder 9007 performs the same or similar operation and / or encoding as that of the arithmetic decoder 7005.

[0205] The inverse quantization processor 9008 according to an embodiment may inversely quantize the decoded attribute bitstream. The inverse quantization processor 9008 performs the same or similar operation and / or method as the inverse quantizer 7006 and / or the inverse quantization method.

[0206] The prediction / lifting / RAHT inverse transform processor 9009 according to an embodiment may process the reconstructed geometry and inverse quantized attributes. The prediction / lifting / RAHT inverse transform processor 9009 performs one or more operations and / or decoding that are the same or similar to the operations and / or decoding of the RAHT converter 7007, the LOD generator 7008 and / or the inverse lifter 7009. The color inverse transform processor 9010 according to an embodiment performs inverse transform encoding to inversely transform the color value (or texture) included in the decoded attribute. The color inverse transform processor 9010 performs the same or similar operations and / or inverse transform encoding as the operations and / or inverse transform encoding of the color inverse converter 7010. The renderer 9011 according to an embodiment may render point cloud data.

[0207] Figure 10 An exemplary structure that can be operated in conjunction with a point cloud data transmitting / receiving method / apparatus according to an embodiment is shown.

[0208] Figure 10 The structure of 1060 represents a configuration in which at least one of the server 1060, the robot 1010, the self-driving vehicle 1020, the XR device 1030, the smart phone 1040, the home appliance 1050, and / or the head mounted display (HMD) 1070 is connected to the cloud network 1000. The robot 1010, the self-driving vehicle 1020, the XR device 1030, the smart phone 1040, or the home appliance 1050 is referred to as a device. In addition, the XR device 1030 may correspond to a point cloud data (PCC) device according to an embodiment or may be operably connected to a PCC device.

[0209] The cloud network 1000 may represent a network that constitutes a part of a cloud computing infrastructure or exists in a cloud computing infrastructure. Here, the cloud network 1000 may be configured using a 3G network, a 4G or Long Term Evolution (LTE) network, or a 5G network.

[0210] The server 1060 may be connected to at least one of the robot 1010 , the self-driving vehicle 1020 , the XR device 1030 , the smart phone 1040 , the home appliance 1050 , and / or the HMD 1070 via the cloud network 1000 , and may assist in at least a portion of the processing of the connected devices 1010 to 1070 .

[0211] The HMD 1070 represents one of implementation types of an XR device and / or a PCC device according to an embodiment. The HMD type device according to an embodiment includes a communication unit, a control unit, a memory, an I / O unit, a sensor unit, and a power supply unit.

[0212] Hereinafter, various embodiments of devices 1010 to 1050 to which the above-described technology is applied will be described. Figure 10 The illustrated devices 1010 to 1050 may be operably connected / coupled to the point cloud data sending device and receiving device according to the above-described embodiments.

[0213] <PCC+XR>

[0214] The XR / PCC device 1030 may employ PCC technology and / or XR (AR+VR) technology, and may be implemented as an HMD, a head-up display (HUD) disposed in a vehicle, a television, a mobile phone, a smart phone, a computer, a wearable device, a home appliance, a digital signage, a vehicle, a fixed robot, or a mobile robot.

[0215] The XR / PCC device 1030 may analyze 3D point cloud data or image data acquired through various sensors or from an external device and generate position data and attribute data about 3D points. Thus, the XR / PCC device 1030 may acquire information about the surrounding space or real objects, and render and output XR objects. For example, the XR / PCC device 1030 may match an XR object including auxiliary information about the recognized object with the recognized object and output the matched XR object.

[0216] <PCC+XR+Mobile Phone>

[0217] The XR / PCC device 1030 may be implemented as a mobile phone 1040 by applying the PCC technology.

[0218] The mobile phone 1040 may decode and display point cloud content based on the PCC technology.

[0219] <PCC+Autopilot+XR>

[0220] The self-driving vehicle 1020 may be implemented as a mobile robot, a vehicle, an unmanned aerial vehicle, etc. by applying PCC technology and XR technology.

[0221] The self-driving vehicle 1020 to which the XR / PCC technology is applied may represent a self-driving vehicle provided with a means for providing an XR image, or a self-driving vehicle as a control / interaction target in the XR image. Specifically, as a control / interaction target in the XR image, the self-driving vehicle 1020 may be distinguished from the XR device 1030 and may be operably connected thereto.

[0222] The self-driving vehicle 1020 having means for providing XR / PCC images may acquire sensor information from sensors including cameras, and output generated XR / PCC images based on the acquired sensor information. For example, the self-driving vehicle 1020 may have a HUD and output XR / PCC images thereto, thereby providing passengers with XR / PCC objects corresponding to real objects or objects presented on the screen.

[0223] When the XR / PCC object is output to the HUD, at least a portion of the XR / PCC object may be output to overlap with the real object to which the passenger's eyes are directed. On the other hand, when the XR / PCC object is output on a display provided in the self-driving vehicle, at least a portion of the XR / PCC object may be output to overlap with the object on the screen. For example, the self-driving vehicle 1220 may output XR / PCC objects corresponding to objects such as a road, another vehicle, a traffic light, a traffic sign, a two-wheeled vehicle, a pedestrian, and a building.

[0224] Virtual reality (VR) technology, augmented reality (AR) technology, mixed reality (MR) technology, and / or point cloud compression (PCC) technology according to an embodiment are applicable to various devices.

[0225] In other words, VR technology is a display technology that only provides CG images of real-world objects, backgrounds, etc. On the other hand, AR technology refers to a technology that displays a virtually created CG image on an image of a real object. MR technology is similar to the above-mentioned AR technology in that the virtual objects to be displayed are mixed and combined with the real world. However, MR technology is different from AR technology in that AR technology clearly distinguishes between real objects and virtual objects created as CG images and uses virtual objects as supplementary objects to real objects, while MR technology treats virtual objects as objects with properties equivalent to real objects. More specifically, an example of the application of MR technology is a holographic service.

[0226] Recently, VR, AR, and MR technologies are often referred to as extended reality (XR) technologies, rather than being clearly distinguished from each other. Therefore, embodiments of the present disclosure are applicable to any of VR, AR, MR, and XR technologies. Encoding / decoding based on PCC, V-PCC, and G-PCC technologies are applicable to such technologies.

[0227] The PCC method / apparatus according to the embodiment may be applied to a vehicle that provides a self-driving service.

[0228] Vehicles providing self-driving services are connected to the PCC device for wired / wireless communication.

[0229] When a point cloud data (PCC) transmitting / receiving device according to an embodiment is connected to a vehicle for wired / wireless communication, the device may receive / process content data related to an AR / VR / PCC service (which may be provided together with a self-driving service) and send it to the vehicle. In the case where the PCC transmitting / receiving device is installed on the vehicle, the PCC transmitting / receiving device may receive / process content data related to the AR / VR / PCC service and provide it to the user based on a user input signal input through a user interface device. The vehicle or user interface device according to the embodiment may receive a user input signal. The user input signal according to the embodiment may include a signal indicating a self-driving service.

[0230] The point cloud data transmission method / device according to the embodiment may be interpreted as referring to Figure 1 The sending device 10000, Figure 1 Point cloud video encoder 10002, Figure 1 Transmitter 10003, Figure 2 Get 20000 / encode 20001 / send 20002, Figure 3 The encoder, Figure 8 The sending device Figure 10 Devices Figure 20 The encoder, Figure 28 Terms such as the method of sending.

[0231] The point cloud data receiving method / device according to the embodiment may be interpreted as referring to Figure 1 The receiving device 10004, Figure 1 Receiver 10005, Figure 1 Point cloud video decoder 10006, Figure 2 Send 20002 / decode 20003 / render 20004, Figure 7 The decoder, Figure 9 The receiving device Figure 10 Devices Figure 21 The decoder, Figure 29 Terms such as reception method, etc.

[0232] The method / apparatus for transmitting or receiving point cloud data according to an embodiment may be simply referred to as method / apparatus.

[0233] According to the embodiment, geometric data, geometric information, position information, and geometry constituting point cloud data will be interpreted as having the same meaning. Attribute data, attribute information, etc. constituting point cloud data will be interpreted as having the same meaning.

[0234] The method / apparatus according to an embodiment may include and perform a method of generating a reference road frame from a plurality of point cloud frames captured by LiDAR.

[0235] Embodiments include a method for generating a reference road frame to efficiently support inter-frame prediction of point cloud content captured by a LiDAR device on a moving vehicle for efficient geometry compression in Geometry-based Point Cloud Compression (G-PCC). The reference road frame can be used for both geometry compression and attribute compression. Embodiments may include, for example, a reference road frame generation method and / or a signaling method.

[0236] Embodiments relate to a method for improving compression efficiency of Geometry-Based Point Cloud Compression (G-PCC) for compressing 3D point cloud data. Hereinafter, an encoder or encoding device is referred to as an encoder, and a decoder or decoding device is referred to as a decoder.

[0237] A point cloud consists of a set of points, and each of the points may have geometric information and attribute information. The geometric information is three-dimensional position (XYZ) information, and the attribute information is the value of color (RGB, YUV, etc.) and / or reflectivity.

[0238] In the G-PCC encoding operation, the point cloud may be partitioned into tiles based on regions, and each tile may be partitioned into slices for parallel processing. The operation may include compressing geometry on a per-slice basis, and compressing attribute information based on reconstructed geometry (decoded geometry) configured with position information changed by compression.

[0239] The G-PCC decoding operation may include decoding the geometry based on the received coded slice-level geometry bitstream and attribute bitstream, and reconstructing the attribute information based on the reconstructed geometry obtained by decoding (see Figure 1 etc.) for decoding.

[0240] The geometric information can be compressed using octree-based, prediction tree-based, or triplet-based compression techniques (see Figure 1 wait).

[0241] Embodiments include a method of generating a reference road frame to improve the efficiency of inter-frame prediction-based compression of content captured by a LiDAR device on a moving vehicle.

[0242] Figure 11 An example of a road point cloud captured by a LiDAR device on a moving vehicle according to an embodiment is illustrated.

[0243] Point cloud data sending method / device according to an embodiment ( Figure 1 The sending device 10000, Figure 1 Point cloud video encoder 10002, Figure 1 Transmitter 10003, Figure 2 Get 20000 / encode 20001 / send 20002, Figure 3 The encoder, Figure 8 The sending device Figure 10 Devices Figure 20 The encoder and Figure 28 A method for sending point cloud data according to an embodiment of the present invention and a method / device for receiving point cloud data according to an embodiment of the present invention. Figure 1 The receiving device 10004, Figure 1 Receiver 10005, Figure 1 Point cloud video decoder 10006, Figure 2 Send 20002 / decode 20003 / render 20004, Figure 7 The decoder, Figure 9 The receiving device Figure 10 Devices Figure 21 The decoder and Figure 29 The receiving method can handle Figure 11 Road point cloud. Figure 1 The point cloud acquirer 10001 can acquire points such as Figure 11 Point cloud of point cloud.

[0244] The content captured by the LiDAR device on the mobile vehicle according to the embodiment may include both roads and objects. Point 1100 may correspond to the object, and point 1101 may correspond to the road. The object may be, for example, a building on the road.

[0245] Figure 12 An example of a separated road point cloud according to an embodiment is illustrated.

[0246] Figure 11 The captured road point cloud described in Figure 12 Organized as shown in .

[0247] The captured points in the point cloud of captured roads and objects may have different characteristics between consecutive frames. In addition, inter-frame prediction may be performed by separating only the road from the point cloud including the road and the object. In other words, Figure 12 Shown with Figure 11 The point cloud is separated into only the point cloud with roads.

[0248] Figure 13 An example of road points captured by LiDAR according to an embodiment is illustrated.

[0249] For roads captured by LiDAR devices, such as Figure 11 and Figure 12 In this case, the relative height from the capture device can be constant. Figure 13 As shown, points may be generated in a circle around a center point which is the location of the sensor.

[0250] Figure 14 An example is illustrated in which a shape partially changes when a vehicle equipped with the LiDAR apparatus according to an embodiment does not move but objects around the vehicle move.

[0251] For point clouds, such as Figure 11 , Figure 12 , Figure 13 As shown in , there are buildings, cars, people, and other objects on the road, and even if only the road is separated from the points captured by the LiDAR device, these objects may cause some points to exist and others to not exist. Figure 14 In the circular area 1400, it can be observed that the point 1401 existing in the area disappears, and the point 1402 that does not exist appears. In other words, the circular area 1400 has areas where points exist and areas where points do not exist. The other areas are almost the same. This distribution of the point cloud means the following. The vehicle with the LiDAR device is not moving, and there is a moving object in the area, which causes a part of the road to be hidden. Point 1401 is a point in the previous frame, and point 1402 is a point in the current frame. In other words, the vehicle including the LiDAR device is stationary and generates frames containing points for the road and the object, and the points contained in the currently captured frame and in the frames captured and generated before the current frame can be as shown. Figure 14 As objects around the vehicle move, missing regions 1400 are generated.

[0252] Figure 15 An example of a point cloud acquired when a vehicle provided with the LiDAR apparatus according to the embodiment turns right is illustrated.

[0253] like Figure 11 , Figure 12 , Figure 13 , Figure 14 The point cloud captured by LiDAR may include Figure 15 For example, Figure 14 As shown, the point 1401 of the previous frame and / or the point 1402 of the current frame may be as shown in Figure 15 Organization shown.

[0254] For example, Figure 15As shown, there may be a situation where a road is captured but the road is not completely visible due to buildings. Buildings may exist in a specific part of the intersection, and the captured road points may not exist due to the building. However, when there is movement, the road point cloud may be affected by the building objects and its overall shape may be affected. In this way, the road may be affected by the objects. In other words, the objects may cause missing points of the road. In this situation, when the road points from the previous frame are used as a reference frame, there may not be enough information to allow accurate prediction to be performed on the current frame. Therefore, it is necessary to generate a reference road frame by finding information from the previous frame that can supplement the information about the road.

[0255] In order to perform inter-frame prediction on content captured by a LiDAR device, inter-prediction coding may be applied to refine the predicted points in the reference frame and send the residuals from the predicted points. Therefore, the method / device according to the embodiment can effectively reduce the difference between the reference frame and the current frame. In particular, since the static road area is filled with road information based on the latest information to maintain all optimal information, the residual value can be reduced by finding the best point that matches the point in the current frame. If there is no point in the reference frame that matches the point in the current frame (e.g., with the same laser ID, azimuth, and radius) at the same position, the residual will increase because the neighbor point determined to be the most suitable point (e.g., with the same laser ID and the nearest azimuth) will be set as the predicted point. The increased residual may result in a larger bitstream. Therefore, a method for optimally maintaining information about static roads may be required. In addition, because simple point accumulation requires a large amount of memory, it may be necessary to consider methods for efficient memory use.

[0256] Embodiments include reference road frame generation methods to efficiently support compression through efficient inter-frame prediction of content captured by a LiDAR device on a moving vehicle. Embodiments may be modified and combined. As used herein, terms are to be understood to the extent widely used in the art based on their intended meanings.

[0257] Motion detection of the road may be performed by the geometry encoder of the PCC encoder and applied to the road for reconstruction by the geometry decoding operation of the PCC decoder. A reference road frame may also be used for attribute reconstruction.

[0258] Embodiments may include: 1) a method for processing uncaptured road point cloud data, 2) a method for generating a reference road frame in the absence of global motion, 3) a method for generating a reference road frame in the presence of only translational motion, 4) a method for generating a reference road frame in the presence of both translational motion and rotational motion, 5) a method for generating a reference road frame in the presence of only rotational motion, and 6) a method for initializing a reference road frame. In the following, each operation is described with reference to the corresponding figure.

[0259] Figure 16 An example of a point cloud captured by a LiDAR device is shown in accordance with an embodiment.

[0260] Figure 11 , Figure 12 , Figure 13 , Figure 14 and / or Figure 15 The point cloud can be captured by LiDAR, such as Figure 16 shown. Figure 1 The transmitting device can capture points including roads / objects and generate frames containing these points. The points contained in the frames can be compressed, transmitted, received and reconstructed.

[0261] The LiDAR according to an embodiment may include multiple sensors. Each sensor has a laser ID (laserId). Each sensor has information about the height, angle, and rotation speed relative to the center of the sensor. As the sensor angle increases, the radius of the captured road may increase. When the sensor has a minimum angle, it may be set on top of the LiDAR device and may capture a point of the road with the minimum radius (see Figure 16 ). When the point cloud is separated into roads and objects, the separated point cloud may not have points for any sensor. Figure 16 As shown, for laserIDs equal to 2 and 3, the laser does not reach the ground due to the presence of the object. Instead, it may first reach the object and reflect off the object, and may be separated into an object point cloud. In this case, when the road and the object are separated, there may be no points with laserIDs equal to 2 and 3 in the road point cloud. That is, for the corresponding area in the road frame, there may be a missing area caused by the object.

[0262] Figure 17 An example in which a road point is changing according to a moving object according to an embodiment is illustrated.

[0263] When a vehicle and / or mobile device capturing a point cloud using a LiDAR device is stationary and another vehicle object 1700 passes by, missing road points caused by the passing vehicle may change in the road point cloud.

[0264] For example, when the cluster of circularly distributed points is a vehicle or mobile device performing LiDAR capture and the object 1700 located nearby moves, a change in the distance and / or position between the LiDAR and the object 1700 may result in a missing area in the road frame during LiDAR capture, such as Figure 17 shown.

[0265] Thus, in the absence of global motion, the method for generating a reference road frame changed by a moving object may be performed as follows. The reference road frame generating method according to the embodiment may be performed by the point cloud data transmitting method / device according to the embodiment ( Figure 1 The sending device 10000, Figure 1 Point cloud video encoder 10002, Figure 1 Transmitter 10003, Figure 2 Get 20000 / encode 20001 / send 20002, Figure 3 The encoder, Figure 8 The sending device Figure 10 Devices Figure 20 The encoder and Figure 28 A method for sending point cloud data according to an embodiment of the present invention and a method / device for receiving point cloud data according to an embodiment of the present invention. Figure 1 The receiving device 10004, Figure 1 Receiver 10005, Figure 1 Point cloud video decoder 10006, Figure 2 Send 20002 / decode 20003 / render 20004, Figure 7 The decoder, Figure 9 The receiving device Figure 10 Devices Figure 21 The decoder and Figure 29 The receiving method) is executed by both.

[0266] The reference road frame generation method according to the embodiment may include the following operations: 1) setting the previous road frame as the reference road frame, and / or 2) encoding or decoding the current frame based on the reference frame. In encoding or decoding, 2-1) the status of the point missing area in the current road frame may be retrieved, 2-2) the point may be searched in the reference frame based on the status, and 2-3) when the point exists in the reference frame, the missing area of ​​the current road frame may be updated based on the point. Then, the current (road) frame updated based on the reference (road) frame becomes the reference frame of the next frame, and the next frame that becomes the current frame may be predictively encoded and predictively decoded based on the updated reference frame.

[0267] For example, 1) when there is no reference road frame, the previous road frame may be set as the reference road frame. 2) In the IPPP mode, when the current frame is the Nth frame, the reference road frame may be referenced for encoding / decoding. Generally, the N-1th road may be referenced. Before encoding the N+1th frame, the following operations may be performed. When there is no global motion between the Nth frame and the N-1th frame, that is, when the vehicle equipped with LiDAR is stationary, the laser ID and angle value without points in the Nth road frame may be searched, and the reference road frame may be searched to check whether there are points therein. When there are points, the Nth road frame may be updated with the points, and the updated Nth road frame may be updated as the reference road frame, which may be referenced for encoding / decoding when the N+1th frame is performed. Alternatively, the reference road frame may search for laser IDs and angle values ​​without points, and the Nth road frame may search for the existence of these points therein. When there are points, the reference road frame may be updated with the points, and the reference road frame may be referenced when encoding / decoding the N+1th frame.

[0268] Thus, the influence of objects / motions etc. already present in the reference road frame can be removed, and problems such as memory burden and reduced prediction accuracy that may be caused by the influence can be solved, thereby achieving accurate and fast predictive encoding / decoding of the road frame.

[0269] Figure 18 An example is illustrated in which a road point is being changed when there is only translational motion according to an embodiment.

[0270] like Figure 11 , Figure 12 , Figure 13 , Figure 14 , Figure 15 , Figure 16 , Figure 17 The point cloud shown in FIG. 1 may have Figure 18 Example shown.

[0271] For example, when a vehicle and / or mobile device that captures a point cloud using a LiDAR device has only translational motion, the stationary object 1801 may move in a direction opposite to the traveling direction. For the moving object 1800, the direction opposite to the traveling direction and its traveling direction may coexist.

[0272] When there is only translational motion, the method of generating a reference road frame that changes due to stationary and moving objects may be performed as follows. Similarly, the reference road frame generation method may be performed by both the transmitting device and the receiving device.

[0273] A method for generating a reference road frame according to an embodiment may include: 1) setting a previous road frame as a reference road frame, 2-1) changing the origin in the reference road frame, transforming the coordinate system into a spherical coordinate system, and calculating a capture point per laser ID, 2-2) searching for an area having missing points in the previous road frame based on a specific condition, and updating the previous road frame based on the points in the reference road frame, and 2-3) encoding or decoding a current road frame with reference to the updated previous road frame.

[0274] For example, 1) when there is no reference road frame, the previous road frame may be set as the reference road frame. 2) In IPPP mode, when the current frame is the Nth frame, the reference road frame may be referenced for encoding / decoding. Typically, the N-1th road may be referenced. Before encoding the N+1th frame, the following operations may be performed. When there is only global translation motion between the Nth frame and the N-1th frame, the origin value may be changed in the reference road frame, and the spherical coordinates may be recalculated to calculate the capture point for each laserID. The laserID and angle value for which there is no point in the Nth road frame may be searched, and the reference road frame may be searched to check whether there is a point therein. When there is a point, the Nth road frame may be updated using the point, and the updated Nth road frame may be updated as the reference road frame. The updated Nth road frame may be updated as the reference road frame and referenced when encoding / decoding the N+1th frame.

[0275] Thus, the influence of objects / motions etc. already present in the reference road frame can be removed, and problems such as memory burden and reduced prediction accuracy that may be caused by the influence can be solved, thereby achieving accurate and fast predictive encoding / decoding of the road frame.

[0276] Figure 19 An example is illustrated in which a road point is being changed when there is a translational motion and a rotational motion according to an embodiment.

[0277] like Figure 11 , Figure 12 , Figure 13 , Figure 14 , Figure 15 , Figure 16 , Figure 17 , Figure 18 The point cloud shown in FIG. 1 may have Figure 19 Example shown.

[0278] That is to say, Figure 19 The example illustrates a case where a vehicle and / or mobile device that captures a point cloud using a LiDAR device has translational and rotational motion.

[0279] In the presence of translational motion and rotational motion, a method of generating a reference road frame that changes due to stationary and moving objects may be performed as follows. Similarly, the reference road frame generation method may be performed by both the transmitting device and the receiving device.

[0280] A method for generating a reference road frame according to an embodiment may include: 1) setting a previous road frame as a reference road frame, 2-1) changing the origin in the reference road frame, rotating the point, transforming the coordinate system into a spherical coordinate system, and calculating a capture point per laser ID, 2-2) searching for a laser ID and / or angle in an area where there is no point in the previous road frame, searching for a point in the reference road frame, and updating the reference road frame based on the found point, and 2-3) encoding / decoding a next road frame based on the updated reference road frame.

[0281] For example, 1) when there is no reference road frame, the previous road frame may be set as the reference road frame. 2) In IPPP mode, when the current frame is the Nth frame, the reference road frame may be referenced for encoding / decoding. Typically, the N-1th road may be referenced. Before encoding the N+1th frame, the following operations may be performed. When there is global translation and rotation motion between the Nth frame and the N-1th frame, the origin value in the reference road frame may be changed, the point may be rotated as much as the rotation in the reference road frame, and the spherical coordinates may be recalculated to calculate the capture point per laserID. The laserID and angle value for which there is no point in the Nth road frame may be searched, and the reference road frame may be searched to check whether there is a point therein. When there is a point, the Nth road frame may be updated using the point, and the updated Nth road frame may be updated as the reference road frame. The updated Nth road frame may be updated as the reference road frame and referenced when encoding / decoding the N+1th frame.

[0282] Thus, the influence of objects / motions etc. already present in the reference road frame can be removed, and problems such as memory burden and reduced prediction accuracy that may be caused by the influence can be solved, thereby achieving accurate and fast predictive encoding / decoding of the road frame.

[0283] In the case where only rotational motion exists, a reference road frame may be generated as follows.

[0284] When the vehicle and / or mobile device that captures the point cloud with the LiDAR device has only rotational motion, the method of generating a reference road frame that changes due to stationary and moving objects may be performed as follows. Similarly, the reference road frame generation method may be performed by both the transmitting device and the receiving device.

[0285] A method for generating a reference road frame according to an embodiment includes: 1) setting a previous road frame as a reference road frame, 2-1) rotating points in the reference road frame, transforming a coordinate system into a spherical coordinate system, calculating captured points per laser ID, and searching for laser IDs and / or angles of areas without points, 2-2) updating a next road frame based on the found points in the reference road frame, the updated next road frame becoming the reference road frame, and encoding and decoding the next road frame of the reference road frame based on the updated reference road frame.

[0286] For example, 1) when there is no reference road frame, the previous road frame may be set as the reference road frame. 2) In IPPP mode, when the current frame is the Nth frame, the reference road frame may be referenced for encoding / decoding. Typically, the N-1th road may be referenced. Before encoding the N+1th frame, the following operations may be performed. When there is only global rotational motion between the Nth frame and the N-1th frame, the frame may be rotated as much as the rotation of the points in the reference road frame, and then the spherical coordinates may be recalculated to calculate the captured points per laserID. The laserID and angle values ​​for which there are no points in the Nth road frame may be searched, and the reference road frame may be searched to check whether there are points therein. When there are points, the Nth road frame may be updated with the points, and the updated Nth road frame may be updated as the reference road frame. The updated Nth road frame may be updated as the reference road frame and referenced when encoding / decoding the N+1th frame. Alternatively, the current frame may be rotated in the opposite direction to the rotation, and then the laserID and angle values ​​of the missing points may be filled from the reference frame.

[0287] Thus, the influence of objects / motions etc. already present in the reference road frame can be removed, and problems such as memory burden and reduced prediction accuracy that may be caused by the influence can be solved, thereby achieving accurate and fast predictive encoding / decoding of the road frame.

[0288] Furthermore, the method / apparatus for transmitting / receiving point cloud data according to an embodiment may include a method for generating a reference road frame and / or a method for initializing a reference road frame.

[0289] When the reference road frame and the current frame are significantly different, the encoder can, for example, calculate a similarity score, generate a flag that can indicate a new scene is given without correlation between the two frames, and pass the flag to the decoder as signaling information. In this case, both the encoder and the decoder can skip the reference frame update. In other words, when the scene changes, the reference road frame is not used to predict the current road frame. Therefore, when the scene changes, the problem of memory burden can be solved.

[0290] For example, refer to Figures 17 to 19, terms such as next, current, and previous for multiple frames may be described in various ways depending on the reference. For example, when N is the current frame and N-1 is the previous frame, they may be described as follows. Because the embodiments include generating a reference road frame, they include a method of updating the reference road frame for encoding / decoding the next N+1 from the current frame. This operation allows the current frame (N) to be updated before encoding / decoding the next frame (N+1) in order to reduce memory consumption. That is, the current frame may be encoded / decoded, and its data may be updated for the next frame by the previous frame N-1. Since the current frame N already has a previous frame (=reference frame N-1), frame N-1 may be deleted by updating the road in the current frame with reference to N-1.

[0291] In other words, if the current frame has been encoded, and then the current frame N has missing points for the next frame N+1, the previous reference road frame N-1 may be taken to update N. The updated information may then be used in N+1.

[0292] Figure 20 A point cloud data transmitting device according to an embodiment is illustrated.

[0293] Figure 20 An apparatus for executing a method for transmitting point cloud data according to an embodiment is illustrated ( Figure 1 The sending device 10000, Figure 1 Point cloud video encoder 10002, Figure 1 Transmitter 10003, Figure 2 Get 20000 / encode 20001 / send 20002, Figure 3 The encoder, Figure 8 The sending device Figure 10 Devices Figure 20 The encoder and Figure 28 Each component may correspond to hardware, software, a processor and / or a combination thereof. The description of the operation of each component may conform to the description of the above-mentioned embodiment.

[0294] The PCC data may be input to an encoder (which may be referred to as an encoding device, a transmitting device, etc.) and encoded to output a geometry information bitstream and an attribute information bitstream.

[0295] According to the embodiment, the geometric information encoding component (or geometric information encoder, geometric encoder, etc.) can be as follows Figure 20 Configure as shown.

[0296] According to an embodiment, the geometric information encoder may correspond to Figure 1 A point cloud video encoder, an encoder, a point cloud encoder, or a component for encoding a point cloud video.

[0297] The geometry information encoding component may segment roads and objects through a road / object splitter. When the frame type is a P frame, the frame may be processed by an LPU splitter, and global motion may be predicted by a motion predictor. Then, the area that needs to be segmented may be segmented by a PU splitter, and local motion may be predicted by a motion predictor. This process may continue until segmentation is no longer required or a specified level is reached. Signaling information related to whether to perform LPU / PU segmentation and predicted motion for each LPU / PU may be generated, and the signaling information may be included in the bitstream and signaled to the decoder.

[0298] The motion predictor can signal the PU motion to the decoder. In other words, it can generate PU motion related parameters and send them to the decoder in the bitstream.

[0299] The motion compensation application component may receive input related to whether to generate a reference road frame, and may generate parameters related to whether to generate a reference road frame for a road PU, and include it in the bitstream to be signaled to the decoder. The similarity of consecutive frames may be compared. When the similarity is less than a threshold, the two frames may be defined as different scenes, and a new scene flag (new_scene_flag) may be set to true and sent to the decoder.

[0300] The motion compensation application part may determine whether to apply the predicted motion to the generated LPU / PU through RDO, and may apply the motion to the reference frame according to the determination. Parameters related to whether to apply motion may be generated and included in a bitstream to be signaled to a decoder.

[0301] Using the compensated points, the geometry information can be compressed by the geometry information inter-frame predictor and the related data can be signaled.

[0302] The motion predictor may perform the following operations.

[0303] An input may be received regarding whether to generate a reference road frame for searching for motion of a road PU. When a reference road frame is generated, a reference road frame generation flag may be signaled to the decoder because both the encoder and the decoder should utilize the same reference road frame for updating.

[0304] The motion compensation application component can perform the following operations.

[0305] Depending on the motion of the road PU, whether a reference road frame is used, and whether the scene is new, the motion may be applied to the road PU based on the generated reference frame.

[0306] When there is no global motion in the road motion, the reference road frame can be updated by finding points that are not captured at the laserID, such as Figure 17 When there is only translational motion, the coordinates can be recalculated by changing the origin position of the sensor, and the reference road frame can be updated by finding the points not captured at the laserID, as shown in Figure 18 When there is translation / rotational motion, the coordinates can be recalculated by changing the origin position of the sensor, a rotation can be applied, and then the reference road frame can be updated by finding points that are not captured at the laserID, as shown in Figure 19 When there is only rotational motion, the rotation can be applied and then the reference road frame can be updated by finding the points not captured at the laserID, as shown in Figure 20 This operation can be performed by both the encoder and the decoder, and the prediction can be performed based on the same reference road frame.

[0307] A threshold value forming the basis for determining the similarity between the reference road frame and the current road frame may be input. When the similarity between the two frames as a result of the comparison between the frames is less than the threshold value, it is determined that the scenes are different, and a new scene flag (new_scene_flag) may be set to true and signaled to the decoder. In this case, the reference road frame may be initialized in both the encoder / decoder.

[0308] Reference Figure 16 , Figure 17 , Figure 18 , Figure 19 The reference road frame generation method according to the embodiment described in the embodiment can be performed by Figure 20 It is executed by encoder, geometric encoder, LPU partitioner, motion predictor, PU partitioner, motion compensation application part, geometric information inter-frame predictor, etc.

[0309] The encoder may include a memory storing encoding instructions and a processor connected to the memory, wherein the encoding instructions may be configured to cause the processor to perform encoding operations.

[0310] Although the reference road frame generation and inter-frame prediction based on the reference road frame has been described in geometric terms, this is equally applicable to the attribute coding process.

[0311] The data input can accept geometry, attributes and related parameters. It can accept frames containing points captured by LiDAR, etc.

[0312] The coordinate transformer can transform the coordinate system of the point into a suitable coordinate system for encoding.

[0313] The geometric information transform quantization processor can quantize the points based on the quantization parameters suitable for encoding.

[0314] The space splitter can split the points into spaces suitable for encoding. The split attributes can be passed to the attribute encoder.

[0315] The geometric encoder may include the following elements.

[0316] The voxelizer can voxelize point clouds with geometry.

[0317] Depending on the type of frame, the geometry encoder may perform segmentation and motion prediction / compensation / inter-prediction when the frame is a P-frame, or may perform intra-prediction when the frame is an I-frame.

[0318] For a P frame, the geometry encoder may divide a road and an object into a maximum prediction unit (LPU) through an LPU divider. For a road frame and an object frame, a motion predictor may predict the motion of a road point and the motion of an object point. The motion predictor may receive signaling information indicating whether to generate a reference road frame for searching for the motion of a road PU, and generate a reference road frame according to the information. For the operation of generating a reference road frame, refer to the description given above.

[0319] The PU splitter can split the LPU into PUs. A road PU and an object PU can be generated separately. The LPU and the PU can be configured in various ways according to the motion prediction / compensation settings.

[0320] The motion compensation application section may apply motion to each of the corresponding road PU and object PU based on the reference frame. As described above, the reference frame may be updated to resolve the missing portion, and the motion of the frame may be compensated based on the updated reference frame to perform inter-frame prediction. Since the problem caused by the missing portion is solved, accurate and fast inter-frame prediction may be performed.

[0321] The inter-frame predictor may predict a current frame based on a motion-compensated reference frame, generate a residual between the geometry of a predicted point and the geometry of a current point, and transmit a bitstream of a size reduced by compressing and entropy encoding only the residual.

[0322] Since the geometry of the points is necessary for attribute encoding, the encoded geometry can be reconstructed and passed to the attribute encoder.

[0323] The geometric encoder performs intra-frame prediction. In the absence of a reference frame, it finds similar points in the current frame from neighbor candidates, generates a residual, and entropy encodes the residual.

[0324] The attribute encoder can transform the attributes into an appropriate color system through a color transform processor.

[0325] When the geometry encoding scheme is based on lossy encoding, the attribute encoder can adjust the color through the recolorizer to reconstruct the attributes (color) for the missing geometry. When the geometry encoding method is lossless, the attribute encoder can skip the recoloring operation.

[0326] In the attribute intra prediction mode, the attribute encoder may perform intra prediction encoding on the attribute information. When the mode is not the intra prediction mode, inter prediction encoding may be performed on the attribute information. As in the case of geometry, in the attribute inter mode, the attributes in the current frame may be predicted by referring to a reference frame, and the attribute inter method may be used for encoding. By predicting the attributes in an inter prediction and / or intra prediction manner, a residual may be generated and the residual may be entropy encoded to generate an attribute information bitstream containing the attribute information. In addition, the encoded attributes may be reconstructed and passed to a reference frame buffer. The reference frame buffer may store the reconstructed geometry information and / or the reconstructed attribute information, and may provide point (geometry / attribute) information about the reference frame when encoding the next frame in an inter prediction manner.

[0327] The transmitting device may encode the point cloud, generate parameters based on the operation, and generate and transmit a bit stream, such as Figures 22 to 27 shown.

[0328] Figure 21 A point cloud data receiving device according to an embodiment is illustrated.

[0329] Figure 21 An apparatus for executing a point cloud data receiving method according to an embodiment is illustrated ( Figure 1 The receiving device 10004, Figure 1 Receiver 10005, Figure 1 Point cloud video decoder 10006, Figure 2 Send 20002 / decode 20003 / render 20004, Figure 7 The decoder, Figure 9 The receiving device Figure 10 Devices Figure 21 The decoder and Figure 29 configuration of the receiving method).

[0330] Each component may correspond to hardware, software, a processor and / or a combination thereof. The description of the operation of each component may conform to the description of the above embodiments.

[0331] Figure 21 Each operation of the receiving device corresponds to Figure 20 Each operation of the sending device, and the receiving operation can follow the reverse process of the sending operation.

[0332] The encoded geometry information bitstream and attribute information bitstream may be input to the decoder, and the decoded and reconstructed PCC data may be output from the decoder.

[0333] According to an embodiment, the geometric information decoder may correspond to Figure 1 Point cloud video decoder, decoding, point cloud decoder and components for decoding point cloud video.

[0334] The decoder may include a memory storing decoding instructions and a processor connected to the memory, wherein the decoding instructions may be configured to cause the processor to perform a decoding operation.

[0335] The motion compensation application component may apply motion to the road PU based on a reference frame generated based on the motion of the transmitted road PU, whether a reference road frame is used, and whether the scene is new.

[0336] The geometry information decoder can receive a geometry information bitstream ( Figures 20 to 27 ).

[0337] The geometry information entropy decoder can decode the geometry information based on the entropy method.

[0338] The geometry information decoder may perform a decoding operation based on whether the geometry coding method is intra coding or inter coding. When the geometry coding method is not geometry intra coding, the LPU / PU segmenter may segment a sequence, a frame, etc. of a bitstream containing a point cloud into roads and / or objects.

[0339] The motion compensation application component can parse the new scene flag, reference road frame generation flag, PU motion, etc. from the bitstream and apply motion compensation to the units into which the road / object is segmented.

[0340] The geometry information inter-frame prediction reconstructor may predict the road / object in the current frame based on the reference frame and reconstruct the geometry by adding the received residual to the prediction value.

[0341] When the geometry coding method is geometry intra coding, the prediction value can be reconstructed from the neighbor candidate points that are most similar to the current geometry in the current frame without a reference frame.

[0342] The geometry information transform inverse quantization processor may inverse quantize the geometry reconstructed by the intra-frame and / or inter-frame method based on the quantization parameter. The reconstructed geometry may then be provided for attribute decoding.

[0343] The geometry information decoder can decode the geometry information.

[0344] The attribute information decoder may decode the received residual through the attribute residual information entropy decoder based on an entropy method.

[0345] Depending on whether the attribute encoding method is intra-frame encoding or inter-frame encoding, the attribute information decoder can perform corresponding operations.

[0346] When the attribute encoding method is not intra-coding, the attribute information inter-prediction reconstructor may generate a prediction for the current frame by referring to a reference frame using an inter-prediction method. As in the case of geometry, a reference frame may be generated.

[0347] When the attribute encoding method is intra-frame encoding, the attribute information intra-frame prediction reconstructor can generate predictions from neighbor candidates with attributes most similar to the current point in the current frame without a reference frame.

[0348] The attribute information decoder may generate the reconstructed attribute information. The reference frame buffer may store the reconstructed geometry and the reconstructed attribute as the geometry and the attribute for the reference frame, and provide the reference information when predicting a next frame after the reference frame.

[0349] The color inverse conversion processor can perform inverse conversion on the color system.

[0350] The attribute information decoder can generate attribute information.

[0351] Figure 22 A bitstream containing point cloud data and parameters according to an embodiment is illustrated.

[0352] A device for executing a point cloud data transmission method according to an embodiment ( Figure 1 The sending device 10000, Figure 1 Point cloud video encoder 10002, Figure 1 Transmitter 10003, Figure 2 Get 20000 / encode 20001 / send 20002, Figure 3 The encoder, Figure 8 The sending device Figure 10[[END Devices ​ The encoder and ​ Sending method) can generate ​ A bit stream, and a device for executing a point cloud data receiving method according to an embodiment ( ​ The receiving device 10004, ​ Receiver 10005, ​ Point cloud video decoder 10006, ​ Send 20002 / decode 20003 / render 20004, ​ The decoder, ​ The receiving device ​ Devices ​ The decoder and ​ The receiving method) can be based on ​ The parameters in the bitstream are used to​ Decodes the geometry and / or attributes in the bitstream.

[0353] Likewise, the method / apparatus according to the embodiment may signal relevant information to add / execute the embodiment. The signaling information according to the embodiment may be used at the transmitting side or the receiving side.

[0354] The encoded point cloud may be configured as follows. A point cloud data encoder configured to perform geometry encoding and / or attribute encoding may generate an encoded point cloud (or a bitstream containing a point cloud) as follows. In addition, signaling information associated with the point cloud data may be generated and processed by a metadata processor of a point cloud data transmitting device and included in the point cloud as follows.

[0355] Each abbreviation has the following meaning. Each abbreviation may be referred to by another term within the scope of equivalent meaning: SPS: sequence parameter set; GPS: geometry parameter set; APS: attribute parameter set; TPS: patch parameter set; Geom: geometry bitstream = geometry slice header + [geometry PU header + geometry PU data] | geometry slice data; Attr: attribute bitstream = attribute data unit header + [attribute PU header + attribute PU data] | attribute data unit data. A slice may be referred to as a data unit, a slice header may be referred to as a data unit header; and slice data may be referred to as unit data.

[0356] The method / apparatus according to the embodiment may generate reference road frame generation information according to the embodiment and add it to the SPS, GPS or TPS to signal it. The reference road frame generation information may be added to the TPS or geometry data header for each slice and signaled therein. Tiles or slices are provided to allow the point cloud to be divided into regions for processing. When the point cloud data is divided into regions, the options for generating different sets of neighbor points for the corresponding regions may be configured to provide a selection method that exhibits low complexity and low result reliability or a selection method that exhibits high complexity and high reliability. This option may be configured differently depending on the capacity of the receiver. Therefore, when the point cloud is divided into tiles, different options may be applied to each tile. When the point cloud is divided into slices, different options may be applied to each slice. A geometry PU may be added, and a geometry PU header and geometry PU data may be added.

[0357] In the following, the description of the method included in the ​ The syntax of parameter sets in the bitstream.

[0358] ​ A sequence parameter set (SPS) according to an embodiment is illustrated.

[0359] ​ The SPS in the ​The SPS in the bitstream of the reference road frame. Option information related to the generation of the reference road frame can be added to the SPS and signaled in the SPS. Therefore, efficient signaling can be performed to support inter-frame prediction of geometry / attributes. As used herein, the name of the signaling information can be understood within the scope of the meaning and function of the signaling information.

[0360] profile_idc: Indicates the profile to which the bitstream conforms. The bitstream shall not contain profile_idc values ​​other than the specified value. Other values ​​of profile_idc are reserved for future use by ISO / IEC.

[0361] profile_compatibility_flags: profile_compatibility_flags equal to 1 indicates that the bitstream conforms to the profile indicated by profile_idc equal to j.

[0362] sps_num_attribute_sets: sps_num_attribute_sets indicates the number of coded attributes in the bitstream. The value of sps_num_attribute_sets can be in the range of 0 to 63.

[0363] attribute_dimension[i]: indicates the number of components of the i-th attribute.

[0364] attribute_instance_id[i]: indicates the instance ID of the i-th attribute.

[0365] reference_road_generation_flag: Indicates whether to generate reference roads in the sequence.

[0366] ​ A geometry parameter set (GPS) according to an embodiment is illustrated.

[0367] ​ The GPS in the ​ The GPS in the bitstream of the reference road frame. Option information related to the generation of the reference road frame can be added to the GPS and signaled in the GPS. Therefore, efficient signaling can be performed to support inter-frame prediction of geometry / attributes. As used herein, the name of the signaling information can be understood within the scope of the meaning and function of the signaling information.

[0368] gps_geom_parameter_set_id: indicates an identifier for GPS to be referenced by other syntax elements. The value of gps_seq_parameter_set_id can be in the range of 0 to 15.

[0369] gps_seq_parameter_set_id: indicates the value of sps_seq_parameter_set_id for the active SPS. The value of gps_seq_parameter_set_id can be in the range of 0 to 15.

[0370] geom_tree_type: geom_tree_type equal to 0 indicates that an octree is used to encode the position information. geom_tree_type equal to 1 indicates that a prediction tree is used to encode the position information.

[0371] new_scene_flag: Indicates whether this frame is a new scene compared to the previous frame.

[0372] reference_road_generation_flag: Indicates whether to generate reference roads in the frame.

[0373] ​ The syntax of a tile parameter set (TPS) according to an embodiment is illustrated.

[0374] ​ The TPS in the representation is contained in ​ The TPS in the bitstream of the reference road frame may be added to the TPS and signaled in the TPS. Therefore, efficient signaling may be performed to support inter-frame prediction of geometry / attributes. As used herein, the name of the signaling information may be understood within the scope of the meaning and function of the signaling information.

[0375] num_tiles: Indicates the number of tiles signaled for the bitstream. When not present, num_tiles is inferred to be 0.

[0376] tile_bounding_box_offset_x[i] indicates the x offset of the i-th tile in Cartesian coordinates. When not present, the value of tile_bounding_box_offset_x[0] is inferred to be sps_bounding_box_offset_x.

[0377] tile_bounding_box_offset_y[i] indicates the y offset of the i-th tile in Cartesian coordinates. When not present, the value of tile_bounding_box_offset_y[0] is inferred to be sps_bounding_box_offset_y.

[0378] tile_bounding_box_offset_z[i] indicates the z offset of the i-th tile in Cartesian coordinates. When not present, the value of tile_bounding_box_offset_z[0] is inferred to be sps_bounding_box_offset_z.

[0379] new_scene_flag: Indicates whether the frame in the tile is a new scene compared to the previous frame.

[0380] reference_road_generation_flag: Indicates whether to generate reference roads in the tile.

[0381] ​ The syntax of the geometry tile header (geometry data header) according to an embodiment is illustrated.

[0382] ​ The GSH in (which may be called geometry slice header, geometry data header, etc.) indicates the information contained in ​ The geometry slice header in the bitstream of the bitstream. Option information related to the generation of the reference road frame can be added to the GSH and signaled in the GSH. Therefore, efficient signaling can be performed to support inter-frame prediction of geometry / attributes. As used in this document, the name of the signaling information can be understood within the scope of the meaning and function of the signaling information. During the geometry information encoding / decoding process, the point reference road frame generation information can be added to the geometry data header and signaled in the geometry data header. The reference road frame generated from this information can also be used for attributes.

[0383] gsh_geometry_parameter_set_id: Indicates the value of gps_geom_parameter_set_id of the active GPS.

[0384] gsh_tile_id: A value indicating the tile ID referenced by the GSH. The value of gsh_tile_id can be in the range of 0 to XX.

[0385] gsh_slice_id: Indicates a slice header used for reference by other syntax elements. The value of gsh_slice_id can be in the range of 0 to XX.

[0386] new_scene_flag: Indicates whether the frame in the slice is a new scene compared to the previous frame.

[0387] reference_road_generation_flag: Indicates whether to generate reference roads in the tile.

[0388] ​The syntax of the Geometry PU header according to an embodiment is shown.

[0389] ​ The geometry PU header is contained in ​ The optional information related to the generation of the reference road frame can be added to the geometry PU header and signaled in the geometry PU header. Therefore, efficient signaling can be performed to support inter-frame prediction of geometry / attributes. The inter-frame prediction related reference road frame generation information can be signaled by creating a geometry PU header.

[0390] pu_tile_id: indicates the tile ID to which the PU belongs.

[0391] pu_slice_id: indicates the slice ID to which the PU belongs.

[0392] pu_cnt: indicates the number of pu contained in the slice.

[0393] pu_id: indicates PU ID.

[0394] pu_split_flag: Indicates whether the PU block is further split later.

[0395] pu_has_motion_vector_flag: indicates whether the PU block has a motion vector.

[0396] pu_motion_mat[][][][]: indicates the motion matrix applied to the PU block.

[0397] pu_motion_rot_vector[][]: indicates the motion rotation vector applied to the PU block.

[0398] pu_motion_trans[][]: indicates the motion translation vector applied to the PU block.

[0399] pu_motion_rot_type[]: indicates the type of motion rotation value applied to the PU block. For example, 0 = radians, 1 = degrees.

[0400] pu_motion_rot[]: indicates the motion rotation applied to the PU block. The rotation axis can be rotated around the road normal vector.

[0401] is_road_flag: Indicates whether the point contained in the LPU block is a road / object.

[0402] object_id: Indicates the object id of a point included in the PU block.

[0403] new_scene_flag[]: Indicates whether the frame in the PU block is a new scene compared to the previous frame.

[0404] reference_road_generation_flag[]: indicates whether to generate a reference road in the PU block.

[0405] ​ A point cloud data transmitting method according to an embodiment is illustrated.

[0406] according to ​ The point cloud data sending method of the implementation method in the embodiment can be ​ The transmitting device 10000, the point cloud video encoder 10002, the transmitter 10003, ​ The encoder, ​ The sending device ​ Devices ​ encoder, etc. to execute.

[0407] S2800: The point cloud data transmission method according to the embodiment may include encoding the point cloud data. In the encoding operation according to the embodiment, ​ The point cloud data is encoded as shown. The encoding operation may include geometry encoding and / or attribute encoding. The encoded point cloud may be included in a bitstream, and parameters related to the encoding may be generated and included in the bitstream ( ​ ). A bitstream including a geometry bitstream and / or an attribute bitstream may be generated. In addition, when encoding a frame containing a point cloud using an inter-frame method, the encoding may include segmenting objects and / or roads and performing predictive encoding. A reference frame for inter-frame prediction may be generated and updated, such as ​ shown.

[0408] S2810: The point cloud data transmission method according to the embodiment may further include transmitting a bit stream containing the point cloud data. In the transmission operation according to the embodiment, the bit stream may be transmitted as follows: ​ The bit stream is shown.

[0409] ​ A point cloud data receiving method according to an embodiment is illustrated.

[0410] ​ The point cloud data receiving method shown can be ​ The receiving device 10004, the receiver 10005, the point cloud video decoder 10006, ​ The decoder, ​ The receiving device ​ Devices ​ decoder, etc. to execute.

[0411] S2900: The point cloud data receiving method according to the embodiment may include receiving a bit stream containing point cloud data. In the receiving operation according to the embodiment, the receiving ​ The bit stream is shown.

[0412] S2910: The point cloud data receiving method according to the embodiment may further include decoding the point cloud data. In the decoding operation according to the embodiment, ​ The point cloud data is decoded as shown in . The decoding may include geometric decoding and / or attribute decoding. Based on parameters related to encoding and / or decoding, the point cloud contained in the bitstream can be decoded. In addition, as shown in ​ As shown, when decoding a frame containing a point cloud using an inter-frame method, decoding may include segmenting objects and / or roads and performing predictive decoding. A reference frame for inter-frame prediction may be generated and updated, such as ​ shown.

[0413] The PCC encoding method, PCC decoding method, and signaling method of the above-mentioned embodiments can provide the following effects.

[0414] Reference ​ According to an embodiment, a sending method may include: encoding point cloud data; and sending a bit stream containing the point cloud data.

[0415] Reference ​ Regarding the configuration of road / object points, the point cloud data according to the embodiment may include road-related points and object-related points. In a frame containing road-related points, the road-related points may be changed by the road-related object or may be missing.

[0416] Reference ​ , regarding the capture points for each LiDAR ID, point cloud data may be acquired by the LiDAR, and the point cloud data may include points based on the laser ID of the LiDAR.

[0417] Reference ​ Regarding the method of generating a reference road frame (e.g., in the absence of global motion), encoding the point cloud data may include encoding a road frame of a current frame including the point cloud data. Encoding the road frame may include: predicting a road frame of the current frame based on a reference road frame for the road frame of the current frame; searching for missing points in the reference road frame in a previous frame of the reference road frame based on at least one of a laser ID or an angle; and updating the missing points in the reference road frame.

[0418] Similarly, encoding a road frame may include: predicting a road frame of a current frame based on a reference road frame of a road frame of the current frame; searching for missing points in the road frame in the reference road frame based on at least one of a laser ID or an angle; and updating the missing points in the road frame. In other words, when there are multiple road frames and there are missing points, the missing points may be updated using a previous road frame for the next encoding and / or decoding to generate an enhanced reference road frame for encoding / decoding of the next road frame. By updating the current frame before encoding / decoding the next frame, memory usage may be effectively reduced. Since the current frame already has a previous frame (i.e., a reference frame), updating the road in the current frame with reference to the previous road information may delete the previous frame.

[0419] Reference ​ , with respect to a method of generating a reference road frame (for example, when there is only translation), encoding the point cloud data may include encoding a road frame of a current frame containing the point cloud data. Encoding the road frame may include: predicting a road frame of a current frame based on a reference road frame for the road frame of the current frame; calculating a point for each laser ID based on a spherical coordinate system when origin coordinate information about the reference road frame is changing; searching for missing points in a previous frame of the reference road frame based on at least one of a laser ID or an angle; and updating the missing points in the reference road frame.

[0420] Similarly, encoding a road frame may include: predicting a road frame of a current frame based on a reference road frame for the road frame of the current frame; calculating a point for each laser ID based on a spherical coordinate system when origin coordinate information about the reference road frame is changing; searching for missing points in the road frame in the reference road frame based on at least one of the laser ID or the angle; and updating the missing points in the road frame. Thus, the method may provide the above-mentioned effects.

[0421] Reference ​ , with respect to generating a reference road frame (e.g., in the presence of translation / rotation), encoding the point cloud data may include encoding a road frame of a current frame containing the point cloud data. Encoding the road frame may include: predicting a road frame of a current frame based on a reference road frame for the road frame of the current frame; rotating the reference road frame in the case where origin coordinate information about the reference road frame is changing, and calculating points for each laser ID based on a spherical coordinate system; searching for missing points in a previous frame of the reference road frame based on at least one of a laser ID or an angle; and updating the missing points in the reference road frame.

[0422] Similarly, encoding a road frame may include: predicting a road frame of a current frame based on a reference road frame for a road frame of the current frame; rotating the reference road frame when origin coordinate information about the reference road frame is changing, and calculating a point for each laser ID based on a spherical coordinate system; searching the reference road frame for missing points in the road frame based on at least one of the laser ID or the angle; and updating the missing points in the road frame. Thus, the method may provide the above-mentioned effects.

[0423] Reference ​ Regarding generating a reference road frame (for example, when there is only rotation), encoding the point cloud data may include encoding a road frame of a current frame containing the point cloud data. Encoding the road frame may include: predicting a road frame of the current frame based on a reference road frame for the road frame of the current frame; rotating the reference road frame and calculating a point for each laser ID based on a spherical coordinate system; searching for missing points in the reference road frame in a previous frame of the reference road frame based on at least one of a laser ID or an angle; and updating the missing points in the reference road frame.

[0424] Similarly, encoding the road frame may include: predicting the road frame of the current frame based on a reference road frame of the road frame of the current frame; rotating the reference road frame and calculating points of each laser ID based on a spherical coordinate system; searching for missing points in the road frame from the reference road frame based on at least one of the laser ID or the angle; and updating the missing points in the road frame.

[0425] Reference ​ Regarding initializing the reference road frame, encoding the point cloud data may further include initializing the reference road frame.

[0426] Reference Fig. 22 Regarding the signaling, the bitstream may include at least one of information indicating whether to generate a reference road frame or information indicating whether the current frame is a new scene. For example, the bitstream may include information indicating whether to generate a reference road frame or information indicating whether the current frame is a new scene, or include information indicating whether to generate a reference road frame and information indicating whether the current frame is a new scene.

[0427] The point cloud data transmission method may be executed by a transmission device. The transmission device may include: an encoder configured to encode the point cloud data; and a transmitter configured to transmit a bit stream containing the point cloud data.

[0428] The receiving method corresponding to the point cloud data transmitting method may perform the reverse process of the transmitting, and may include, for example: receiving a bit stream including the point cloud data, and decoding the point cloud data.

[0429] Similarly, the point cloud data receiving method may include decoding the point cloud data, the decoding including decoding a road frame of a current frame containing the point cloud data. The decoding of the road frame may include: predicting a road frame of the current frame based on a reference road frame for the road frame of the current frame; searching for missing points in a reference road frame in a previous frame of the reference road frame based on at least one of a laser ID or an angle; and updating the missing points in the reference road frame.

[0430] Decoding the road frame may include predicting a road frame of the current frame based on a reference road frame for the road frame of the current frame; searching the reference road frame for missing points in the road frame based on at least one of a laser ID or an angle; and updating the missing points in the road frame.

[0431] The point cloud data receiving method may be performed by a receiving device. The receiving device may include: a receiver configured to receive a bit stream containing point cloud data; and a decoder configured to decode the point cloud data.

[0432] Thus, road data containing insufficient information can be supplemented to achieve accurate and fast compression / reconstruction. In other words, missing road points can be added and updated by coordinate origin matching and rotation from the previous frame. According to the movement of the LiDAR device, the object, etc., the reference road frame can be updated differently. In other words, both the LiDAR movement and the object movement can be compensated. The road frame can be compressed to a smaller bitstream size and reconstructed. This can solve the problem of reduced compression performance caused by the influence of the object.

[0433] In a scenario where frames are captured and stored one by one using a LiDAR device on a moving vehicle, there may be continuity between frames, and therefore, compression may be performed efficiently using inter-frame prediction techniques. Since the motion of the captured road and object point clouds are different, segmenting the road and objects in order to efficiently apply the inter-frame prediction technique and performing fast and accurate motion prediction for each of the road and the object may affect the compression time, and improving the efficiency of the inter-frame prediction technique through accurate prediction may reduce the bitstream size. Inaccurate motion prediction may significantly increase the bitstream size, which may reduce compression efficiency. Therefore, an embodiment may utilize the characteristics captured by the LiDAR device on a moving vehicle to quickly and accurately predict motion.

[0434] Embodiments provide methods for road / object segmentation to support efficient geometric compression of content captured by a LiDAR device on a moving vehicle, and methods for generating reference road frames used in inter-frame prediction to improve the efficiency of inter-frame prediction of roads. The reference road frames may be generated to cumulatively update portions of a road that are affected by changes due to an object moving on the same road, to compensate for road points missing due to an object or a capture environment, and to generate reference road frames only for portions where reference road frames are needed, taking into account an increase in memory usage.

[0435] Thus, embodiments can increase the inter-geometry compression efficiency of an encoder / decoder for geometry-based point cloud compression (G-PCC) for compressing 3D point cloud data to provide a point cloud content stream. The reference road frame constructed by this method can also be used as a reference road frame in attribute compression, thereby improving compression efficiency.

[0436] Therefore, the sending method / apparatus according to the implementation manner can efficiently compress point cloud data to send the data and transmit signaling information for the data. Therefore, the receiving method / apparatus according to the implementation manner can efficiently decode / reconstruct the point cloud data.

[0437] The operations of the transmitting and receiving devices according to the above-described embodiments may be described in conjunction with the point cloud compression process.

[0438] The embodiments are described in terms of methods and / or devices, and the description of the methods and the description of the devices can be applied complementary to each other.

[0439] Although the drawings are described separately for simplicity, new embodiments can be designed by combining the embodiments shown in the corresponding figures. Designing a recording medium that can be read by a computer and records the program for executing the above-mentioned embodiments according to the needs of those skilled in the art also falls within the scope of the attached claims and their equivalents. The apparatus and method according to the embodiment may not be limited to the configuration and method of the above-mentioned embodiment. Various modifications can be made to the embodiment by selectively combining all or some of the embodiments. Although preferred embodiments are described with reference to the drawings, it will be understood by those skilled in the art that various modifications and changes can be made to the embodiment without departing from the spirit or scope of the present disclosure described in the attached claims. These modifications should not be understood separately from the technical ideas or viewpoints of the embodiments.

[0440] Various elements of the device of the embodiment can be implemented by hardware, software, firmware or a combination thereof. Various elements in the embodiment can be implemented by a single chip (e.g., a single hardware circuit). According to the embodiment, the components according to the embodiment can be implemented as separate chips respectively. According to the embodiment, at least one or more components of the device according to the embodiment may include one or more processors capable of executing one or more programs. One or more programs may execute any one or more operations / methods according to the embodiment or include instructions for executing them. The executable instructions for executing the method / operation of the device according to the embodiment may be stored in a non-transitory CRM or other computer program product configured to be executed by one or more processors, or may be stored in a temporary CRM or other computer program product configured to be executed by one or more processors. In addition, the memory according to the embodiment can be used as not only covering volatile memory (e.g., RAM), but also covering the concepts of non-volatile memory, flash memory and PROM. In addition, it can also be implemented in the form of a carrier (e.g., transmission via the Internet). In addition, the processor-readable recording medium can be distributed to a computer system connected via a network so that the processor-readable code can be stored and executed in a distributed manner.

[0441] In the present disclosure, " / " and "," should be interpreted as indicating "and / or". For example, the expression "A / B" may mean "A and / or B". Furthermore, "A, B" may mean "A and / or B". Furthermore, "A / B / C" may mean "at least one of A, B, and / or C". In addition, "A / B / C" may mean "at least one of A, B, and / or C". Furthermore, in the present specification, the term "or" should be interpreted as indicating "and / or". For example, the expression "A or B" may mean 1) only A, 2) only B, or 3) both A and B. In other words, the term "or" used in this document should be interpreted as indicating "in addition or alternatively".

[0442] Terms such as first and second can be used to describe various elements of the embodiments. However, various components according to the embodiments should not be limited by the above terms. These terms are only used to distinguish one element from another element. For example, a first user input signal can be referred to as a second user input signal. Similarly, a second user input signal can be referred to as a first user input signal. The use of these terms should not be interpreted without departing from the scope of various embodiments. Both the first user input signal and the second user input signal are user input signals, but unless the context clearly stipulates otherwise, they do not mean the same user input signal.

[0443] The terms used to describe the embodiments are used to describe specific embodiments and are not intended to limit the embodiments. As used in the description of the embodiments and the claims, unless the context clearly stipulates otherwise, the singular form includes plural referents. The expression "and / or" is used to include all possible combinations of terms. Terms such as "including" or "having" are intended to indicate the presence of graphics, quantities, steps, elements and / or components, and should be understood as not excluding the possibility of additional presence of graphics, quantities, steps, elements and / or components. As used herein, conditional expressions such as "if" and "when" are not limited to optional situations, and are intended to perform related operations or interpret related definitions according to specific conditions when specific conditions are met.

[0444] The operations according to the embodiments described in this specification may be performed by a transmitting / receiving device including a memory and / or a processor according to the embodiments. The memory may store a program for processing / controlling operations according to the embodiments, and the processor may control the various operations described in this specification. The processor may be referred to as a controller, etc. In an embodiment, the operations may be performed by firmware, software, and / or a combination thereof. Firmware, software, and / or a combination thereof may be stored in a processor or a memory.

[0445] The operations according to the above-described embodiments may be performed by a sending device and / or a receiving device according to the embodiments. The sending / receiving device may include a sender / receiver configured to send and receive media data, a memory configured to store instructions (program code, algorithm, flow chart and / or data) for the process according to the embodiments, and a processor configured to control the operation of the sending / receiving device.

[0446] The processor may be referred to as a controller, etc., and may correspond to, for example, hardware, software, and / or a combination thereof. The operations according to the above-described embodiments may be performed by the processor. In addition, the processor may be implemented as an encoder / decoder for the operations of the above-described embodiments.

[0447] Invention Mode

[0448] As described above, relevant details are described in the best mode for carrying out the embodiment.

[0449] Industrial Applicability

[0450] As described above, the embodiments may be applied in whole or in part to a point cloud data transmitting / receiving device and system.

[0451] A skilled person can change or modify the embodiments in various ways within the scope of the embodiments.

[0452] The embodiments may include changes / modifications within the scope of the claims and their equivalents.

Claims

1. A method for sending point cloud data, the method comprising the following steps: Encoding point cloud data; and A bitstream containing the point cloud data is sent.

2. The method according to claim 1, wherein: The point cloud data includes points related to the road and points related to the object, In which, in the frame containing the point related to the road, the point related to the road is changed according to the object related to the road, or is missing.

3. The method according to claim 1, wherein: The point cloud data is obtained by LiDAR. Wherein, the point cloud data includes points based on the laser ID of the LiDAR.

4. The method according to claim 1, wherein: The step of encoding the point cloud data comprises: encoding a road frame of a current frame containing the point cloud data, The step of encoding the road frame comprises: predicting the road frame of the current frame based on a reference road frame of the road frame of the current frame; searching the reference road frame for missing points in the road frame based on at least one of a laser ID or an angle; and The missing points in the road frame are updated.

5. The method according to claim 1, wherein: The step of encoding the point cloud data comprises: encoding a road frame of a current frame containing the point cloud data, The step of encoding the road frame comprises: predicting the road frame of the current frame based on a reference road frame for the road frame of the current frame; calculating a point for each laser ID based on a spherical coordinate system while the origin coordinate information about the reference road frame is changing; searching the reference road frame for missing points in the road frame based on at least one of the laser ID or angle; and The missing points in the road frame are updated.

6. The method according to claim 1, wherein: The step of encoding the point cloud data comprises: encoding a road frame of a current frame containing the point cloud data, The step of encoding the road frame comprises: predicting the road frame of the current frame based on a reference road frame for the road frame of the current frame; rotating the reference road frame while origin coordinate information about the reference road frame is changing, and calculating a point for each laser ID based on a spherical coordinate system; searching the reference road frame for missing points in the road frame based on at least one of the laser ID or angle; and The missing points in the road frame are updated.

7. The method according to claim 1, wherein: The step of encoding the point cloud data comprises: encoding a road frame of a current frame containing the point cloud data, The step of encoding the road frame comprises: predicting the road frame of the current frame based on a reference road frame for the road frame of the current frame; Rotating the reference road frame and calculating a point for each laser ID based on a spherical coordinate system; searching the reference road frame for missing points in the road frame based on at least one of the laser ID or angle; and The missing points in the road frame are updated.

8. The method according to claim 4, wherein: The step of encoding the point cloud data further includes: The reference road frame is initialized.

9. The method according to claim 1, wherein: The bitstream includes at least one of the following: Information indicating whether to generate a reference road frame; or Information indicating whether the current frame is a new scene.

10. A device for transmitting point cloud data, the device comprising: an encoder configured to encode point cloud data; as well as A transmitter is configured to transmit a bit stream containing the point cloud data.

11. A method for receiving point cloud data, the method comprising the following steps: receiving a bitstream containing point cloud data; as well as The point cloud data is decoded.

12. The method according to claim 11, wherein: The point cloud data includes points related to the road and points related to the object, In which, in the frame containing the point related to the road, the point related to the road is changed according to the object related to the road, or is missing.

13. The method according to claim 11, wherein: The point cloud data is obtained by LiDAR. Wherein, the point cloud data includes points based on the laser ID of the LiDAR.

14. The method according to claim 11, wherein: The step of decoding the point cloud data comprises: decoding a road frame of a current frame containing the point cloud data, The step of decoding the road frame comprises: predicting the road frame of the current frame based on a reference road frame for the road frame of the current frame; searching the reference road frame for missing points in the road frame based on at least one of a laser ID or an angle; and The missing points in the road frame are updated.

15. A device for receiving point cloud data, the device comprising: a receiver configured to receive a bitstream containing point cloud data; as well as A decoder is configured to decode the point cloud data.