Point cloud compression method and device
The point cloud is approximated into multiple triangles through inter prediction method, and local motion compensation and entropy coding technology are used to solve the problem of low compression efficiency of point cloud data and achieve higher coding efficiency.
Patent Information
- Application Number
- CN202380036136.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-04-28
- Filing Date
- 2023-04-28
- Publication Date
- 2024-12-06
AI Technical Summary
The prior art is difficult to effectively process and compress point cloud data, resulting in inefficient encoding.
The inter-frame prediction method is used to approximate the reference point cloud into multiple triangles, and the encoding efficiency of the point cloud is improved through local motion compensation and entropy coding technology.
The encoding efficiency of video signals is improved, and the encoding efficiency is significantly improved by approximating point clouds into triangles.
Smart Images

Figure CN119096550A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to encoders and decoders, and more particularly, to methods and apparatus for encoding and decoding for point cloud compression. Background Art
[0002] Point cloud content is content represented as a point cloud, which is a collection of points belonging to a coordinate system representing a three-dimensional space. Point clouds can represent three-dimensional media and can be used to provide various services such as VR (virtual reality), AR (augmented reality), MR (mixed reality), XR (extended reality), and autonomous driving services.
[0003] Since a large amount of point data is required to represent point cloud content, a method for efficiently processing the point cloud data is needed. Summary of the invention
[0004]
Technical issues
[0005] The present invention proposes an inter-frame prediction method and device for compressing point clouds.
[0006] In this case, the inter-frame prediction method proposed in the present invention aims to provide high encoding efficiency by approximating a reference point cloud into a plurality of triangles and using the plurality of triangles for prediction.
[0007]
Technical Solution
[0008] A point cloud compression method and device for solving the above-mentioned problem are provided.
[0009] According to an embodiment of the present disclosure, a point cloud compression method and device may: perform global motion compensation based on a previous frame of a current frame; divide the current frame into a plurality of prediction units; determine a motion compensated point within the current prediction unit by performing local motion compensation on the current prediction unit; and determine geometric information of a current point within the current prediction unit based on the motion compensated point.
[0010] According to an embodiment of the present disclosure, a point cloud compression method and apparatus may: derive a predicted statistic based on motion-compensated points included in a predefined specific area around a current point; and determine geometric information of the current point based on the predicted statistic.
[0011] In a point cloud compression method and apparatus according to one embodiment of the present disclosure, the geometric information of the current point may include a symbol indicating whether the current point is occupied.
[0012] In a point cloud compression method and apparatus according to one embodiment of the present disclosure, a symbol indicating whether a current point is occupied may be entropy encoded using a predicted statistical value as a probability.
[0013] In a point cloud compression method and apparatus according to one embodiment of the present disclosure, a specific area around a current point may be defined as an area including voxels adjacent to the front, back, left, right, top, and bottom of the current point.
[0014] In a point cloud compression method and apparatus according to one embodiment of the present disclosure, a specific area around a current point is defined as a hexahedral area centered at the current point and including adjacent voxels adjacent to the current point.
[0015] According to an embodiment of the present disclosure, a point cloud compression method and apparatus may: divide a current prediction unit into at least one triangle based on a motion compensated point; and determine a point included in a plane of the divided triangle based on the motion compensated point.
[0016] The point cloud compression method and apparatus according to one embodiment of the present disclosure may determine geometric information of a current point based on at least one of a motion-compensated point or a point included in a plane of a divided triangle.
[0017] According to an embodiment of the present disclosure, a point cloud compression method and apparatus may: obtain multiple vertices based on multiple motion compensated points adjacent to an edge of a current prediction unit; and divide the current prediction unit into at least one triangle by connecting the multiple vertices.
[0018] In a point cloud compression method and apparatus according to one embodiment of the present disclosure, a current prediction unit is divided into regions having a size of 1x1x1 based on a predefined tree structure, and a current point corresponds to a center position of the region having a size of 1x1x1.
[0019] In a point cloud compression method and apparatus according to one embodiment of the present disclosure, the predefined tree structure includes at least one of a binary tree, a quadtree, or an octree.
[0020]
Technical Effect
[0021] The point cloud compression method and device according to the present invention can improve the encoding efficiency of video signals.
[0022] In addition, by approximating a reference point cloud according to the present invention into a plurality of triangles and using the plurality of triangles for inter-frame prediction, encoding efficiency can be improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 is a block diagram illustrating an example of a point cloud encoder according to one embodiment of the present disclosure.
[0024] Figure 2 is a block diagram illustrating an example of a point cloud decoder according to one embodiment of the present disclosure.
[0025] Figure 3 is a block diagram illustrating an example of a geometry information encoding unit according to one embodiment of the present disclosure.
[0026] Figure 4 is a block diagram illustrating an example of a geometric information decoding unit according to one embodiment of the present disclosure.
[0027] Figure 5 is a diagram showing an example of expressing a point in a three-dimensional space in two dimensions according to one embodiment of the present disclosure.
[0028] Figure 6 is a diagram illustrating an example of statistic value prediction of surrounding points performed by an intra prediction unit according to one embodiment of the present disclosure.
[0029] Figure 7 is a diagram illustrating an example of local motion compensation performed in an inter prediction unit according to one embodiment of the present disclosure. Figure 8 is a diagram illustrating an example of a method for generating prediction statistics using motion compensation points in an inter prediction unit according to an embodiment of the present disclosure.
[0030] Fig. 9 is a diagram illustrating an example of a method for generating prediction statistics using motion compensation points in an inter prediction unit according to one embodiment of the present disclosure.
[0031] Fig.10 is a diagram illustrating an example of a method for generating prediction statistics using motion compensation points in an inter prediction unit according to an embodiment of the present disclosure.
[0032] Fig.11 is a diagram illustrating an example of a method for generating a triangle for inter prediction according to one embodiment of the present disclosure.
[0033] Fig.12 is a diagram illustrating an example of a method for generating prediction statistics using motion compensation points in an inter prediction unit according to an embodiment of the present disclosure.
[0034] Fig.13 is a diagram illustrating an example of a method for generating prediction statistics using motion compensation points in an inter prediction unit according to an embodiment of the present disclosure.
[0035] Fig.14 is a flow chart illustrating a point cloud compression method according to one embodiment of the present disclosure. DETAILED DESCRIPTION
[0036] The embodiments of the present disclosure will be described in detail so that those skilled in the art can easily implement the embodiments by referring to the drawings attached to this specification. However, the present disclosure can be implemented in different forms, and the present disclosure is not limited to the embodiments described herein. Moreover, in order to clearly describe the present disclosure in the drawings, parts that are not related to the description are omitted, and similar reference numerals are attached to similar parts throughout this specification.
[0037] Throughout this specification, when a part is referred to as being “connected” to another part, it can include an electrical connection as well as a direct connection with other elements present therebetween.
[0038] In addition, throughout this specification, when a part is referred to as “comprising” a component, this means that other components may also be included without excluding other components, unless otherwise stated.
[0039] In addition, terms such as first, second, etc. may be used to describe various components, but the components should not be limited by these terms. The terms are only used to distinguish one component from other components.
[0040] In addition, for the embodiments of the device and method described in this specification, some configurations of the device or some steps of the method may be omitted. In addition, the order of some configurations of the device or some steps of the method may be changed. In addition, other configurations or other steps may be inserted into some configurations of the device or some steps of the method.
[0041] In addition, some configurations or some steps of the first embodiment of the present disclosure may be added to the second embodiment of the present disclosure, or may replace some configurations or some steps of the second embodiment.
[0042] In addition, the construction units shown in the embodiments of the present disclosure are shown separately to represent different characteristic functions, and they do not mean that each construction unit is configured with separate hardware or a software construction unit. In other words, for ease of description, each construction unit is described by being listed as each construction unit, and at least two construction units in each construction unit can be combined to form a construction unit, or a construction unit can be divided into multiple construction units to perform functions. The integrated implementation and separation implementation of each construction unit are also included in the scope of the rights of the present disclosure, unless they depart from the spirit of the present disclosure.
[0043] First, the terms used in this application can be briefly described as follows.
[0044] The decoding device (video decoding equipment) to be described later may be a device included in a server terminal (for example, a civilian security camera device, a civilian security system, a military security camera device, a military security system, a personal computer (PC), a notebook computer, a portable multimedia player (PMP), a wireless communication terminal, a smart phone, a TV application server, and a service server, etc.), and the decoding device (video decoding equipment) may refer to a user terminal equipped with various equipment, a communication device including a communication modem for communicating with a wired / wireless communication network, a memory for storing various programs and data for decoding images or performing intra-frame prediction or inter-frame prediction for decoding, a microprocessor for executing programs and performing operations and controls, and other various devices.
[0045] In addition, the image encoded as a bit stream by the encoder can be transmitted to the image decoding device in real time or non-real time through various communication interfaces (e.g., cables, universal serial bus (USB), etc.) or through wired or wireless communication networks (e.g., the Internet, wireless local area network, wireless LAN network, Wi-Bro network, mobile communication network, etc.), decoded and reconstructed and reproduced as an image. Alternatively, the bit stream generated by the encoder can be stored in a memory. The memory may include both volatile memory and non-volatile memory. In this specification, the memory may be represented as a recording medium storing a bit stream.
[0046] Generally, a video may be configured with a series of pictures, and each picture may be divided into coding units such as blocks. In addition, a person skilled in the art to which the present embodiment belongs may understand that the term "picture" input below may be used by replacing it with other terms having the same meaning as image, frame, etc. In addition, a person skilled in the art to which the present embodiment belongs may understand that the term "coding unit" may be used by replacing it with other terms having the same meaning as unit block, block, etc.
[0047] Hereinafter, with reference to the accompanying drawings, embodiments of the present disclosure are described in more detail. When describing the present disclosure, for the same components, repeated description is omitted.
[0048] Figure 1 is a block diagram illustrating an example of a point cloud encoder according to one embodiment of the present disclosure.
[0049] Reference Figure 1, the point cloud encoder (100) may include a coordinate system transformation unit (110), a geometric information encoding unit (120), a storage unit (130), a geometric information decoding unit (140), an attribute information correction unit (150), an attribute information encoding unit (160), an attribute information decoding unit (170) and an entropy encoding unit (180). Figure 1 The point cloud encoder (100) shown is an example, but the present disclosure is not limited thereto, and the point cloud encoder (100) may be configured in addition to Figure 1 Add other configurations than shown or by omitting Figure 1 Some of the configurations shown can be implemented.
[0050] The point cloud encoder (100) can receive a raw point cloud and perform encoding to generate a bitstream. The point cloud can be a point cloud content represented as a point cloud, which is a collection of points located in a three-dimensional space, and the point cloud can be referred to as point cloud video data, etc. The point cloud according to an embodiment of the present disclosure can include one or more frames. A frame represents a still image / picture. Therefore, the point cloud can include a point cloud image / frame / picture and can be referred to as any one of a point cloud image, a frame, and a picture.
[0051] In the present disclosure, a point cloud may include one or more frames. In addition, point cloud data may include geometric information and / or attribute information. The geometric information may include location information of the point. The attribute information may include information indicating the attributes of the point. As an example, the attribute information may include information about the texture information, color (YCbCr or RGB), reflectivity, transparency, etc. of the point.
[0052] The coordinate system transformation unit (110) can perform coordinate system transformation by receiving information of the original point cloud. The original point cloud can be obtained from one or more cameras. In this case, since the coordinate system of each camera can be different, the coordinate system transformation of the point can be performed based on the position coordinates of each camera. The coordinate system transformation can be performed on the points of the point cloud video obtained from each camera to generate a point cloud data. Therefore, a point cloud content representing a wide range can be generated, or a point cloud content with a high density of points can be generated.
[0053] As an example, the coordinate system transformation may mean a transformation from a world coordinate system to a frame coordinate system. Alternatively, it may mean a transformation from an orthogonal coordinate system to a cylindrical coordinate system or a spherical coordinate system. Alternatively, it may mean a transformation from a cylindrical coordinate system or a spherical coordinate system to an orthogonal coordinate system. The geometric information of the coordinate system transformation may be sent to a geometric information encoding unit (120).
[0054] The geometry information encoding unit (120) may receive the geometry information and the restored point cloud, and encode them to generate geometry information related symbols. The generated geometry information related symbols may be sent to the entropy encoding unit (180) and the geometry information decoding unit (140). The geometry information encoding unit (120) will be later Figure 3 Described in detail in.
[0055] The geometry information decoding unit (140) may receive the geometry information related symbols and the restored point cloud, and decode them to restore the geometry information of the point cloud. The restored geometry information may be sent to the attribute information correction unit (150). In addition, the restored geometry information may be sent to the storage unit (130) together with the restored point cloud attribute information. The geometry information decoding unit (140) will be later Figure 4 Described in detail in.
[0056] The attribute information correction unit (150) may receive the restored geometric information and the original point cloud, correct the attribute information, and then transmit the corrected attribute information to the attribute information encoding unit (160). Here, the attribute information correction may be a process of correcting the attribute information by taking into account an error caused by encoding / decoding of the geometric information. In this case, a new point may be created or removed during the geometric information encoding / decoding process. In this case, a process of reallocating the attribute information according to the restored geometric information based on the attribute information of the original point cloud may be performed. As an example, the attribute information may be allocated as a value calculated using the attribute information of a nearby point based on the distance between the current point and a point of the original point cloud.
[0057] The attribute information encoding unit (160) can encode the attribute information of the point cloud using the input restored geometric information and the corrected attribute information, and generate a symbol related to the attribute information. The generated attribute information related symbol can be transmitted to the entropy encoding unit (180). In addition, the attribute information related symbol can be transmitted to the attribute information decoding unit (170).
[0058] The entropy coding unit (180) may perform entropy coding on the received geometric information related symbols and attribute information related symbols to generate a bit stream. The generated bit stream may be output from the point cloud encoder (100).
[0059] The attribute information decoding unit (170) can decode the input attribute information related symbols to restore the point cloud attribute information. The restored attribute information can be transmitted to the storage unit (130) together with the restored geometric information.
[0060] The storage unit (130) can store the geometric information and attribute information of the input restored point cloud. The stored point cloud can be sent to the geometric information encoding unit (120) and the geometric information decoding unit (140).
[0061] Figure 2 is a block diagram illustrating an example of a point cloud decoder according to one embodiment of the present disclosure.
[0062] Reference Figure 2 , the point cloud decoder (200) may include an entropy decoding unit (210), a geometric information decoding unit (220), an attribute information decoding unit (230), a storage unit (240) and a coordinate system inverse transformation unit (250). Figure 2 The point cloud decoder (200) shown is an example, but the present disclosure is not limited thereto, and the point cloud decoder (200) may be configured in addition to Figure 2 Adding other components than those shown or by omitting Figure 1 Some of the components shown are implemented. In addition, Figure 2 The point cloud decoder (200) may be executed with Figure 1 The operations of the point cloud encoder (100) are substantially the same as those of the point cloud encoder (100). In this regard, the Figure 1 Repeat any description.
[0063] The point cloud decoder can receive the bitstream and perform decoding to recover the point cloud.
[0064] The entropy decoding unit (210) can receive the bit stream and decode it to recover the geometric information related symbols and attribute information related symbols of the point cloud. The recovered geometric information related symbols can be transmitted to the geometric information decoding unit (220). In addition, the recovered attribute information related symbols can be transmitted to the attribute information decoding unit (230).
[0065] The geometry information decoding unit (220) can receive the restored geometry information related symbols and the restored point cloud and decode them to restore the geometry information of the point cloud. The restored geometry information can be transmitted to the storage unit (240) in the form of a restored point cloud together with the attribute information decoding unit and the restored attribute information.
[0066] The attribute information decoding unit (230) may receive the restored attribute information related symbols and the restored geometric information and decode them to restore the attribute information. The restored attribute information may be stored in the storage unit (240) in the form of a restored point cloud together with the restored geometric information.
[0067] The storage unit (240) can store the received restored point cloud. Thereafter, the stored point cloud can be transmitted to the geometric information decoding unit (220) for the next point decoding. In addition, the restored point cloud can be transmitted to the coordinate system inverse transformation unit (250) for output.
[0068] The coordinate system inverse transformation unit (250) can output (or present) by performing an inverse coordinate system transformation on the geometric information of the input restored point cloud. Here, the coordinate system inverse transformation can be an inverse process of the coordinate system transformation performed by the point cloud encoder. In this case, the coordinate system inverse transformation can be performed based on information received from the point cloud encoder (100).
[0069] Figure 3 is a block diagram illustrating an example of a geometry information encoding unit according to one embodiment of the present disclosure.
[0070] Reference Figure 3 , the geometry information encoding unit (300) may include a point segmentation unit (310), a geometry information symbol generation unit (320), an intra-frame prediction unit (330) and an inter-frame prediction unit (340). Figure 3 The geometric information encoding unit (300) shown is an example, but the present disclosure is not limited thereto, and the geometric information encoding unit (300) may be configured in addition to Figure 3 Add other configurations than shown or by omitting Figure 3 Some of the configurations shown can be implemented. Figure 3 The geometric information encoding unit (300) may be the above Figure 1 An example of a geometric information encoding unit (120).
[0071] The geometry information encoding unit (300) can receive the geometry information of the original point cloud and the restored point cloud, and encode them to generate a geometry information symbol.
[0072] The point segmentation unit (310) may receive geometric information of the original point cloud and segment the points. In this case, the point segmentation may be a step of recursively segmenting the three-dimensional volume based on a binary tree, a quadtree or an octree.
[0073] Here, the last node of the segmented tree structure can be a three-dimensional cubic space with a volume of 1x1x1. In the present disclosure, the last node of this segmented tree structure can be called a voxel. A voxel can mean the smallest unit representing position information in a three-dimensional space. A point cloud according to an embodiment of the present disclosure can be included in one or more voxels. A point group in a three-dimensional space can be matched with a voxel. As an example, a binary tree, a quadtree, or an octree segmentation structure can represent a point that matches a voxel based on a binary tree, a quadtree, or an octree structure, respectively.
[0074] As an example, when segmentation is performed according to a binary tree, quadtree, or octree segmentation structure, segmentation may be performed until the leaf nodes of each tree structure become voxels. In this case, in the process of performing recursive segmentation, whether at least one point is included in the node may be considered. That is, additional segmentation may not be performed on nodes that do not include at least one point.
[0075] The segmented point cloud may be sent to the geometry information symbol generation unit (320). That is, the point segmentation unit (310) may mean a process of converting and representing the coordinate values in the three-dimensional space as a tree structure.
[0076] The intra-frame prediction unit (330) can receive the restored point cloud as an input, generate a predicted statistical value of occupancy information about the current point, and send the predicted statistical value to the geometric information symbol generation unit (320). Here, the predicted statistical value can be a probability value about whether the current point occupies the current position. Alternatively, the predicted statistical value can be a probability value of a flag about whether to segment in the segmentation information generated in the process of being segmented into a tree structure. The predicted statistical value of the present disclosure is a value used to generate a geometric information symbol representing occupancy information, and is not limited to its name, and can also be referred to as prediction, statistics, probability, predicted value, statistical value, probability value, occupancy prediction, occupancy statistics, occupancy probability, occupancy prediction value, occupancy statistical value, occupancy probability value, context, occupancy context, etc.
[0077] The inter-frame prediction unit (340) can perform global motion compensation and / or local motion compensation on the input restored point cloud, generate prediction statistics using compensated points around the current point, and transmit the prediction statistics to the geometry information symbol generation unit (320).
[0078] The geometry information symbol generation unit (320) can receive the segmented point cloud and the predicted statistical value, and generate a geometry information symbol of the current point. The generated geometry information symbol can be output.
[0079] Figure 4 is a block diagram illustrating an example of a geometric information decoding unit according to one embodiment of the present disclosure.
[0080] Reference Figure 4 , the geometric information decoding unit (400) may include a partition structure recovery unit (410), an intra-frame prediction unit (420) and an inter-frame prediction unit (430). Figure 4 The geometric information decoding unit (400) shown is an example, but the present disclosure is not limited thereto, and the geometric information decoding unit (400) may be implemented by Figure 4 Adding other components than those shown or by omitting Figure 4 Some of the components shown are implemented. Figure 4 The geometric information decoding unit (400) may be the one described above. Figure 1 A geometric information decoding unit (140) and Figure 2 An example of a geometric information decoding unit (220).
[0081] in addition, Figure 4 The geometric information decoding unit (400) can perform the same Figure 3 The operation of the geometric information encoding unit (300) is substantially the same as that of the geometric information encoding unit (300). In this regard, the Figure 3 Repeat any description.
[0082] The geometry information decoding unit (400) can receive the restored geometry information symbol and the restored point cloud and decode them to restore the geometry information of the point cloud.
[0083] The intra prediction unit (420) may receive the restored point cloud, generate an intra prediction statistic about occupancy information of a current point, and send the intra prediction statistic to the segmentation structure restoration unit (410).
[0084] The inter-frame prediction unit (430) can perform global motion compensation and local motion compensation on the input restored point cloud, generate prediction statistics using compensated points around the current point, and transmit the prediction statistics to the segmentation structure restoration unit (410).
[0085] The segmentation structure recovery unit (410) can use the received recovered geometric information symbols and prediction statistics to recover the tree segmentation structure. As an example, the segmentation structure recovery unit (410) can be a geometric information recovery unit or a geometric information generation unit. The geometric information of the recovered points according to the tree structure can be output.
[0086] Figure 5 is a diagram showing an example of expressing a point in a three-dimensional space in two dimensions according to one embodiment of the present disclosure.
[0087] Reference Figure 5 For example, Figure 5 The three circles (or dots) in represent points respectively. The current point and the point to be encoded / decoded may be included in the current frame. In the present disclosure, the point to be encoded / decoded may mean a point to be encoded / decoded after the current point according to the encoding / decoding order.
[0088] Each point exists at a specific position within the current frame, and the coordinate value of the point can be expressed as a real number. In this case, unlike general images, since point clouds do not exist at all positions, there may be many limitations in using information about the points recovered around when using encoding / decoding.
[0089] In addition, in the case of general images / videos, if the starting position such as the upper left or lower right is determined, encoding / decoding can be performed sequentially, but in the case of point clouds, it is difficult to determine the order of all points, and since the original point cloud data is in the form of general text and the geometric information and attribute information of the points are listed, there is a problem that the order of the points does not match the spatial order. To solve this problem, even if the order is given by spatial position, there may be a mismatch between the distance in the order and the distance in the space because not all spaces are filled.
[0090] Therefore, in order to solve these problems, in order to more efficiently represent the position information of the point in the process of geometric information encoding / decoding, a tree structure can be used to represent the position information of the point as a voxel of a volume having a size of 1x1x1. As described above, a voxel may mean the smallest unit representing position information in a three-dimensional space. A point cloud according to an embodiment of the present disclosure may be included in one or more voxels. A point group in a three-dimensional space may be matched with a voxel. In this case, a voxel is a concept similar to a pixel of an image / video, but may additionally have information about whether it is occupied.
[0091] Figure 6 is a diagram illustrating an example of statistic value prediction of surrounding points performed by an intra prediction unit according to one embodiment of the present disclosure.
[0092] Reference Figure 6 , the current frame can be divided into a plurality of voxels having a size of 1x1x1. Each voxel can have occupancy information of a point. In this case, since encoding the occupancy information of all voxels results in low coding efficiency, the occupancy information can be efficiently encoded through a recursive tree partitioning structure.
[0093] In one embodiment of the present disclosure, the predicted statistical value of the current point (610) may be calculated (or derived) based on the occupancy information of the surrounding voxels of the current point (610).
[0094] As an example, the predicted statistical value of the current point (600) may be derived based on the points included in the statistical information extraction area (610).
[0095] The statistics extraction area (610) may be Figure 6 The shaded area in , which represents the area used to obtain the predicted statistical value. In addition, Figure 6 The three-dimensional area in the right figure of may be an area representing the statistical information extraction area (610) represented in two dimensions in three-dimensional space.
[0096] The symbol of the current point (600) can be determined according to the occupancy of the points included in the statistical information extraction area (610). In other words, the predicted statistical value of the current point (600) can be determined according to the occupancy of the points included in the statistical information extraction area (610), and a symbol representing the occupancy information of the current point (600) can be generated based on this.
[0097] Figure 7 is a diagram showing an example of local motion compensation performed in an inter-frame prediction unit according to one embodiment of the present disclosure. For ease of representation, Figure 7 It can be a two-dimensional representation of a three-dimensional space.
[0098] Reference Figure 7 As described above, the inter-frame prediction unit may first perform global motion compensation on the previous frame and then perform local motion compensation. Global motion compensation may refer to a higher level of motion compensation, and local motion compensation may refer to a lower level of motion compensation. For example, global motion compensation may be performed at a frame level, and local motion compensation may be performed at a block (or may be referred to as a processing unit, prediction unit, or voxel) level within a frame.
[0099] In this case, the current frame and the previous frame may be divided into a plurality of blocks for local motion compensation. Local motion compensation may be performed individually for the divided blocks.
[0100] The motion information used for motion compensation can be sent to the decoder via the entropy encoder. The decoder can receive the information and perform local motion compensation in the same way as the encoder. As an example, the motion information can be Figure 7 Alternatively, the motion information may be in the form of a matrix for three-dimensional coordinate system transformation.
[0101] Figure 8 is a diagram illustrating an example of a method for generating prediction statistics using a point of motion compensation in an inter prediction unit according to an embodiment of the present disclosure.
[0102] Reference Figure 8 , points for local motion compensation (points indicated by hatching) may exist around the current point (800).
[0103] In this case, based on Figure 8 The predicted statistical value is obtained by extracting the occupancy information of the compensated points around the current point (800) in the statistical information extraction area (810).
[0104] As an example, the predicted statistic may be calculated based on the number of restored points included in the statistical information extraction area (810). For example, the predicted statistic may be derived as a ratio of the number of points that may be included in the statistical information extraction area (810) to the number of points currently included.
[0105] As an example, see Figure 8 In the example of the left figure, the number of points that can be included in the statistical information extraction area (810) is 5, and the number of restored points currently included is 1, in which case the predicted statistical value can be derived as 0.2.
[0106] Alternatively, as an example, if a maximum of 11 points can be included in the statistical information extraction area (810) represented in three dimensions, as in Figure 8 In the example of the right figure of , the predicted statistical value can be derived as 1 / 11. As another example, the predicted statistical value can be calculated by applying the distance between the position of the current point (800) and the position of the points that can be included in the statistical information extraction area (810) as a weight. In this case, various known distance calculation methods can be used, such as Manhattan distance, Euclidean distance, and Hamming distance. Thereafter, the predicted statistical value can be obtained by dividing the sum of the weights of the restored points included in the statistical information extraction area (810) by the sum of all weights.
[0107] The prediction statistic can be used for entropy encoding / decoding in a subsequent entropy encoding unit / entropy decoding unit. As an example, the entropy encoding unit / entropy decoding unit can use the prediction statistic as a probability value of a symbol indicating whether the current point (800) is occupied.
[0108] Fig. 9 is a diagram illustrating an example of a method for generating prediction statistics using a point of motion compensation in an inter prediction unit according to one embodiment of the present disclosure.
[0109] Reference Fig. 9 , points for local motion compensation (points indicated by hatching) may exist around the current point.
[0110] Can be based on Fig. 9 The predicted statistical value is obtained by extracting the occupancy information of the compensated points around the current point (900) in the statistical information extraction area (910). In this case, since the motion compensated points can exist in all surrounding environments centered on the current point, regardless of the encoding / decoding order of the current point (900), it can be defined as follows Fig. 9 The statistical information extraction area (910) is shown.
[0111] based on Fig. 9 The statistical information extraction area (910) may include upper, lower, left and right surrounding points, and based on Fig. 9 In the three-dimensional area of the right figure, the statistical information extraction area (910) may include surrounding points at the top, bottom, left, right, front, and back.
[0112] like Fig. 9 As shown, the predicted statistical value can be obtained by including the surrounding occupancy information in the statistical information extraction area (910).
[0113] Fig.10 is a diagram illustrating an example of a method for generating prediction statistics using a point of motion compensation in an inter prediction unit according to an embodiment of the present disclosure.
[0114] Reference Fig.10 , points for local motion compensation (points indicated by hatching) may exist around the current point.
[0115] Can be based on Fig.10 The predicted statistical value is obtained by extracting the occupancy information of the compensated points around the current point (1000) in the statistical information extraction area (1010).
[0116] In this case, since the motion-compensated point can exist in all surroundings centered on the current point regardless of the encoding / decoding order of the current point, it can be defined as follows Fig.10 The statistical information extraction area (1010) is shown.
[0117] based on Fig.10 The statistical information extraction area (1010) may include surrounding points at the top, bottom, left, right, upper left, upper right, lower left and lower right, and based on Fig.10 In the three-dimensional area of the right figure, the statistical information extraction area (1010) may include surrounding points of the hexahedral area.
[0118] like Fig.10 As shown, the predicted statistical value can be obtained through the occupancy information of the area included in the statistical information extraction area (1010).
[0119] Fig.11 is a diagram illustrating an example of a method for generating a triangle for inter prediction according to one embodiment of the present disclosure.
[0120] Reference Fig.11 , you can Fig.11As shown in the left figure of , there are points for local motion compensation (points indicated by hatching). In this case, the points for motion compensation may not exist at a position that completely overlaps with the current point, and may not exist around the current point. This may reduce the encoding efficiency of the point occupancy information.
[0121] In order to solve this problem, according to one embodiment of the present disclosure, Fig.11 As shown in the right figure, a triangle can be generated based on the motion compensated points in the current block, and the voxels overlapping the triangle are considered to be occupied points, so that the local motion compensated points can be approximated as the faces of the triangle.
[0122] In this case, triangle vertices may be generated based on points close to corners of the current block, and triangles may be generated by connecting the vertices of the generated triangles. The number of triangles may be determined according to the number of vertices of the generated triangles.
[0123] As an example, when generating multiple triangles, the order of the vertices may be determined based on the positions of the vertices in the three-dimensional space. For example, the order of the vertices may be determined by projecting the vertices onto the xy plane, the yz plane, and the zx plane; determining a plane based on the area of the rectangular surface generated from the vertices; and determining the order of the vertices projected onto the plane based on a clockwise direction. Alternatively, the order may be determined based on a counterclockwise direction. Thereafter, multiple triangles may be generated by connecting the vertices according to the determined order.
[0124] Fig.12 is a diagram illustrating an example of a method for generating prediction statistics using a point of motion compensation in an inter prediction unit according to an embodiment of the present disclosure.
[0125] Reference Fig.12 , the points indicated by the hatched lines can be found through the Fig.11 The points generated by the triangular approximation described in .
[0126] In this case, based on Fig.12 The predicted statistical value is obtained by extracting the occupancy information of the compensated points around the current point (1200) in the statistical information extraction area (1210).
[0127] Here, the triangle approximation point may exist in all surroundings centered on the current point (1200), regardless of the order of encoding / decoding the current point (1200). Fig.12 In the three-dimensional statistical information extraction area (1210) of the right figure, the predicted statistical value can be obtained through the occupancy information around the current point (1200) (above, below, front, back, left and right).
[0128] Fig.13 is a diagram illustrating an example of a method for generating prediction statistics using a point of motion compensation in an inter prediction unit according to an embodiment of the present disclosure.
[0129] Reference Fig.13 , the points indicated by the hatched lines can be found through the Fig.11 The points generated by the triangular approximation described in .
[0130] Can be based on Fig.13 The predicted statistical value is obtained by extracting the occupancy information of the compensated points around the current point (1300) in the statistical information extraction area (1310).
[0131] Here, the triangle approximation point can exist in all surroundings centered on the current point, regardless of the order of encoding / decoding the current point (1300). Fig.13 The predicted statistical value is obtained by extracting the three-dimensional statistical information of the area (1310) on the right and the occupancy information of the hexahedral area centered on the current point (1300).
[0132] Fig.14 is a flow chart illustrating a point cloud compression method according to one embodiment of the present disclosure.
[0133] Reference Fig.14 , the point cloud compression device can perform global motion compensation based on a previous frame of the current frame (S1400).
[0134] The point cloud compression apparatus may divide the current frame into a plurality of prediction units (S1410).
[0135] The point cloud compression apparatus may determine a point of motion compensation within the current prediction unit by performing local motion compensation on the current prediction unit ( S1420 ).
[0136] The point cloud compression apparatus may determine geometric information of a current point within a current prediction unit based on the motion compensated point ( S1430 ).
[0137] As described above, the point cloud compression device can derive prediction statistics based on motion-compensated points included in a predefined specific area around the current point, and determine geometric information of the current point based on the prediction statistics.
[0138] In addition, as described above, the geometric information of the current point may include a symbol indicating whether the current point is occupied. The symbol indicating whether the current point is occupied may be entropy encoded using the prediction statistic as a probability.
[0139] In addition, as mentioned above Fig. 9 and Fig.12As described in, the specific area around the current point can be defined as an area including voxels adjacent to the front side, the back side, the left side, the right side, the upper side, and the lower side of the current point.
[0140] In addition, as mentioned above Fig.10 and Fig.13 As described in , the specific area around the current point can be defined as a hexahedral area centered at the current point and including neighboring voxels adjacent to the current point.
[0141] In addition, as mentioned above Figures 11 to 13 The current prediction unit may be divided into at least one triangle based on the motion compensated point, and a point included in a plane of the divided triangle may be determined based on the motion compensated point. Geometric information of the current point may be determined based on at least one of the motion compensated point or the point included in the plane of the divided triangle. A plurality of vertices may be obtained based on a plurality of motion compensated points adjacent to a corner of the current prediction unit, and the current prediction unit may be divided into at least one triangle by connecting the plurality of vertices.
[0142] In addition, as described above, the current prediction unit may be divided into a region having a size of 1x1x1 based on a predefined tree structure, and the current point may correspond to a center position of the region having a size of 1x1x1. In this case, the predefined tree structure may include at least one of a binary tree, a quadtree, or an octree.
[0143] The above-mentioned embodiments may be a combination of components and features of the present disclosure in a predetermined form. Unless otherwise expressly stated, each component or feature should be considered to be selective. Each component or feature may be implemented in a form not combined with other components or features. In addition, some components and / or features may be combined to configure the embodiments of the present disclosure. The order of the operations described in the embodiments of the present disclosure may be changed. Some configurations or features of the embodiments may be included in other embodiments, or may be replaced with configurations or features corresponding to other embodiments. It is obvious that claims that do not have a clear reference relationship in the scope of the claims may be combined to configure the embodiments, or may be included as new claims by modification after the application.
[0144] The embodiments according to the present disclosure may be implemented in various ways, such as hardware, firmware, software, or a combination thereof, etc. For implementation by hardware, the embodiments of the present disclosure may be implemented by one or more ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), processors, controllers, microcontrollers, microprocessors, etc.
[0145] In addition, for the implementation by firmware or software, the embodiments of the present disclosure can be implemented in the form of modules, programs, functions, etc. that perform the above functions or operations, and can be recorded in a readable recording medium by various computer means. Here, the recording medium may include program instructions, data files, data structures, etc., alone or in combination. The program instructions recorded in the recording medium may be those program instructions specially designed and configured for the present disclosure, or those program instructions available by notifying computer software technicians. For example, the recording medium includes: magnetic media, such as hard disks, floppy disks, and tapes; optical media, such as CD-ROMs (compact disk read-only memory) and DVDs (digital video disks); magneto-optical media, such as optical floppy disks; and hardware devices specially configured to store and execute program instructions, such as ROMs, RAMs, flash memories, etc. Examples of program instructions may include high-level language codes that can be executed by a computer using an interpreter, etc., and machine language codes similar to those made by a compiler. Such a hardware device may be configured to operate as at least one software module for performing the operations of the present disclosure, and vice versa.
[0146] In addition, the device or terminal according to the present disclosure can be driven by a command that causes at least one processor to perform the above functions and processes. For example, such a command may include, for example, an interpreted command, such as a script command such as a JavaScript or ECMAScript command, or other commands stored in a computer-readable medium readable or executable code. In addition, the device according to the present disclosure can be implemented in a distributed manner across a network such as a server group, or can be implemented in a single computer device.
[0147] In addition, a computer program (also referred to as a program, software, software application, script or code) provided together with an apparatus according to the present disclosure and executing a method according to the present disclosure can be written in a programming language including a compiled or interpreted language or any form of a priori or procedural language, and can be deployed in any form including independent programs, modules, components or subroutines or other units suitable for use in a computer environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a single file provided for a requested program, or in a plurality of interactive files (e.g., a file storing a portion of at least one module, subroutine or code), or in a portion of a file having other programs or data (e.g., at least one script stored in a markup language document). A computer program may be located in a site or distributed across multiple sites, and may be deployed to be executed on a computer or on multiple computers interconnected by a communication network.
[0148] It is obvious to those skilled in the art that the present disclosure may be implemented in other specific forms without departing from the basic features of the present disclosure. Therefore, the detailed description described above should not be interpreted restrictively in all aspects, but should be regarded as illustrative. The scope of the present disclosure should be determined by a reasonable interpretation of the attached claims, and all changes within the equivalent range of the present disclosure are included within the scope of the present disclosure.
Claims
1. A method for compressing a point cloud, comprising: performing global motion compensation based on previous frames of the current frame; Dividing the current frame into a plurality of prediction units; determining a point of motion compensation within the current prediction unit by performing local motion compensation on the current prediction unit; and Geometric information of a current point within the current prediction unit is determined based on the motion compensated point.
2. The method according to claim 1, wherein: Determining the geometric information of the current point includes: deriving a prediction statistic based on motion compensated points included in a predefined specific area around the current point; and The geometric information of the current point is determined based on the predicted statistical value.
3. The method according to claim 2, wherein: The geometric information of the current point includes a symbol indicating whether the current point is occupied, and The predicted statistical value is used as a probability to perform entropy coding on a symbol indicating whether the current point is occupied.
4. The method according to claim 2, wherein: The specific area around the current point is defined as an area including voxels adjacent to the front side, the back side, the left side, the right side, the upper side, and the lower side of the current point.
5. The method according to claim 2, wherein: The specific area around the current point is defined as a hexahedral area centered at the current point and including neighboring voxels adjacent to the current point.
6. The method according to claim 1, further comprising: Dividing the current prediction unit into at least one triangle based on the motion compensated point; as well as Points included in a plane of the divided triangle are determined based on the motion-compensated points.
7. The method according to claim 6, wherein: Determining the geometric information of the current point includes determining the geometric information of the current point based on at least one of the motion compensated point or a point included in a plane of the divided triangle.
8. The method according to claim 6, wherein: The dividing into the at least one triangle comprises: obtaining a plurality of vertices based on a plurality of motion compensated points adjacent to an edge of the current prediction unit; and The current prediction unit is divided into the at least one triangle by connecting the plurality of vertices.
9. The method according to claim 1, wherein: Divide the current prediction unit into regions having a size of 1x1x1 based on a predefined tree structure, and The current point corresponds to the center position of an area having a size of 1x1x1.
10. The method according to claim 9, wherein: The predefined tree structure includes at least one of a binary tree, a quadtree, or an octree.
11. An apparatus for compressing a point cloud, comprising: a processor configured to control the device; as well as a memory coupled to the processor and configured to store data, Wherein, the processor is configured to: Perform global motion compensation based on previous frames of the current frame, Divide the current frame into a plurality of prediction units, determining a point of motion compensation within the current prediction unit by performing local motion compensation on the current prediction unit, and Geometric information of a current point within the current prediction unit is determined based on the motion compensated point.
