Encoding device, decoding device, storage medium, and computer program

Abstract

Provided is an encoding device with which it is possible to encode point cloud data without reducing efficiency even when a floating object or a falling object exists in a space.　The encoding device for encoding point cloud data is characterized by having: an acquisition means for acquiring first information on a floating object or a falling object in a space; and an encoding means for encoding the difference between the predicted position of a target point predicted on the basis of the first information and the position of the point to be encoded.

Description

Encoding device, decoding device, storage medium, and computer program

[0001] The present invention relates to an encoding device, a decoding device, a storage medium, a computer program, and the like.

[0002] Point cloud data generated by LiDAR (Light Detection and Ranging) and the like is being utilized for driving assistance for automobiles, autonomous mobility, monitoring and analysis of traffic environments, etc. LiDAR is primarily used to measure the distance to an object.

[0003] When measuring the distance to an object using LiDAR during rainfall, for example, if it is raining, the distance to the raindrops and the object will be measured, and the ratio of the distance between the raindrops and the object will vary depending on the size of the raindrops and the amount of rainfall, and the range of distances at which the distance to the raindrops can be measured is limited to short distances.

[0004] To accurately detect objects, an algorithm is required to remove point cloud data corresponding to raindrops. Furthermore, if raindrop characteristics are to be utilized, the point cloud including raindrops must be stored and transmitted / received. However, point cloud data generally contains a large number of points and associated data for each point, and therefore, transmitting and receiving the data and storing it on a storage medium can take a long time.

[0005] As a technique for addressing such issues, for example, Non-Patent Document 1 describes a technique for compressing point clouds by creating prediction information using information on points that have been previously coded, and then coding the difference between the prediction information and the information on the points to be coded.

[0006] G-PCC codec description v2, ISO / IEC MPEG (JTC 1 / SC 29 / WG11), N18189

[0007] However, in the above conventional example, when an object is present, the difference in distance measurement positions between adjacent scanning points with adjacent laser scanning intervals is small, and based on the assumption that there is spatial correlation between positions, the position of an input point is predicted based on the positions of points input in the past.Then, the difference between the position of the input point and the predicted position is encoded.

[0008] Therefore, for example, when light emitted from the LiDAR is reflected by raindrops and detected by the LiDAR during rainfall, there is a high possibility that the input points and past points used for prediction will contain points whose positions are significantly different from other points. This poses a problem in that the error in the prediction will become large and the efficiency of encoding will decrease.

[0009] Therefore, one of the objects of the present invention is to provide an encoding device that can encode point cloud data without reducing efficiency even when floating or falling objects exist in space.

[0010] In order to solve the above-mentioned problems, an encoding device according to one aspect of the present invention is an encoding device for encoding point cloud data, characterized in that it has: an acquisition means for acquiring first information relating to floating or falling objects in space; and an encoding means for encoding the difference between a predicted position of a target point predicted based on the first information and the position of the target point to be encoded.

[0011] According to the present invention, it is possible to provide an encoding device that can encode point cloud data without reducing efficiency even when floating or falling objects exist in space.

[0012] 9A is a functional block diagram showing an example of a process for encoding a G-PCC according to the first embodiment of the present invention. FIG. 9B is a functional block diagram illustrating an example of a process for predictive encoding according to the first embodiment. FIG. 9C is a diagram illustrating an example of a process for encoding the difference between the result of predicting the position of a point by predictive encoding and the position of the point. FIG. 9D is a diagram illustrating an example of a tree corresponding to point cloud data generated by a rotational LiDAR. FIG. 9E is a functional block diagram illustrating an example of a process for decoding a G-PCC according to the first embodiment. FIG. 9F is a functional block diagram illustrating an example of a process for predictive decoding according to the first embodiment. FIG. 9G is a functional block diagram illustrating an example of a configuration of an encoding device according to the first embodiment. FIG. 9H is a diagram illustrating an example of a process for acquiring nodes outside a raindrop range. FIG. 9H is a flowchart illustrating an example of a process for acquiring nodes outside a raindrop range by a second position prediction unit 706 according to the first embodiment. FIG. 9H is a flowchart illustrating an example of a process subsequent to FIG. 9A. FIG. 9H is a functional block diagram illustrating an example of a configuration of a decoding device according to the first embodiment.

[0013] First Embodiment FIG. 1 is a functional block diagram showing an example of a G-PCC encoding process according to a first embodiment of the present invention. The G-PCC (Geometry-based Point Cloud Compression) encoding process will be described with reference to FIG. 1.

[0014] When point cloud data is input, it is first decomposed into position information 105 and attribute information 106 by a decomposition module 101. The position information 105 is converted into a position information bit stream 107 by a position information encoding module 102.

[0015] The attribute information 106, together with the position information 105, is converted into an attribute information bit stream 108 by the attribute information encoding module 103. Then, the position information bit stream 107 and the attribute information bit stream 108 are input to the combining module 104, which combines the encoded data and outputs it.

[0016] Predictive Geometry Coding is used as the position information encoding method in the position information encoding module 102 of the G-PCC. Predictive coding is a method in which the position of an input point is predicted based on position information of multiple points input in the past, and the difference between the predicted position and the position information of the input point is encoded.

[0017] FIG. 2 is a functional block diagram for explaining an example of the predictive coding process according to the first embodiment, and shows details of the predictive coding process in the position information coding module 102 of FIG.

[0018] First, information for one point is extracted from the position information of the point cloud to obtain the position information of the point. This position information is three-dimensional coordinate information. When the position information of the point is input, it is added as a node of a tree (prediction tree) in the tree management module 201.

[0019] A tree is made up of multiple nodes connected together. Data indicating information about individual points is registered in each node. For example, position data indicating the position of the point and attribute data indicating information other than the point's position are registered in each node. Each node also holds an identifier that enables it to uniquely identify its ancestor nodes, such as its parent.

[0020] The identifier represents the existence of a branch between a node and its parent node. Note that the tree can be arbitrarily created or updated based on the position of each point, and typically branches are established between nearby nodes.

[0021] After managing the tree, the position prediction module 202 performs position prediction using the tree and calculates a difference value with respect to the predicted position. A prediction mode number is assigned in advance to the prediction method for position prediction in order to uniquely identify the prediction method.

[0022] The difference value, prediction mode, and parent node identifier are input to an arithmetic coding module 203. In the arithmetic coding module 203, the difference value, prediction mode, and parent node identifier are arithmetically coded and output as a point position bitstream.

[0023] FIG. 3 is a diagram illustrating an example of processing for encoding the difference between the result of predicting the position of a point by predictive encoding and the position of the point.

[0024] Each black dot in Figure 3 represents a node in a space defined by a left-handed three-dimensional Cartesian coordinate system having an X-axis, a Y-axis, and a Z-axis. Node n0 corresponds to a point (target encoding point) whose position is to be predicted. Node n1 is the parent node of node n0. Node n2 is the parent node of node n1.

[0025] Node n3 is the parent node of node n2. Here, the position of an arbitrary node x is represented as Pos(x), the predicted vector is represented as Pred(x), and the position of node x predicted by predictive coding is represented as PredPos(x).

[0026] In this case, prediction method 1 for predicting node (n0) can be expressed by, for example, equations (1) and (2). Also, prediction method 2 for predicting node (n0) can be expressed by, for example, equations (3) and (4).

[0027] Pred(n0)=Pos(n1)-Pos(n2)...Formula (1) PredPos(n0)=Pos(n1)+Pred(n0)...Formula (2) Pred(n0)=Pos(n1)-Pos(n3)...Formula (3) PredPos(n0)=Pos(n2)+Pred(n0)...Formula (4)

[0028] Each prediction method is assigned a number in advance to identify the prediction method, and this number is referred to as a prediction mode number. For example, prediction method 1 is assigned a number 1, prediction method 2 is assigned a number 2, and so on.

[0029] The difference Delta(n0) between the position Pos(n0) of node n0 and the position PredPos(n0) of node n0 predicted by predictive coding is calculated by the following equation (5): Delta(n0)=Pos(n0)-PredPos(n0) (5)

[0030] In general, prediction method 1 and prediction method 2 are used to calculate the difference Delta(n0) using each method, and the prediction method that produces the smallest difference is adopted.

[0031] Next, arithmetic coding is applied to the difference Delta(n0), the prediction mode number, and the identifier of node n1, thereby outputting coded data indicating the position of node n0.

[0032] In predictive coding, the smaller the difference Delta(n0), the higher the coding efficiency, and the larger the difference Delta(n0), the lower the coding efficiency. To improve coding efficiency, it is necessary to improve the accuracy of the position PredPos(n0) of node x predicted by predictive coding and reduce the difference Delta(n0).

[0033] Next, Figure 4 is a diagram showing an example of a tree corresponding to point cloud data generated by a rotational LiDAR, and with reference to Figure 4, we will explain the case where predictive coding is applied to point cloud data acquired by a rotational LiDAR.

[0034] 4 is a diagram showing an example of a tree corresponding to point cloud data generated by a rotational LiDAR, in which each point constituting the point cloud data is a measurement point (measurement data) measured by the rotational LiDAR.

[0035] The rotational LiDAR in this embodiment is a LiDAR configured such that, for example, N lasers are arranged at equal intervals along the rotation axis of a rotating body, and light from the N lasers can be simultaneously emitted parallel to each other in a direction perpendicular to the rotation axis of the rotating body. The cross section of the cylinder perpendicular to the rotation axis is circular, and the cross section including the rotation axis and parallel to the rotation axis is rectangular.

[0036] Furthermore, by rotating the rotor, it is possible to simultaneously irradiate light from N lasers in the same azimuth angle direction at each rotation angle (azimuth angle) of the rotor. Therefore, by rotating the rotor, it is possible to simultaneously perform parallel scanning of N lines at equal intervals in the direction of the rotation axis. Note that this embodiment also includes the case where N=1.

[0037] 4 shows a space defined by a three-dimensional Cartesian coordinate system, which is defined by an axis indicating the azimuth angle, an axis indicating the laser ID unique to each laser mounted on the rotary LiDAR, and an axis indicating the distance from each laser. Note that in the rotary LiDAR of this embodiment, the angle at which each laser emits laser light relative to the rotation axis, i.e., the elevation angle, does not change.

[0038] Therefore, by using the laser ID instead of the elevation angle, the amount of data can be reduced. Note that, as mentioned above, N in Fig. 4 represents the number of laser light sources, and is, for example, "16" or "32." The laser ID is data assigned to each laser.

[0039] For each point acquired using each laser, a node is added so that the parent point is a point acquired earlier using the same laser, forming a tree, as shown in Figure 3. The difference Delta is calculated using the above-mentioned equations (1) to (5), and arithmetic coding is applied to the difference Delta(n0), the prediction mode number, and the identifier of node n1, thereby outputting coded data indicating the position of node n0.

[0040] Next, the process of decoding encoded information will be described. Fig. 5 is a functional block diagram showing an example of the decoding process of G-PCC according to the first embodiment. Input encoded data is decomposed by a decomposition module 501, and a position bitstream 505 and an attribute bitstream 506 are output. The position bitstream 505 is decoded into position information by a position information decoding module 502.

[0041] The attribute bit stream 506 is input to an attribute information decoding module 503 together with position information 507, and is decoded into attribute information 508. The position information 507 and the attribute information 508 are input to a synthesis module 504, and are output as synthesized point cloud data.

[0042] FIG. 6 is a functional block diagram for explaining an example of the predictive decoding process according to the first embodiment, and shows an example of the configuration of the position information decoding module 502 shown in FIG.

[0043] Information for one point is extracted from the position bitstream and input to the arithmetic decoding module 601, which restores the parent node identifier, the difference, and the prediction mode number.

[0044] The tree management module 602 determines the parent node n1 using the parent node identifier, and adds the node n0 as a child of n1. Next, the position prediction module 603 calculates the position information of the point from the difference using a prediction method linked to the prediction mode number.

[0045] If the predicted position is represented by PredPos(n0), the predicted position difference is represented by Delta(n0), and the point position information is represented by Pos(n0), PredPos(n0) is calculated by prediction method 1 or prediction method 2 based on the prediction mode number.

[0046] Furthermore, the position of the node n0 is decoded by the following equation (6), and the decoded position is registered in the node n0. Pos(n0)=PredPos(n0)+Delta(n0) (6)

[0047] 1 to 6, the positions of points included in input point cloud data are predicted based on the positions of multiple points previously input. Therefore, when points with significantly different positions (for example, airborne objects such as raindrops or snowflakes, or airborne obstacles such as falling objects) exist in Pos(n0), Pos(n1), Pos(n2), and Pos(n3), the prediction error becomes large and the coding efficiency decreases.

[0048] 7 is a functional block diagram showing an example of the configuration of an encoding device according to the first embodiment. The encoding device 700 for encoding point cloud data can encode the positions of input encoding target points. The encoding device 700 shows an example of the configuration of the position information encoding module 102 in FIG. 1.

[0049] Note that some of the functional blocks shown in Figure 7 are realized by having a CPU or the like serving as a computer (not shown) included in the encoding system execute a computer program stored in a memory (not shown) serving as a storage medium.

[0050] However, some or all of these functions may be implemented by hardware, such as a dedicated circuit (ASIC) or a processor (reconfigurable processor, DSP).

[0051] Furthermore, the functional blocks shown in Fig. 7 do not have to be housed in the same housing, but may be configured as separate devices connected to each other via signal paths. The above explanation regarding Fig. 7 also applies to Figs. 10, 11, and 12.

[0052] The encoding device 700 includes a tree management unit 701, a raindrop range memory unit 702, an average distance calculation unit 703, a raindrop prediction memory unit 704, a first position prediction unit 705, a second position prediction unit 706, a third position prediction unit 707, a prediction mode determination unit 708, and an arithmetic encoding unit 709.

[0053] Although raindrops will be used as an example below, the target floating or falling objects are not limited to raindrops. In other words, the floating or falling objects can be similarly applied to other aerial obstacles that exist in the air and cause noise, and include at least one of rain, snow, sleet, hail, and sleet. The target falling objects are not objects that have already fallen on the road, etc., but objects that are falling (objects in the air).

[0054] When position information of the encoding target point is input, it is added as node n0 of a tree (prediction tree) in a tree management unit 701. Here, the tree management unit 701 functions as a point cloud management means that acquires position information indicating the position of each point of the point cloud data.

[0055] The first position prediction unit 705 performs raindrop prediction coding to determine the difference between the predicted raindrop position obtained from the raindrop prediction memory unit 704 and the position of node n0, and outputs the difference and a prediction mode number indicating the prediction mode of the raindrop prediction coding method.

[0056] Here, the prediction mode of the raindrop predictive coding method is called the first prediction mode. The first position prediction unit 705 functions as a first position prediction unit that predicts the position of the encoding target point based on the first prediction model when the position of the encoding target point is within a predetermined range.

[0057] The raindrop range storage unit 702 is set in advance as a raindrop range (e.g., 5 to 12 m) based on a raindrop model previously learned by machine learning using AI. The average distance calculation unit 703 acquires the raindrop range from the raindrop range storage unit 702. If the input position of node n0 is within the raindrop range, the average of the predicted raindrop position acquired from the raindrop prediction storage unit 704 and the position of node n0 is calculated and output to the raindrop prediction storage unit 704.

[0058] The raindrop prediction storage unit 704 records the input average position as a predicted raindrop position. Here, the predicted raindrop position includes the predicted position of floating or falling objects. In this embodiment, the raindrop range storage unit 702, the average distance calculation unit 703, and the raindrop prediction storage unit 704 function as an acquisition unit that acquires first information regarding floating or falling objects in space.

[0059] Here, the first information includes information on the range of raindrops and the predicted positions of raindrops, i.e., information on the distance or predicted positions of floating or falling objects. In this embodiment, the first information is created or updated based on past observation results of floating or falling objects, but may also be a constant value set by, for example, simulation.

[0060] The second position prediction unit 706 acquires the raindrop range from the raindrop range storage unit 702, calculates a predicted value using the parent nodes of the target point node n0 that are outside the raindrop range, and outputs the difference value from the position of node n0, the prediction mode number, and the identifier of node n1.

[0061] Here, the second position prediction unit 706 functions as a second position prediction means for predicting the position of the encoding target point based on the positions of points excluding points that are within a predetermined range. In this embodiment, the predetermined range is calculated based on past observation results of floating objects or falling objects.

[0062] 8 is a diagram showing an example of a process for acquiring nodes outside the raindrop range. In FIG. 8, n0 is the node of the encoding target point. n1 is the parent node of n0 and is a point within the raindrop range. n2 is the parent node of n1 and is a point outside the raindrop range.

[0063] n3 is the parent node of n2 and is a point within the raindrop range. n4 is the parent node of n3 and is a point within the raindrop range. n5 is the parent node of n4 and is a point outside the raindrop range. n6 is the parent node of n5 and is a point outside the raindrop range.

[0064] As described above, a tree is formed by adding nodes whose parent is a point previously obtained using the same laser. If the node between its parent node is represented as one layer, the number of layers between node n0 and node n2 is two, and the number of layers between node n2 and node n5 is three.

[0065] To obtain nodes outside the raindrop range, the system checks whether the parent node n1 of the target node n0 is outside the raindrop range. If it is not outside the range, that is, if it is within the raindrop range, the system checks the parent node of n1 in the same way. In this way, the system goes back one layer at a time from a given node to its parent node until it finds a node outside the range.

[0066] If there is a node outside the range, that node is saved, and in order to normalize the vectors between nodes outside the raindrop range when calculating the predicted value, the number of layers between the node where the search started and the child node of the node outside the range is saved. After saving, the search starts again. At this time, the node where the search starts will be the out-of-range node found one time before.

[0067] When the search starts from n0, the parent node n1 of n0 is not outside the raindrop range, i.e., it is within the raindrop range, so the same judgment is made for the parent node n2 of n1. Since n2 is outside the raindrop range, the value of n2 and the number of layers between the node n0 where the search started and the child node of n2, 1, are saved.

[0068] Next, the search starts from n2. As described above, in order to perform prediction method 1 and prediction method 2 to calculate the predicted value, three parent nodes of the target point node n0 are required, so the processing ends when the number of saved nodes and the number of layers reach 3. Details of the process of acquiring nodes outside the raindrop range will be described later.

[0069] Here, the number of layers between the node where the search started and the child node of the node outside the raindrop range is i, j, and k, respectively. Also, the nodes outside the saved raindrop range are ni+1, ni+j+2, and ni+j+k+3.

[0070] In this case, prediction method 3 for predicting node n0 is expressed by, for example, equations (7) and (8), and prediction method 4 is expressed by, for example, equations (9) and (10).

[0071] Pred(n0) = (Pos(ni+1)-Pos(ni+j+2)) / (j+1)...Equation (7) PredPos(n0) =Pos(ni+1)+(i+1)×Pred(n0)...Equation (8) Pred(n0) =(Pos(ni+1)-Pos(ni+j+k+3)) / (j+k+2)...Formula (9) PredPos(n0) =Pos(ni+nj+2)+(i+j+2)×Pred(n0)...Formula (10)

[0072] The difference Delta(n0) between the position Pos(n0) of node n0 and the position PredPos(n0) of node n0 predicted by predictive coding is calculated by the above-mentioned equation (5).

[0073] The difference Delta(n0) is calculated for each of prediction methods 3 and 4, and the difference Delta(n0) of the prediction method with the smallest difference, the prediction mode number, and the identifier of node n1 are output.

[0074] The third position prediction unit 707 calculates the difference Delta(n0) for each of the prediction methods 3 and 4 using the above-mentioned prediction method 3 and prediction method 4 and equation (5), and outputs Delta(n0) of the prediction method with the smallest difference, the prediction mode number, and the identifier of node n1. Here, the third position prediction unit 707 functions as a third position prediction means that predicts the position of the encoding target point without using the first information.

[0075] The prediction mode determination unit 708 compares the differences Delta(n0) input from the third position prediction unit 707, the second position prediction unit 706, and the first position prediction unit 705, and outputs the smallest Delta(n0) and its prediction mode.

[0076] That is, encoding is performed using the smallest difference among the difference based on the prediction result of the first position prediction means, the difference based on the prediction result of the second position prediction means, and the difference based on the prediction result of the third position prediction means. Note that encoding may also be performed using the smaller of the difference based on the prediction result of the first position prediction means and the difference based on the prediction result of the second position prediction means.

[0077] The arithmetic coding unit 709 applies arithmetic coding to the input difference Delta(n0), prediction mode number, and identifier of node n1, and outputs a position bitstream indicating the position of node n0.

[0078] In addition, the first position prediction unit 705, the second position prediction unit 706, the third position prediction unit 707, the prediction mode determination unit 708 and the arithmetic coding unit 709 function as coding means that encode the difference between the predicted position of the target point predicted based on the first information and the position of the target point to be coded.

[0079] 9A is a flowchart showing an example of a process for acquiring nodes outside the raindrop range by the second position prediction unit 706 according to the first embodiment, and FIG. 9B is a flowchart showing a continuation of the process shown in FIG. 9A. The CPU as a computer in the encoding device executes a computer program stored in memory. The operation of each step in the flowcharts of FIGS. 9A and 9B is controlled so as to be performed sequentially.

[0080] When the raindrop distance measurement range is input from the raindrop range storage unit 702 and the node n0 of the encoding target point is input from the tree management unit 701, the target point node n0 is set as the reference node ni in step S901.

[0081] In step S902, the inter-node count i is set to 0. In step S903, the parent node ni+1 of the reference node ni is acquired. In step S904, it is determined whether the parent node ni+1 is outside the raindrop range, i.e., outside the raindrop distance range.

[0082] If it is outside the raindrop range, the process proceeds to step S905, where node ni+1 and inter-node count i are saved. If it is within the raindrop range, the process proceeds to step S906, where 1 is added to node count i, and the process proceeds to step S903.

[0083] In step S907, node ni+1 is set as the reference node nj. In step S908, the inter-node count j is set to 0. In step S909, the parent node nj+1 of the reference node nj is acquired.

[0084] In step S910, it is determined whether parent node nj+1 is outside the raindrop range. If it is outside the raindrop range, proceed to step S911, where node nj+1 and node count j are saved. If it is within the raindrop range, proceed to step S912, where 1 is added to node count j, and proceed to step S909.

[0085] In step S913, node nj+1 is set as the reference node nk. In step S914, the inter-node count k is set to 0. In step S915, the parent node nk+1 of the reference node nk is obtained.

[0086] In step S916, it is determined whether the parent node nk+1 is outside the raindrop range. If it is outside the raindrop range, the process proceeds to step S917, where the node nk+1 and the inter-node count k are saved. If it is within the raindrop range, the process proceeds to step S918, where 1 is added to the inter-node count k, and the process proceeds to step S915.

[0087] By performing the processes of FIGS. 9A and 9B as described above, the count values ​​i, j, and k between the nodes ni+1, nj+1, and nk+1 outside the raindrop range are obtained.

[0088] In the first embodiment, the raindrop prediction storage unit stores the average value calculated by the average distance calculation unit, but the value stored in the raindrop prediction storage unit may be a constant value preset based on a raindrop model. Also, in the first embodiment, an attribute indicating that the node n0 is a raindrop may be assigned to the attribute of the encoding target point n0 using the raindrop prediction method.

[0089] As described above, in the encoding device 700 of the first embodiment, in addition to the method of predicting the position of node n0 using the predicted vector Pred(n0), the position of node n0 is calculated using the raindrop range and raindrop prediction. As a result, even if the position of node n0 of the target point whose position is to be predicted is far from the position of the parent node, the difference Delta(n) can be reduced, and encoding efficiency can be improved.

[0090] Even if there are points located far away in the parent node, the raindrop range can be used to select the parent node, thereby improving the prediction accuracy of the predicted vector Pred (n0) and reducing the difference Delta (n). This can improve coding efficiency.

[0091] A decoding device corresponding to the first embodiment of the present invention will be described below. Fig. 10 is a functional block diagram showing an example of the configuration of a decoding device corresponding to the first embodiment. A point cloud decoding device 1000 inputs and decodes a position bitstream, which is input encoded point cloud position data.

[0092] The point cloud decoding device 1000 includes an arithmetic decoding unit 1001, a tree management unit 1002, a switch 1003, a raindrop range storage unit 1004, an average distance calculation unit 1005, and a raindrop prediction storage unit 1006. The point cloud decoding device 1000 further includes a first position prediction unit 1007, a second position prediction unit 1008, and a third position prediction unit 1009.

[0093] When information for one point is extracted from the position bitstream and input to the arithmetic decoding unit 1001, the parent node identifier, the position difference value, and the prediction mode number are restored in the arithmetic decoding unit 1001. The tree management unit 1002 determines the parent node n1 using the parent node identifier, and adds the node n0 as a child of n1.

[0094] Switch 1003 is a selection unit for selecting a prediction method associated with the prediction mode number. If the prediction mode number is a number associated with the raindrop prediction mode, the third position prediction unit is selected. If the prediction mode number is a number associated with prediction method 3 or prediction method 4, the second position prediction unit is selected. If the prediction mode number is a number associated with prediction method 1 or prediction method 2, the first position prediction unit is selected.

[0095] The raindrop distance measurement range is stored in advance in the raindrop range storage unit 1004. The first position prediction unit 1007 acquires the difference of the decoding target node n0 and the raindrop prediction value recorded in the raindrop prediction storage unit, and decodes the position information of the decoding target node n0 by adding the difference to the raindrop prediction value.

[0096] The average distance calculation unit 1005 compares the position information of the decoded point with the raindrop range acquired from the raindrop range storage unit, and if the position of the decoded point is within the range of the raindrop, calculates the average of the raindrop prediction value acquired from the raindrop prediction storage unit and the position of the decoded point, and outputs the calculation result to the raindrop prediction storage unit.

[0097] The second position prediction unit 1008 acquires the raindrop distance measurement range from the raindrop range storage unit, and acquires the nodes outside the raindrop range and the inter-node count. The processing flow in the second position prediction unit 1008 may be the same as the flow shown in Fig. 9, and after calculating a predicted value using the above-mentioned prediction method 3 or prediction method 4 according to the input prediction mode number, the position information of the decoding target node n0 is decoded using the above-mentioned equation (6).

[0098] The third position prediction unit 1009 calculates a predicted value using the above-mentioned prediction method 1 or prediction method 2 in accordance with the prediction mode number, and then decodes the position information of the node n0 to be decoded using the above-mentioned equation (6). The decoded position information of the node n0 is output as position information, and is also output to the tree management unit and registered as the position information of the node n0.

[0099] <Embodiment 2> Hereinafter, embodiment 2 of the present invention will be described. Fig. 11 is a functional block diagram showing an example of the configuration of an encoding device according to embodiment 2. An encoding device 1100 encodes the position of an input encoding target point.

[0100] The encoding device 1100 includes a tree management unit 1101, a raindrop range storage unit 1102, a ray number determination unit 1103, a position frequency calculation unit 1104, a raindrop prediction storage unit 1105, a first position prediction unit 1106, a second position prediction unit 1107, and a third position prediction unit 1108. The encoding device 1100 also includes a prediction mode determination unit 1109 and an arithmetic coding unit 1110.

[0101] When position information of the encoding target point is input, it is added as node n0 of a tree (Prediction Tree) in the tree management unit 1101. The first position prediction unit 1106 performs raindrop prediction encoding to obtain the difference between the predicted raindrop position acquired from the raindrop prediction storage unit 1105 and the position of node n0, and outputs the difference and the prediction mode number of the raindrop prediction encoding method.

[0102] The ray number determination unit 1103 stores in advance the ID of a laser that is irradiated into the sky without hitting any object. It also stores the azimuth angle and distance range of that laser, or the XYZ range in three-dimensional space. The ray number determination unit 1103 determines whether the position of node n0 is within that range, and outputs the determination result and the position of node n0.

[0103] The location frequency calculation unit 1104 obtains the determination result, and if the location of node n0 is included in the range, updates the location frequency distribution. The location frequency calculation unit 1104 obtains a peak position from the updated location frequency distribution and outputs it to the raindrop prediction storage unit 1105 as a raindrop prediction position.

[0104] Furthermore, the position frequency calculation unit 1104 calculates the range of the position frequency distribution (for example, standard deviation σ × n) and outputs it to the raindrop range storage unit 1102. In this way, in the second embodiment, the predicted raindrop positions and the raindrop range as the first information are acquired from a space where no encoding target points exist.

[0105] The second position prediction unit 1107 acquires the raindrop range from the raindrop range storage unit 1102, and calculates the difference Delta(n0) in each of the above prediction methods 3 and 4 and equation (5).

[0106] The calculated difference Delta(n0) of prediction method 3 and the difference Delta(n0) of prediction method 4 are compared, and the prediction mode number of the prediction method with the smaller value and the identifiers of the differences Delta(n0) and n1 are output.

[0107] The third position prediction unit 1108 calculates the difference Delta(n0) for each method using the above-mentioned prediction method 1 and prediction method 2 and equation (5), and outputs Delta(n0) of the prediction method with the smallest difference, as well as the prediction mode number and the identifier of node n1.

[0108] The prediction mode determination unit 1109 compares the differences Delta(n0) input from the first position prediction unit, the second position prediction unit, and the third position prediction unit, and outputs the smallest Delta(n0) and its prediction mode.

[0109] The arithmetic coding unit 1110 applies arithmetic coding to the input difference Delta(n0), prediction mode number, and node n1 identifier to output a position bitstream indicating the position of node n0. In the second embodiment, an attribute indicating that the node n0 is a raindrop may be added to the attribute of the encoding target point node n0 using the raindrop prediction method.

[0110] The above describes the encoding device according to embodiment 2. In the encoding device 1100, in addition to the technique of predicting the position of node n0 using the predicted vector Pred(n0), the position of node n0 is calculated using the raindrop range and raindrop prediction.

[0111] As a result, even if the position of the node n0 of the target point whose position is to be predicted is distant from the position of the parent node, the difference Delta (n) can be reduced and the encoding efficiency can be improved. Also, even if there is a point whose position is distant in the parent node, by selecting the parent node using the raindrop range, the prediction accuracy of the predicted vector Pred (n0) can be improved, the difference Delta (n) can be reduced, and the encoding efficiency can be improved.

[0112] A composite apparatus corresponding to the second embodiment of the present invention will be described below. Fig. 12 is a functional block diagram showing an example of the configuration of a decoding apparatus corresponding to the second embodiment. A point cloud decoding apparatus 1200 decodes a position bitstream, which is input encoded point cloud position data.

[0113] 12 , the point cloud decoding device 1200 includes an arithmetic decoding unit 1201, a tree management unit 1202, a switch 1203, a raindrop range storage unit 1204, a ray number determination unit 1205, a position frequency calculation unit 1206, and a raindrop prediction storage unit 1207. The point cloud decoding device 1200 also includes a first position prediction unit 1208, a second position prediction unit 1209, and a third position prediction unit 1210.

[0114] When information for one point is extracted from the position bitstream and input to the arithmetic decoding unit 1201, the parent node identifier, the difference, and the prediction mode number are restored in the arithmetic decoding unit 1201. The tree management unit 1202 determines the parent node n1 using the parent node identifier, and adds the node n0 as a child of n1.

[0115] Switch 1203 is a selection unit for selecting a prediction method associated with the prediction mode number. If the prediction mode number is a number associated with the raindrop prediction mode, the third position prediction unit is selected. If the prediction mode number is a number associated with prediction method 3 or prediction method 4, the second position prediction unit is selected. If the prediction mode number is a number associated with prediction method 1 or prediction method 2, the first position prediction unit is selected.

[0116] The first position prediction unit 1208 obtains the difference of the node n0 to be decoded and the raindrop prediction value recorded in the raindrop prediction memory unit 1207, and decodes the position information of the node n0 to be decoded by adding the difference to the raindrop prediction value.

[0117] The ray number determination unit 1205 stores in advance the ID of a laser that is irradiated into the sky without hitting any object. The azimuth angle and distance range of the laser, or the XYZ range in three-dimensional space, are also recorded. The ray number determination unit 1205 determines whether the position of the node n0 to be decoded is within the range, and outputs the determination result and the position of node n0.

[0118] The location frequency calculation unit 1206 obtains the determination result and updates the location frequency distribution if the location of node n0 is within the range. The unit 1206 obtains a peak position from the updated location frequency distribution and outputs it to the raindrop prediction storage unit 1207. The unit 1206 then calculates the range of the location frequency distribution and outputs it to the raindrop range storage unit 1204.

[0119] The second position prediction unit 1209 acquires the raindrop range from the raindrop range storage unit and acquires nodes outside the raindrop range. The processing flow in the second position prediction unit 1209 may be the same as the flow shown in Fig. 9, and calculates a predicted value using the above-mentioned prediction method 3 or prediction method 4 according to the prediction mode number, and decodes the position information of the decoding target node n0 using the above-mentioned equation (5).

[0120] The third position prediction unit 1210 calculates a predicted value using the above-mentioned prediction method 1 or prediction method 2 according to the prediction mode number, and then decodes the position information of the node n0 to be decoded using the above-mentioned equation (6). The decoded position information of the node n0 is output as position information, and is also output to the tree management unit and registered as the position information of the node n0.

[0121] As described above, according to the first and second embodiments, even if there are points (such as floating or falling objects in space) among the input points and past points used for prediction that are significantly separated from other points, it is possible to encode and decode point cloud data without reducing compression efficiency.

[0122] Although the present invention has been described in detail above based on preferred embodiments 1 to 4, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Parts of the above-described embodiments 1 to 4 may be combined as appropriate.

[0123] The present invention also includes a case where a software program that realizes the functions of the above-described embodiments is supplied to a system or device having a computer that can execute the program directly from a recording medium or via wired / wireless communication, and the program is executed.

[0124] Therefore, the program code itself that is supplied to and installed on a computer to realize the functional processing of the present invention also realizes the present invention. In other words, the computer program itself for realizing the functional processing of the present invention is also included in the present invention.

[0125] In this case, as long as the program functions are provided, the program may be in any form, such as object code, a program executed by an interpreter, or script data supplied to an OS.

[0126] The recording medium for supplying the program may be, for example, a hard disk, a magnetic recording medium such as a magnetic tape, an optical / magneto-optical storage medium, or a non-volatile semiconductor memory.

[0127] Alternatively, the program may be supplied by storing the computer program forming the present invention in a server on a computer network, and a connected client computer may download the computer program.

[0128] This application claims the benefit of Japanese Patent Application No. 2024-019277, filed February 13, 2024. The contents of the above Japanese patent application are incorporated herein by reference in their entirety.

Claims

1. An encoding device for encoding point cloud data, comprising: an acquisition means for acquiring first information relating to floating or falling objects in space; and an encoding means for encoding the difference between the predicted position of a target point predicted based on the first information and the position of the target point to be encoded.

2. The encoding device according to claim 1, wherein the predicted positions include predicted positions of the floating objects or the falling objects.

3. The encoding device according to claim 1, wherein the floating or falling objects include at least one of rain, snow, sleet, hail, and grit.

4. The encoding device according to claim 1, further comprising a point cloud management means for acquiring position information indicating the position of each point of said point cloud data.

5. The encoding device according to claim 1, characterized in that the encoding means has a first position prediction means for predicting the position of the encoding target point based on a first prediction model when the position of the encoding target point is within a predetermined range.

6. The encoding device according to claim 5, characterized in that the encoding means has a second position prediction means for predicting the position of the encoding target point based on the positions of points excluding points within the specified range.

7. The encoding device according to claim 6, wherein the predetermined range is calculated based on past observation results of the floating object or the falling object.

8. The encoding device according to claim 6, characterized in that the encoding means performs the encoding using the smaller of the difference based on the prediction result of the first position prediction means and the difference based on the prediction result of the second position prediction means.

9. The encoding device according to claim 6, wherein said encoding means comprises third position prediction means for predicting the position of said encoding target point without using said first information.

10. The encoding device described in claim 6, characterized in that the encoding means performs the encoding using the smallest difference among the difference based on the prediction result of the first position prediction means, the difference based on the prediction result of the second position prediction means, and the difference based on the prediction result of the third position prediction means.

11. The encoding device according to claim 1, wherein the first information is created or updated based on past observation results of the floating object or the falling object.

12. The encoding device according to claim 1, wherein the first information includes information regarding the distance or predicted position of the floating object or the falling object.

13. The encoding device according to claim 1, wherein said acquisition means acquires said first information from a space in which said encoding target point does not exist.

14. A decoding device that decodes the point cloud data encoded by the encoding device according to any one of claims 1 to 13.

15. A computer program for controlling each means of the encoding device according to any one of claims 1 to 13 by a computer.

16. A computer program for controlling each means of the decoding device according to claim 14 by a computer.

17. A storage medium storing the computer program according to claim 15 or 16.

Images