Encoder, decoder, encoding method, decoding method, and program

Abstract

To provide an encoder, a decoder, an encoding method, a decoding method, and a program capable of processing with high efficiency and low delay.SOLUTION: An encoder (1000) has: surface information calculation means (wall estimation module 1004) that calculates surface information by using processed point position data that is included in point group data and that indicates the position of a processed point on which encoding processing is completed; and a position prediction module (1005) including direction information prediction means (1005a) that predicts direction information of a target point to be a target of the encoding processing by using the processed point position data, and first distance prediction means (1005b) that predicts the distance to the target point on the basis of the surface information and the direction information.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present invention relates to an encoding device, a decoding device, an encoding method, a decoding method, and a program. [Background technology]

[0002] Conventionally, a method using point cloud data generated by LiDAR (Light Detection and Ranging) or the like has been known to realize the autonomous movement of an automatic guided vehicle (AGV). Generally, point cloud data includes a large number of points and data associated with each point, and therefore, it may take a long time to transmit, receive, or store the data in a storage medium.

[0003] Non-Patent Document 1 discloses a method for encoding a point cloud using G-PCC (Geometry-based Point Cloud Compression). Predictive geometry coding, which is one form of G-PCC, is a method for predicting the position of an input point based on position information of multiple points input in the past, and encoding the difference between the predicted position information and the position information of the input point. Patent Document 1 also discloses a method for fitting a model represented by parameters to a point cloud, and encoding the parameters of the model after fitting and residual data of the fitting. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Special Publication No. 2019-521417 [Non-patent literature]

[0005] [Non-Patent Document 1] ISO / IEC JTC 1 / SC 29 / WG7N 0151, G-PCC codec description v12, July 2021 Summary of the Invention [Problem to be solved by the invention]

[0006] However, in the method disclosed in Non-Patent Document 1, the position of a newly input point is predicted based on the positions of multiple points previously input and the laser angle of the LIDAR, but prediction is not made based on the shape of the feature, which may result in prediction errors and reduced coding efficiency.

[0007] Furthermore, in the method disclosed in Patent Document 1, in order to fit a model to a point cloud, it is necessary to acquire all points necessary for fitting, and processing cannot start until acquisition of points from a lidar, etc. is complete. This causes delays, and the method may not be suitable for real-time applications.

[0008] Therefore, an object of the present invention is to provide an encoding device that is capable of highly efficient and low-delay processing. [Means for solving the problem]

[0009] An encoding device according to one aspect of the present invention includes a surface information calculation means for calculating surface information using processed point position data included in point cloud data, the processed point position data indicating the positions of processed points for which encoding processing has been completed; a direction information prediction means for predicting direction information of a target point that is the subject of the encoding processing using the processed point position data; and a first distance prediction means for predicting a distance to the target point based on the surface information and the direction information.

[0010] Other objects and features of the present invention will be described in the following embodiments. [Effects of the Invention]

[0011] According to the present invention, it is possible to provide an encoding device that is capable of highly efficient and low-delay processing. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 2 is an explanatory diagram of an encoding process according to each embodiment. [Figure 2] 1A to 1C are diagrams illustrating examples of trees corresponding to point cloud data generated by a rotary LIDAR according to each embodiment. [Figure 3] FIG. 1 is a diagram showing an example of point cloud data generated by using a rotary lidar to measure a vertical wall with a laser that is horizontal to the floor. [Figure 4] FIG. 10 is a diagram showing an example of point cloud data generated by using a rotary lidar to measure a vertical wall with a laser that is not horizontal to the floor. [Figure 5] FIG. 1 is a diagram showing an example of point cloud data generated by using a rotary lidar to measure a vertical wall with a laser that is horizontal to the floor. [Figure 6] FIG. 10 is a diagram showing an example of point cloud data generated by using a rotary lidar to measure a vertical wall with a laser that is not horizontal to the floor. [Figure 7] FIG. 10 is an explanatory diagram of the coordinates of a rotary rider according to each embodiment. [Figure 8] 10A and 10B are explanatory diagrams illustrating the relationship between coordinates and points constituting a point group according to each embodiment. [Figure 9] 1 is a database showing the relationship between the laser ID and the elevation angle of the rotary LID according to each embodiment. [Figure 10] FIG. 1 is an explanatory diagram of an encoding device according to a first embodiment. [Figure 11] 3 is a flowchart showing an encoding method according to the first embodiment. [Figure 12] FIG. 10 is an explanatory diagram of a decoding device according to a second embodiment. [Figure 13] 10 is a flowchart showing a decoding method according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0014] <G-PCCについて> First, a process (encoding process) for encoding the difference between the result of predicting the position of a point by predictive coding (predicted position) and the actual position of the point (actual position) will be described with reference to Fig. 1. Predictive Geometry Coding is one of the G-PCC (Geometry-based Point Cloud Compression) methods of the Moving Picture Experts Group (MPEG). G-PCC is disclosed in Non-Patent Document 1.

[0015] Predictive coding is a technique in which the position of an input point is predicted based on the positions of previously input points, and the difference between the input point position and the predicted position is encoded. When predictive coding is applied to point cloud data, a prediction tree is created and managed. The tree is a combination of multiple nodes. Data indicating information about each point is registered in each node. For example, each node is registered with position data indicating the position of the point and attribute data indicating information other than the point position. Each node also has a unique identifier that enables it to be uniquely identified from others, and holds the identifiers of its parent, which is the node located along the branch toward the root, and its ancestors, such as its parent. The tree can be arbitrarily created or updated based on the position of each point, and branches are typically established between nearby nodes.

[0016] FIG. 1 is an explanatory diagram of a process (encoding process) for encoding the difference between the result of predicting the position of a point using predictive coding and the position of the point. FIG. 1 shows nodes in a space defined by a left-handed three-dimensional coordinate system having an X-axis, a Y-axis, and a Z-axis. Node n corresponds to a point whose position is the target of the process of predicting. Node n0 is the parent node of node n. Node n1 is the parent node of node n0. Node n2 is the parent node of node n1. The position of any node x is represented as Pos(x), and the predicted vector is represented as Pred(x).

[0017] When the position of node x predicted by predictive coding is represented as PredPos(x), the following equations (1) to (4) hold. In the predictive coding of each embodiment, the position of node n is predicted using either equations (1) and (2) or equations (3) and (4).

[0018] Pred(n)=Pos(n0)-Pos(n1) …(1) PredPos(n)=Pos(n0)+Pred(n) …(2) Pred(n)=Pos(n0)-Pos(n2) …(3) PredPos(n)=Pos(n1)+Pred(n) …(4) The difference Delta(n) between the position Pos(n) of node n and the position PredPos(n) of node n predicted by predictive coding is calculated by the following equation (5).

[0019] Delta(n)=Pos(n)-PredPos(n) …(5) Arithmetic coding is applied to the difference Delta(n) and the identifier of node n0 to generate a position bitstream indicating the position of node n. The position bitstream is then combined with an attribute bitstream that is generated separately by applying coding to attribute data, and a coded bitstream is output.

[0020] Typically, in predictive coding, the smaller the difference Delta(n), the higher the coding efficiency, and the larger the difference Delta(n), the lower the coding efficiency. To improve coding efficiency, it is necessary to improve the accuracy of the position PredPos(n) of node x predicted by predictive coding and reduce the difference Delta(n).

[0021] In the above description, an example is given in which the nodes exist in a space defined by a left-handed three-dimensional coordinate system, but the coordinate system defining the space is not particularly limited. For example, the space in which the nodes exist may be defined by a right-handed three-dimensional coordinate system.

[0022] Next, the process of decoding the coded bitstream (decoding process) will be described. The coded bitstream is separated into a position bitstream and an attribute bitstream. Arithmetic decoding is applied to the position bitstream to generate a difference Delta(n) and an identifier of the parent node n0 of node n. A node corresponding to the point to be decoded is added to the tree based on the identifier, and the position of the node is predicted using the method described above. The position of the node is then decoded using the following equation (6). After that, the attribute bitstream is decoded.

[0023] Pos(n)=Delta(n)+PredPos(n) …(6) Next, with reference to Fig. 2, a case where predictive coding is applied to point cloud data acquired by a rotary LIDAR will be described. Fig. 2 is a diagram showing an example of a tree corresponding to point cloud data generated by a rotary LIDAR. Here, each point constituting the point cloud data is a measurement point (measurement data) measured by the rotary LIDAR. Fig. 2 shows a space defined by three-dimensional coordinates defined by an axis indicating the azimuth direction, an axis indicating a laser ID unique to each laser mounted on the rotary LIDAR, and an axis indicating the distance from the LIDAR.

[0024] In a rotating LIDAR, the vertical angle at which each laser emits laser light, i.e., the elevation angle, does not change. Therefore, by using the laser ID instead of the elevation angle, the amount of data can be reduced. The natural number N shown in Figure 2 represents the number of lasers. For example, the natural number N is "16" or "32". As shown in Figure 2, points acquired using each laser are added to nodes that have previously been acquired using the same laser, forming a tree.

[0025] The method described with reference to FIG. 1 and the method described with reference to FIG. 2 each predict the position of a point included in input point cloud data based on the positions of multiple previously input points. That is, the predicted position of the input point is determined from the positions of ancestor nodes such as the parent and its ancestor, and prediction is performed using either equations (1) and (2) or equations (3) and (4). Therefore, in these methods, if the position of the input point does not follow equations (1) and (2) or equations (3) and (4), prediction error may increase, potentially reducing coding efficiency. This example will be described below with reference to FIGS. 3 to 6.

[0026] Referring to Figure 3, it will be explained that a difference occurs between the result of predicting the position of a point (predicted position) and the actual position of a point (actual position) using the method described with reference to Figure 1. Figure 3 is a diagram showing the use of a rotary LIDAR in a scene where a flat wall and floor exist. F1 is the horizontal floor (ground), and W1 is a wall perpendicular to the floor F1. S1 is a LIDAR (rotary LIDAR) installed above the floor F1. Points 301 to 309 are points (point cloud) obtained by the LIDAR S1 measuring the wall W1 with an elevation angle of zero, i.e., a laser parallel to the floor F1, and point 305 is a point where not only the elevation angle but also the azimuth angle is zero.

[0027] The lidar S1 performs measurements while changing the azimuth angle using a motor that rotates at a constant speed. Point 301 is the first point measured, point 302 is the next point measured, point 303 is the next point measured, and point 309 is the last point measured. Points 301 to 309 are aligned in a straight line on wall W1, but the distance between them is not constant. From point 301 to point 305, the distance between point 301 and point 302 is wide, the distance between point 302 and point 303 is slightly narrow, the distance between point 303 and point 304 is even narrower, and the distance between point 304 and point 305 is the smallest. Meanwhile, from point 305 to point 309, the distance between point 305 and point 306 is narrow, but the distance between point 306 and point 307 is wide, the distance between point 307 and point 308 is even wider, and the distance between point 308 and point 309 is the largest.

[0028] As described above, the interval between two consecutive points obtained by measuring the wall W1 is not constant. In the method described with reference to FIG. 1, that is, the method of making a prediction using either Equation (1) and Equation (2) or Equation (3) and Equation (4), a prediction error occurs because the method assumes that the interval between the two points referenced for prediction is constant.

[0029] FIG. 4 is a diagram showing the use of a rotary LIDAR in a scene with a flat wall and floor, similar to FIG. 3. In FIG. 4, points 402 to 409 represent points (point cloud) obtained by measuring wall W1 using a laser from LIDAR S1 with a downward elevation angle. The distance between any two adjacent points among points 402 to 409 is not constant, as with points 301 to 309 described with reference to FIG. 3. Points 301 to 309 are aligned on a straight line in space, but points 402 to 409 are not. The method described with reference to FIG. 1, i.e., the method of prediction using either equations (1) and (2) or equations (3) and (4), assumes that the distance between any two adjacent points is constant and that the points are aligned on a straight line, resulting in prediction errors.

[0030] Referring to Figure 5, it will be explained that a difference occurs between the result of predicting the position of a point (predicted position) and the actual position of a point (actual position) using the method described with reference to Figure 2. Figure 5 is a top view of the scene described with reference to Figure 3. 501 is a line segment connecting the origin of rider S1 to point 304, and represents the distance between rider S1 and point 304. 502 is a line segment connecting the origin of rider S1 to point 305, and represents the distance between rider S1 and point 305. 503 is a line segment connecting the origin of rider S1 to point 306, and represents the distance between rider S1 and point 306.

[0031] The distance to point 305, obtained after point 304, is smaller, and the distance to point 306, obtained after that, is larger. The method described with reference to Figure 2, i.e., the method of prediction using either equations (1) and (2) or equations (3) and (4), assumes that the distance from the lidar changes constantly between the points referenced for prediction, resulting in prediction errors. 504 represents the predicted position of point 306, and because the distance to point 305, obtained after point 304, is smaller, the distance to the next point, 306, is predicted to be even smaller.

[0032] FIG. 6 is a top view of the scene described with reference to FIG. 4. Reference numeral 601 denotes a line segment connecting the origin of the rider S1 to point 404, representing the distance between the rider S1 and point 404. Reference numeral 602 denotes a line segment connecting the origin of the rider S1 to point 405, representing the distance between the rider S1 and point 405. Reference numeral 603 denotes a line segment connecting the origin of the rider S1 to point 604, representing the distance between the rider S1 and point 604. Point 604 represents the predicted position of point 406. When the position of point 406 is predicted using the method described with reference to FIG. 2, the predicted position of point 406 becomes point 604, which is closer to the rider S1 than its actual position, as in FIG. 5.

[0033] In the following embodiments, a technique for solving the above problems will be described. Note that in each embodiment, a wall that is perpendicular to the ground is an example, and even a non-planar feature, such as a cylindrical feature such as a pillar, can be applied by regarding it as a locally vertical plane. Furthermore, in each embodiment, a LIDAR is an example, and the present invention can be applied to point cloud acquisition devices other than LIDAR as long as they are devices that can acquire point clouds.

[0034] First Embodiment First, a first embodiment of the present invention will be described. A suitable condition for this embodiment is that in a scene where there is a horizontal floor (ground) and a vertical wall, a rotary LIDAR (point cloud acquisition device) is placed at an angle parallel to the floor. The rotary LIDAR measures the floor and walls to generate a point cloud (point cloud data). In other words, the point cloud is data acquired from a point cloud acquisition device placed on the ground. The generated point cloud is input to an encoding device and encoded. Note that the point cloud acquisition device is not limited to a configuration in which it is placed on the ground. The point cloud acquisition device may be attached to a vehicle, a drone, or the like as long as it is parallel to the ground.

[0035] Coordinates in this embodiment will be described below. Each point constituting the point cloud is acquired by a rotary LIDAR and has position information based on the rotary LIDAR.

[0036] First, the coordinates of the rider S1 will be explained with reference to Figure 7. Figure 7 is an explanatory diagram of the coordinates of the rider S1, showing a space defined by a right-handed three-dimensional coordinate system having an X-axis, a Y-axis, and a Z-axis based on the rider S1. 703 is the Z-axis, which is an axis that points vertically from the center of the laser of the rider S1 into the sky when the rider S1 is installed parallel to the floor. 701 is the X-axis, and 702 is the Y-axis. When the rider S1 is installed at an angle parallel to the floor, the X-axis 701 and the Y-axis 702 are both parallel to the floor.

[0037] Next, the relationship between the points that make up the point cloud and their coordinates will be described with reference to Fig. 8. Fig. 8 is an explanatory diagram of the relationship between the points that make up the point cloud and their coordinates. 801 is a measured point. When the coordinates (x, y, z) represent the position of a point in a right-handed three-dimensional coordinate system having an X axis, a Y axis, and a Z axis, position 803 is the x value on the X axis, position 804 is the y value on the Y axis, and position 805 is the z value on the Z axis.

[0038] When point 801 is lowered perpendicularly onto a plane (XY plane) containing the X-axis and Y-axis, the point of contact is imaginary point 802. Angle 806 between the Y-axis and the line segment connecting origin O and imaginary point 802 is azimuth angle θ. Angle 807 between the line segment connecting origin O and imaginary point 802 and the line segment connecting origin O and point 801 is elevation angle φ. The length of the line segment connecting origin O and point 801 is the distance r between lidar S1 and point 801. When the position of point 801 is expressed by azimuth angle θ, elevation angle φ, and distance r, it is expressed as coordinates (θ, φ, r).

[0039] As mentioned above, the elevation angle φ of each laser in the LIDAR S1 does not change. Therefore, if there is a database that defines the relationship between the laser ID unique to each laser and the elevation angle φ as shown in Fig. 9, the position of point 801 can be expressed using the laser ID instead of the elevation angle φ. For example, when the laser ID is represented as n, the position of point 801 is represented by the coordinates (θ, n, r).

[0040] Next, the encoding device in this embodiment will be described. Point cloud data acquired from a rotating LIDAR is input to the encoding device. The encoding device extracts position data representing positions from the point cloud data. The position data is data consisting of position information of each point included in the point cloud, and the position information of each point is represented by coordinates (θ, n, r) expressed by an azimuth angle θ, a laser ID (n), and a distance r.

[0041] Next, a process (encoding process) for encoding position information by the encoding device 1000 will be described in detail. First, the encoding device 1000 will be described with reference to FIG. 10. FIG. 10 is an explanatory diagram of the encoding device 1000. The encoding device 1000 has a coordinate transformation module 1002, a tree management module 1003, a wall estimation module 1004, a position prediction module 1005, a subtraction module 1006, and an arithmetic coding module 1007. The encoding device 1000 also has a conventional encoding processing module 1008 and a comparison module 1009. The encoding device 1000 also has control means 1021 such as a CPU that controls each module, and storage means (memory) 1022 that stores programs, data, etc. required for executing each module.

[0042] The wall estimation module 1004 functions as a surface information calculation means that calculates surface information using processed point position data that indicates the positions of processed points that have been encoded and are included in the point cloud data.

[0043] The position prediction module 1005 functions as a direction information prediction means 1005a that predicts the direction information of the target point that is the subject of the encoding process using processed point position data, and a first distance prediction means 1005b that predicts the distance to the target point based on the surface information and direction information.

[0044] The conventional encoding processing module 1008 performs encoding processing on the position information using a conventional method. That is, the conventional encoding processing module 1008 functions as a second distance prediction means that predicts the position of a target point based on position information of a plurality of point groups that have been input in the past, and predicts the distance to the target point based on the difference between the predicted position and the actual position of the target point.

[0045] The comparison module 1009 functions as a switching means for switching between the first distance prediction means 1005b and the conventional encoding processing module (second distance prediction means) 1008. For example, the comparison module 1009 switches between the first distance prediction means and the second distance prediction means so as to use the data based on the distance predicted by the first distance prediction means or the data based on the distance predicted by the second distance prediction means, whichever has the smaller amount of data.

[0046] The coordinate conversion module 1002 receives coordinates (θ, n, r), which are point position information 1001, as input, and converts them into coordinates (θ, φ, r) defined by the azimuth angle θ, the elevation angle φ, and the distance r, and coordinates (x, y, z) defined by the X-axis, Y-axis, and Z-axis. To generate coordinates (θ, φ, r) from the coordinates (θ, n, r), the coordinate conversion module 1002 uses a database D that defines the relationship between the laser ID (n) and the elevation angle φ. Next, the coordinate conversion module 1002 calculates coordinates (x, y, z) from the coordinates (θ, φ, r) using the following equations (7), (8), and (9), and outputs the coordinates to the tree management module 1003.

[0047] x = r × cos(φ) × sin(θ) …(7) y = r × cos(φ) × cos(θ) …(8) z = r × sin(φ) …(9) The tree management module 1003 adds the input coordinates (θ, φ, r) and coordinates (x, y, z) to the tree T1 as a new node. Unlike the tree described with reference to FIG. 2, the tree T1 has two types of coordinates represented by coordinates (x, y, z) and coordinates (θ, φ, r). Next, the tree management module 1003 extracts from the tree T1 the coordinates (x1, y1, z1) and coordinates (θ1, φ, r1) of the parent node of the added node and the coordinates (x2, y2, z2) and coordinates (θ2, φ, r2) of the parent node. The tree management module 1003 then outputs x1, y1, x2, and y2 to the wall estimation module 1004 and outputs θ1, θ2, and φ to the position prediction module 1005. The tree management module 1003 also outputs an identifier indicating the parent node and an identifier indicating the parent node to the arithmetic coding module 1007.

[0048] The wall estimation module 1004 calculates a plane equation expressed by the following formulas (10) to (14) that represents a plane perpendicular to the floor from the input x1, y1, x2, and y2, and outputs the values ​​of parameters a, b, and d to the position prediction module 1005. That is, the wall estimation module 1004 calculates surface information using processed point position data that indicates the positions of processed points that have been encoded and are included in the point cloud data. Note that c is always 0 for a plane perpendicular to the floor, so it is not necessary to output c.

[0049] a×x+b×y+c×z+d=0 …(10) a = y2 - y1 …(11) b = x1 - x2 …(12) c=0 …(13) d=x1×(y1-y2)-y1×(x1-x2) …(14) The position prediction module 1005 receives θ1, θ2, φ, a, b, and d as input, and predicts coordinates (θp, n, rp) that represent the predicted position of the point. That is, the position prediction module 1005 predicts the direction information of the target point that is the target of the encoding process using the processed point position data. First, the position prediction module 1005 calculates the predicted position (predicted azimuth angle) θp using the following equation (15). Equation (15) is a version of equations (1) and (2) limited to the azimuth angle.

[0050] θp=2×θ1-θ2 …(15) Next, the position prediction module 1005 predicts the predicted distance rp from the origin to the intersection of the line indicating the laser direction represented by θp and φ and the plane represented by a, b, and d using the following equation (16): That is, the position prediction module 1005 predicts the distance to the target point based on the surface information and direction information.

[0051] rp=-d÷(a×cos(φ)×sin(θp)+b×cos(φ)×cos(θp)) …(16) Finally, the position prediction module 1005 outputs θp and rp as the predicted position to the subtraction module 1006.

[0052] The subtraction module 1006 receives θ, θp, r, and rp as inputs and calculates the respective prediction differences Δθ and Δr, which are calculated for azimuth angle and range using equation (5). The prediction differences Δθ and Δr are output to the arithmetic coding module 1007.

[0053] The arithmetic coding module 1007 arithmetically encodes the input prediction differences Δθ, Δr, the identifier indicating the parent node, and the identifier indicating the parent node, to generate a position information bit stream α and output it to the comparison module 1009.

[0054] The conventional encoding processing module 1008 receives the coordinates (θ, n, r) which are position information as input, performs encoding processing using the method described above with reference to Figure 2, and outputs the position information bit stream β to the comparison module 1009.

[0055] The comparison module 1009 compares the data sizes of the position information bit stream α and the position information bit stream β, selects the smaller one, and adds information indicating the selected one to output as the position information bit stream 1010 of the point.

[0056] Next, a process (encoding method) for encoding position information by the encoding device 1000 according to this embodiment will be described with reference to Fig. 11. Fig. 11 is a flowchart of the encoding method according to this embodiment. Fig. 11 shows an example of a process for encoding target point position information and generating a target point position bitstream.

[0057] First, in step S1101, the encoding apparatus 1000 extracts position information for one point. Then, the encoding apparatus 1000 executes steps S1102 to S1108 and steps S1109 to S1113 in parallel or sequentially.

[0058] In step S1102, the coordinate conversion module 1002 receives coordinates (θ, n, r) that are position information of a point as input, and converts them into coordinates (θ, φ, r) defined by the azimuth angle θ, the elevation angle φ, and the distance r, and coordinates (x, y, z) defined by the X-axis, Y-axis, and Z-axis.

[0059] Next, in step S1103, the tree management module 1003 adds the input coordinates (θ, φ, r) and coordinates (x, y, z) to the tree T1 as a new node. Next, in step S1104, the tree management module 1003 extracts from the tree T1 the coordinates (x1, y1, z1), (θ1, φ, r1) of the parent node of the newly added node and the coordinates (x2, y2, z2), (θ2, φ, r2) of the parent node.

[0060] Next, in step S1105, the wall estimation module 1004 calculates parameters a, b, and d of a plane equation representing a wall perpendicular to the floor from x1, y1, x2, and y2. Next, in step S1106, the position prediction module 1005 receives θ1, θ2, φ, a, b, and d as inputs and predicts coordinates (θp, n, rp) representing the predicted position of the point.

[0061] Next, in step S1107, the subtraction module 1006 calculates prediction differences Δθ and Δr from the coordinates (θ, n, r) of the point position and the coordinates (θp, n, rp) representing the predicted position. Next, in step S1108, the arithmetic coding module 1007 arithmetically codes the input prediction differences Δθ and Δr and the identifier indicating the ancestor node to generate a position information bit stream α.

[0062] Steps S1109 to S1113 are the conventional method described with reference to FIG. 2 and executed by the conventional encoding processing module 1008. First, in step S1109, the conventional encoding processing module 1008 adds a node to the tree and registers position information for the added node. Next, in step S1110, the conventional encoding processing module 1008 obtains position information registered for an ancestor node from the tree. Next, in step S1111, the conventional encoding processing module 1008 predicts the position of the added node using the position information of the ancestor node. Next, in step S1112, the conventional encoding processing module 1008 calculates the difference between the predicted position and the position of the added node. Next, in step S1113, the conventional encoding processing module 1008 calculates and encodes the difference between the identifier of the ancestor node and the predicted position, thereby generating a position information bitstream β.

[0063] Next, in step S1114, the comparison module 1009 compares the bit sizes (data amounts) of the position information bit streams α and β, and selects the one with the smaller data amount. That is, if the position information bit stream α is smaller than the position information bit stream β, the process proceeds to step S1115. On the other hand, if the position information bit stream β is smaller than the position information bit stream α, the process proceeds to step S1116.

[0064] In step S1115, the encoding apparatus 1000 adds identification information of the position information bitstream α (information indicating that the position information bitstream α has been selected) to the position information bitstream α, and outputs the result as a position information bitstream 1010 of the point.

[0065] In step S1116, the encoding apparatus 1000 adds identification information of the position information bitstream β (information indicating that the position information bitstream β has been selected) to the position information bitstream β, and outputs the result as a position information bitstream 1010 of the point.

[0066] The encoding device 1000 of this embodiment assumes that the target point to be processed is part of a continuous point group obtained from a vertical plane such as a wall or a shape close to a plane, and encodes its position on the plane as a predicted position. This makes it possible to efficiently generate a target point position bitstream indicating the position of the target point. The encoding device 1000 also simultaneously performs encoding using conventional techniques and selects the encoding result with the highest encoding efficiency. This allows for efficient encoding processing.

[0067] Note that this embodiment is not limited to the coordinates described with reference to FIGS. 7 and 8, and may be, for example, a left-handed three-dimensional coordinate system. Furthermore, the azimuth angle may be defined based on the X axis rather than the Y axis. Although a parent node and its parent node are used to estimate the plane, this embodiment is not limited to this. For example, any combination of ancestors, such as a grandparent's grandparent node, may be used. Furthermore, this embodiment is not limited to equation (15). For example, three ancestor nodes may be used, and equations (3) and (4) may be used with the azimuth angle limited to them. Furthermore, the input position information is not limited to coordinates (θ, n, r), and may be expressed in other ways, such as (x, y, z). In this case, the coordinate transformation module 1002 may perform a transformation according to the input information.

[0068] Second Embodiment Next, a decoding device 1200 according to a second embodiment of the present invention will be described. The decoding device 1200 decodes a target point position bitstream that is included in a point cloud bitstream and indicates the positions of target points that are the subject of decoding processing. The point cloud bitstream is input to the decoding device 1200. The decoding device 1200 extracts a position bitstream from the point cloud bitstream.

[0069] First, the process (decoding process) of decoding position information by the decoding device 1200 will be described in detail. First, the decoding device 1200 will be described with reference to Fig. 12. Fig. 12 is an explanatory diagram of the decoding device 1200. The decoding device 1200 has a bitstream determination module 1201, an arithmetic decoding module 1202, a tree management module 1203, a wall estimation module 1004, a position prediction module 1005, and an addition module 1204. The decoding device 1200 also has a coordinate transformation module 1002 and a conventional decoding process module (second distance prediction means) 1205. The decoding device 1200 also has a control means 1021 such as a CPU that controls each module, and a storage means (memory) 1022 that stores programs, data, etc. required for executing each module.

[0070] The bitstream determination module 1201 determines whether the position information bitstream of the input point was generated from the position information bitstream α or the position information bitstream β. Then, the bitstream determination module 1201 extracts the position information bitstream α or the position information bitstream β according to the determination result. That is, the bitstream determination module 1201 functions as a switching means for switching between the first distance prediction means and the second distance prediction means. When the position information bitstream α is extracted, the position information bitstream α is output to the arithmetic decoding module 1202. On the other hand, when the position information bitstream β is extracted, the position information bitstream β is output to the conventional decoding processing module 1205.

[0071] The arithmetic decoding module 1202 performs arithmetic decoding on the position information bitstream α to decode an identifier indicating a parent node, an identifier indicating its parent node, and prediction differences Δθ and Δr. The identifier indicating the parent node and the identifier indicating its parent node are output to a tree management module 1203, and the prediction differences Δθ and Δr are output to an addition module 1204.

[0072] The tree management module 1203 uses the input identifier of the parent node and the identifier of the parent node to extract the coordinates (x1, y1, z1) and coordinates (θ1, φ, r1) of the parent node from the tree T2. The tree management module 1203 also extracts the coordinates (x2, y2, z2) and coordinates (θ2, φ, r2) of the parent node and a laser ID (n). The tree management module 1203 then outputs x1, y1, x2, and y2 to the wall estimation module 1004, outputs θ1, θ2, and φ to the position prediction module 1005, and outputs n to the addition module 1204. The input coordinates (θ, φ, r), coordinates (x, y, z), and n are added to the tree T2 as a new node.

[0073] The wall estimation module 1004 calculates plane equations expressed by equations (10) to (14) that represent walls perpendicular to the floor from the input x1, y1, x2, and y2, and outputs the values ​​of a, b, and d to the position prediction module 1005. The position prediction module 1005 receives θ1, θ2, φ, a, b, and d as input, predicts coordinates (θp, n, rp) that represent the predicted position of the point, and outputs θp and rp to the addition module 1204.

[0074] The addition module 1204 adds the input predicted positions θp, rp and the predicted differences Δθ, Δr, respectively, to calculate θ and r. The addition module 1204 also combines the input n to generate coordinates (θ, n, r), and outputs point position information 1001 as the output of the position data decoding unit. The coordinates (θ, n, r) are also output to the coordinate transformation module 1002.

[0075] The coordinate conversion module 1002 receives coordinates (θ, n, r) as input and converts them into coordinates (θ, φ, r) defined by the azimuth angle θ, the elevation angle φ, and the distance r, and coordinates (x, y, z) defined by the X-axis, Y-axis, and Z-axis. To generate coordinates (θ, φ, r) from the coordinates (θ, n, r), the coordinate conversion module 1002 uses a database D that defines the relationship between the laser ID and the elevation angle φ. Next, the coordinate conversion module 1002 calculates coordinates (x, y, z) from the coordinates (θ, φ, r) using equations (7) to (9), and outputs the coordinates (θ, φ, r), coordinates (x, y, z), and n to the tree management module 1203.

[0076] The conventional decoding processing module 1205 uses the aforementioned conventional technology to decode the input position information bit stream β to decode the coordinates (θ, n, r), and outputs the point position information 1001 as the output of the position data decoding unit.

[0077] Next, a process (decoding method) for decoding position information by the decoding device 1200 according to this embodiment will be described with reference to Fig. 13. Fig. 13 is a flowchart of the decoding method.

[0078] First, in step S1301, the decoding apparatus 1200 extracts the position information bitstream 1010 for one point. Next, in step S1302, the bitstream determination module 1201 determines whether the position information bitstream 1010 for one point was generated from the position information bitstream α or the position information bitstream β. Then, the bitstream determination module 1201 extracts the position information bitstream α or the position information bitstream β based on the determination result. If it is determined that the position information bitstream 1010 was generated from the position information bitstream α, the process proceeds to step S1303. On the other hand, if it is determined that the position information bitstream 1010 was generated from the position information bitstream β, the process proceeds to step S1311.

[0079] In step S1303, the arithmetic decoding module 1202 applies arithmetic decoding to the point position information bitstream α to decode the ancestor node identifier and Δθ and Δr representing the prediction difference. Then, in step S1304, the tree management module 1203 references the ancestor node identifier, searches for a parent node in tree T2, and adds a new node. Then, in step S1305, the tree management module 1203 obtains the coordinates (x1, y1, z1) and coordinates (θ1, φ, r1) of the parent node and the coordinates (x2, y2, z2) and coordinates (θ2, φ, r2) of the parent node from tree T2.

[0080] Next, in step S1306, the wall estimation module 1004 calculates parameters a, b, and d of a plane equation representing a wall perpendicular to the floor from x1, y1, x2, and y2. Next, in step S1307, the position prediction module 1005 receives θ1, θ2, φ, a, b, and d as input and predicts coordinates (θp, n, rp) representing the predicted position of the point.

[0081] Next, in step S1308, the addition module 1204 calculates coordinates (θ, n, r) that are point position information 1001 from predicted differences Δθ, Δr that are the difference between predicted positions and coordinates (θp, n, rp) that represent the predicted position. Next, in step S1309, the coordinate transformation module 1002 transforms the point position information. Next, in step S1310, the tree management module 1203 registers the coordinate-transformed point position information in the tree T2 as a new node.

[0082] Steps S1311 to S1316 are conventional techniques executed by the conventional decoding processing module 1205. First, in step S1311, the conventional decoding processing module 1205 applies arithmetic decoding to the point position information bitstream β to decode the difference between the parent node identifier and the predicted position. Next, in step S1312, the conventional decoding processing module 1205 references the parent node identifier, searches the tree for a node corresponding to the parent, and adds a new node. Next, in step S1313, the conventional decoding processing module 1205 obtains position information registered in the ancestor node from the tree. Next, in step S1314, the conventional decoding processing module 1205 calculates point position information using the position information of the ancestor node. Next, in step S1315, the conventional decoding processing module 1205 adds the difference in predicted position to the predicted position to calculate point position information. Next, in step S1316, the conventional decoding processing module 1205 registers the position information of the point in the added node. In this way, the decoding device 1200 calculates the coordinates (θ, n, r) that are the position information 1001 of the point.

[0083] Subsequently, in step S1317, the decoding device 1200 outputs the position information 1001 of the point generated in step S1310 or step S1316.

[0084] The decoding device of this embodiment decodes the object point position bitstream, which indicates the position of the object point to be processed, using either conventional decoding technology or predicting the position on a plane by assuming that the bitstream is part of a continuous point group obtained from a vertical plane such as a wall or a shape close to a plane, thereby decoding the object point position data, which indicates the position of the object point.

[0085] <Third embodiment> Next, a third embodiment of the present invention will be described. In the first embodiment, under the assumption that the rotary LIDAR is measuring a plane perpendicular to the ground, the plane is estimated from three pieces of coded position information, and the encoding target point is predicted. However, it is also possible to estimate a plane that is not limited to being vertical using three pieces of coded position information. That is, the wall estimation module 1004 calculates surface information based on at least three pieces of processed point position data. Furthermore, the wall estimation module 1004 can also calculate three-dimensional surface information as surface information based on at least three pieces of processed point position data. In this case, the three coded points are not aligned on the same line. That is, in the rotary LIDAR, at least one point must be position information obtained from a different laser.

[0086] When the coordinates of three positions (points) are (x1, y1, z1), (x2, y2, z2), and (x3, y3, z3), respectively, the equation of the plane is expressed as in equation (17). In addition, by substituting the respective coordinates into the following equations (18), (19), and (20), and solving the simultaneous equations, the parameters a, b, and d of the equation of the plane can be calculated.

[0087] a×x+b×y+z+d=0 …(17) a×x1+b×y1+z1+d=0 …(18) a×x²+b×y²+z²+d=0 …(19) a×x3+b×y3+z3+d=0 …(20) The predicted distance rp from the origin to the intersection of the line indicating the laser direction represented by θp and φ and the plane represented by a, b, and d can be predicted using the following equation (21):

[0088] rp=-d÷(a×cos(φ)×sin(θp)+b×cos(φ)×cos(θp)+c×sin(φ)) …(21) <Fourth embodiment> Next, a fourth embodiment of the present invention will be described. In the third embodiment, a plane is estimated and predicted using three pieces of coded position information, but four or more pieces of coded position information may also be used. The parameters a, b, and d of the plane equation can be calculated by applying the least squares method to equation (17). If the number of points used to estimate the plane increases, it becomes necessary to wait until the required number of points have been acquired, which increases the delay, but there is a possibility that the method will be more robust against noise contained in the position information of the points.

[0089] <Other embodiments> The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0090] According to each embodiment, by predicting the position of a feature by assuming that the feature has a three-dimensional planar shape perpendicular to the ground, it is possible to perform encoding with a higher compression rate and lower latency than conventional techniques for point clouds obtained from planar features such as walls. Furthermore, even for features that are not planar, such as cylindrical features such as pillars, they can be locally regarded as planar, allowing for improved compression rate and lower latency encoding.

[0091] Therefore, according to each embodiment, it is possible to provide an encoding device, a decoding device, an encoding method, a decoding method, and a program that are capable of highly efficient and low-delay processing.

[0092] The disclosure of each embodiment includes the following configurations and methods. (Configuration 1) a surface information calculation means for calculating surface information using processed point position data indicating positions of processed points for which encoding processing has been completed, which are included in the point cloud data; a direction information prediction means for predicting direction information of a target point that is a target of the encoding process using the processed point position data; a first distance prediction means for predicting a distance to the target point based on the surface information and the direction information; An encoding device comprising: (Configuration 2) the surface information calculation means calculates the surface information based on the two processed point position data; the first distance prediction means predicts the distance to the target point based on an intersection on a surface calculated based on the surface information and the direction information; 2. The encoding device according to configuration 1, (Configuration 3) The surface information is information about a plane perpendicular to the ground. 3. The encoding device according to configuration 2. (Configuration 4) The point cloud data is data acquired from a point cloud acquisition device. 4. The encoding device according to configuration 3. (Configuration 5) the surface information calculation means calculates the surface information based on at least three of the processed point position data; 2. The encoding device according to configuration 1, (Configuration 6) a second distance prediction means for predicting a position of the target point based on position information of a plurality of point groups input in the past, and predicting the distance to the target point based on a difference between the predicted position and the position of the target point; a switching means for switching between the first distance prediction means and the second distance prediction means; 6. The encoding device according to any one of configurations 1 to 5, further comprising: (Configuration 7) the switching means switches between the first distance prediction means and the second distance prediction means so as to use the data based on the distance predicted by the first distance prediction means or the data based on the distance predicted by the second distance prediction means, whichever has a smaller amount of data. 7. The encoding device according to configuration 6. (Method 1) calculating surface information using processed point position data indicating positions of processed points for which encoding processing has been completed, the processed point position data being included in the point cloud data; a step of predicting direction information of a target point that is a target of the encoding process using the processed point position data; predicting a distance to the target point based on the surface information and the direction information; 10. A method for encoding a video signal, comprising: (Configuration 8) calculating surface information using processed point position data indicating positions of processed points for which encoding processing has been completed, the processed point position data being included in the point cloud data; a step of predicting direction information of a target point that is a target of the encoding process using the processed point position data; predicting a distance to the target point based on the surface information and the direction information; A program characterized by causing a computer to execute the above. (Configuration 9) An encoding device that decodes a target point position bitstream that indicates the positions of target points that are targets of a decoding process and are included in a point cloud bitstream, comprising: a surface information calculation means for calculating surface information using processed point position data indicating the position of the processed point for which the decoding process has been completed; a direction information prediction means for predicting direction information of the target point using the processed point position data; a first distance prediction means for predicting a distance to the target point based on the surface information and the direction information; A decoding device comprising: (Configuration 10) the surface information calculation means calculates the surface information based on the two processed point position data; the distance prediction means predicts the distance to the target point based on an intersection on a surface calculated based on the surface information and the direction information. 10. The decoding device according to configuration 9, (Configuration 11) The surface information is information about a plane perpendicular to the ground. 11. The decoding device according to configuration 10, (Configuration 12) The point cloud bitstream is data acquired from a point cloud acquisition device placed on the ground. 12. The decoding device according to claim 11, (Configuration 13) the surface information calculation means calculates the surface information based on at least three of the processed point position data; 10. The decoding device according to configuration 9, (Configuration 14) a second distance prediction means for predicting a position of the target point based on position information of a plurality of point groups input in the past, and predicting a distance to the target point based on a difference between the predicted position and the position of the target point; a switching means for switching between the distance prediction means and the second distance prediction means; 14. The decoding device according to any one of configurations 9 to 13, further comprising: (Method 2) An encoding method for decoding a target point position bitstream that indicates the positions of target points that are targets of a decoding process and that are included in a point cloud bitstream, comprising: calculating surface information using processed point position data indicating the positions of processed points for which the decoding process has been completed; predicting orientation information of the target points using the processed point position data; predicting a distance to the target point based on the surface information and the direction information; A decoding method comprising: (Configuration 15) An encoding method for decoding a target point position bitstream that indicates the positions of target points that are targets of a decoding process and that are included in a point cloud bitstream, comprising: calculating surface information using processed point position data indicating the positions of processed points for which the decoding process has been completed; predicting orientation information of the target points using the processed point position data; predicting a distance to the target point based on the surface information and the direction information; A program characterized by causing a computer to execute the above.

[0093] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0094] 1000 encoding device 1004 Wall estimation module (surface information calculation means) 1005a Directional information prediction means 1005b First distance prediction means

Claims

1. a surface information calculation means for calculating surface information using processed point position data indicating the positions of processed points for which encoding processing has been completed, which are included in the point cloud data; a direction information prediction means for predicting direction information of a target point that is a target of the encoding process using the processed point position data; a first distance prediction means for predicting a distance to the target point based on the surface information and the direction information; An encoding device comprising:

2. the surface information calculation means calculates the surface information based on the two processed point position data; the first distance prediction means predicts the distance to the target point based on an intersection on a surface calculated based on the surface information and the direction information; 2. The encoding device according to claim 1.

3. The surface information is information about a plane perpendicular to the ground.

3. The encoding device according to claim 2.

4. The point cloud data is data acquired from a point cloud acquisition device.

4. The encoding device according to claim 3.

5. the surface information calculation means calculates the surface information based on at least three of the processed point position data; 2. The encoding device according to claim 1.

6. a second distance prediction means for predicting a position of the target point based on position information of a plurality of point groups input in the past, and predicting the distance to the target point based on a difference between the predicted position and the position of the target point; a switching means for switching between the first distance prediction means and the second distance prediction means; 6. The encoding device according to claim 1, further comprising:

7. the switching means switches between the first distance prediction means and the second distance prediction means so as to use the data based on the distance predicted by the first distance prediction means or the data based on the distance predicted by the second distance prediction means, whichever has a smaller amount of data.

7. The encoding device according to claim 6.

8. calculating surface information using processed point position data indicating positions of processed points for which encoding processing has been completed, the processed point position data being included in the point cloud data; a step of predicting direction information of a target point that is a target of the encoding process using the processed point position data; predicting a distance to the target point based on the surface information and the direction information; 10. A method for encoding a video signal, comprising:

9. calculating surface information using processed point position data indicating positions of processed points for which encoding processing has been completed, the processed point position data being included in the point cloud data; a step of predicting direction information of a target point that is a target of the encoding process using the processed point position data; predicting a distance to the target point based on the surface information and the direction information; A program characterized by causing a computer to execute the above.

10. An encoding device that decodes a target point position bitstream that indicates the positions of target points that are targets of a decoding process and are included in a point cloud bitstream, comprising: a surface information calculation means for calculating surface information using processed point position data indicating the position of the processed point for which the decoding process has been completed; a direction information prediction means for predicting direction information of the target point using the processed point position data; a first distance prediction means for predicting a distance to the target point based on the surface information and the direction information; A decoding device comprising:

11. the surface information calculation means calculates the surface information based on the two processed point position data; the distance prediction means predicts the distance to the target point based on an intersection on a surface calculated based on the surface information and the direction information.

11. The decoding device according to claim 10.

12. The surface information is information about a plane perpendicular to the ground.

12. The decoding device according to claim 11,

13. The point cloud bitstream is data acquired from a point cloud acquisition device placed on the ground.

13. The decoding device according to claim 12.

14. the surface information calculation means calculates the surface information based on at least three of the processed point position data; 11. The decoding device according to claim 10.

15. a second distance prediction means for predicting a position of the target point based on position information of a plurality of point groups input in the past, and predicting a distance to the target point based on a difference between the predicted position and the position of the target point; a switching means for switching between the distance prediction means and the second distance prediction means; 15. Decoding device according to any one of claims 10 to 14, further comprising:

16. An encoding method for decoding a target point position bitstream that indicates the positions of target points that are targets of a decoding process and that are included in a point cloud bitstream, comprising: calculating surface information using processed point position data indicating the positions of processed points for which the decoding process has been completed; predicting orientation information of the target points using the processed point position data; predicting a distance to the target point based on the surface information and the direction information; A decoding method comprising:

17. An encoding method for decoding a target point position bitstream that indicates the positions of target points that are targets of a decoding process and that are included in a point cloud bitstream, comprising: calculating surface information using processed point position data indicating the positions of processed points for which the decoding process has been completed; predicting orientation information of the target points using the processed point position data; predicting a distance to the target point based on the surface information and the direction information; A program characterized by causing a computer to execute the above.

Images