Encoder, decoder, and computer program

The encoding device enhances LiDAR data compression by predicting points and encoding differences, addressing inefficiencies in complex scan paths and environmental dependencies, thus maintaining high compression efficiency.

JP2024162356AInactive Publication Date: 2024-11-21CANON KK

Patent Information

Application Number
JP2023077766
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-10
Publication Date
2024-11-21
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing LiDAR data compression methods face inefficiencies when dealing with complex scan paths and environmental dependencies, leading to decreased encoding efficiency.

Method used

An encoding device that calculates predicted points based on measurement model information, encodes differences between measurement points, and utilizes angular coding to reduce data volume, especially for rotating LiDAR systems.

Benefits of technology

Improves encoding efficiency by accurately predicting positions and reducing data volume, maintaining high compression efficiency even with complex scan paths and environmental variations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024162356000001_ABST
    Figure 2024162356000001_ABST
Patent Text Reader

Abstract

To provide an encoder that can improve the efficiency of encoding when performing scanning by using irradiation with a laser beam.SOLUTION: An encoder comprises: acquisition means that acquires point group data consisting of a plurality of measurement points from measurement means; calculation means that calculates prediction points corresponding to the plurality of measurement points on the basis of measurement model information related to a measurement pattern of the measurement means; operation means that operates the difference between the measurement points and the prediction points corresponding to the measurement points; and encoding means that encodes the difference.SELECTED DRAWING: Figure 7
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an encoding device, a decoding device, a computer program, and the like. [Background technology]

[0002] In recent years, with the development of autonomous driving technology and the spread of VR (Virtual Reality) technology, the development of LiDAR (Light Detection And Ranging) technology, which can measure three-dimensional position information, has been progressing.

[0003] LiDAR is a device that measures the 3D position information of surrounding objects by discretely emitting a laser while changing the irradiation angle of the laser at high speed and measuring the time it takes for the reflected light of the laser to return at each irradiation angle. LiDAR outputs a collection of points that indicate the 3D position information of surrounding objects as point cloud data based on the distance information to the object at each laser irradiation angle.

[0004] Generally, point cloud data includes a large number of points, and each point has multiple attribute information such as time stamp information, intensity information, RGB information, etc. Since point cloud data is a large amount of data, it may be costly to store and communicate the data, and it may take time to process the data.

[0005] In response to these issues, there is a demand for reducing point cloud data. For example, MPEG (Moving Picture Experts Group) provides G-PCC (Geometry based Point Cloud Compression), which is a method for encoding point clouds, as disclosed in Non-Patent Document 1.

[0006] G-PCC encodes the geometry, which is the three-dimensional coordinates of each point in the point cloud data, and the attribute information, such as timestamp and intensity, using separate processes.

[0007] Furthermore, for example, Patent Document 1 describes a data compression device that generates, as compressed data, parameters for approximating measurement data acquired by LiDAR using a template that represents the outline of a feature. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] JP 2018-116452 A [Non-patent literature]

[0009] [Non-Patent Document 1] G-PCC codec description v2, ISO / IEC MPEG (JTC 1 / SC 29 / WG11), N18189 Summary of the Invention [Problem to be solved by the invention]

[0010] However, in the method described in Non-Patent Document 1, when the LiDAR has a complex scanning path, the difference between the azimuth and elevation angles predicted from the past scan and the azimuth and elevation angles of the next scan becomes large, resulting in a decrease in compression efficiency during encoding. In addition, the method in Patent Document 1 requires a template that represents the outline of a feature, and is therefore susceptible to the influence of the surrounding environment.

[0011] Therefore, one object of the present invention is to provide a coding device capable of improving the coding efficiency. [Means for solving the problem]

[0012] In order to achieve the above objective, the encoding device of the present invention comprises: An acquisition means for acquiring point cloud data consisting of a plurality of measurement points from a measurement means; a calculation means for calculating predicted points corresponding to the plurality of measurement points based on measurement model information related to a measurement pattern of the measurement means; A calculation means for calculating a difference between the measurement point and the prediction point corresponding to the measurement point; encoding means for encoding the difference; The present invention is characterized by comprising: Effect of the Invention

[0013] According to the present invention, it is possible to provide a coding device capable of improving the coding efficiency. [Brief description of the drawings]

[0014] [Figure 1] FIG. 2 is a functional block diagram showing an example of a G-PCC encoding process according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a functional block diagram illustrating the encoding process of predictive coding according to an embodiment of the present invention. [Diagram 3] 2 is a diagram illustrating an example of processing performed by a position prediction module 203 and a difference calculation module 204 in predictive coding according to an embodiment of the present invention. FIG. [Figure 4] 1 is a diagram illustrating an example of a tree generated in Angular Coding Mode according to an embodiment of the present invention. FIG. [Diagram 5] FIG. 2 is a functional block diagram illustrating the decoding process of G-PCC according to an embodiment of the present invention. [Figure 6] FIG. 2 is a functional block diagram illustrating a decoding process of predictive coding according to an embodiment of the present invention. [Figure 7] FIG. 2 is a diagram showing an example of a scanning path of a LiDAR for measuring an encoding target in the first embodiment of the present invention. [Figure 8] FIG. 1 is a functional block diagram illustrating an example of a schematic configuration of a point group encoding device 800 according to a first embodiment of the present invention. [Figure 9]FIG. 1 is a functional block diagram showing an example of a schematic configuration of a point group decoding device 900 according to a first embodiment of the present invention. [Figure 10] FIG. 2 is a diagram showing an example of an angle model according to the first embodiment of the present invention. [Figure 11] 1 is a functional block diagram illustrating an example of a schematic configuration of an angle information encoding device according to a first embodiment of the present invention. [Figure 12] 1 is a functional block diagram illustrating an example of a schematic configuration of an angle information decoding device according to a first embodiment of the present invention. [Figure 13] 11 is a flowchart showing an example of processing by an angle calculation unit 1102 according to the first embodiment of the present invention. [Figure 14] 13 is a flowchart showing an example of processing by an angle calculation unit 1203 according to the first embodiment of the present invention. [Figure 15] 10 is a flowchart showing an example of processing performed by an azimuth angle / elevation angle difference calculation unit 1103 according to the first embodiment of the present invention. [Figure 16] 10 is a flowchart showing an example of processing by an azimuth angle / elevation angle calculation unit 1204 according to the first embodiment of the present invention. [Figure 17] FIG. 11 is a functional block diagram illustrating an example of a schematic configuration of an angle information encoding device 1700 according to a second embodiment of the present invention. [Figure 18] FIG. 18 is a functional block diagram illustrating an example of a schematic configuration of an angle information decoding device 1800 according to a second embodiment of the present invention. [Figure 19] 13 is a flowchart showing an example of processing by an angle calculation unit 1702 according to the second embodiment of the present invention. [Figure 20] 13 is a flowchart showing an example of processing by an angle calculation unit 1803 according to the second embodiment of the present invention. [Figure 21] FIG. 11 is a functional block diagram illustrating an example of a schematic configuration of an angle information encoding device 2100 according to a third embodiment of the present invention. [Figure 22] FIG. 11 is a functional block diagram illustrating an example of a schematic configuration of an angle information decoding device 2200 according to Example 3 of the present invention. [Figure 23] 13 is a flowchart showing an example of processing by an angle model update unit 2102 according to the third embodiment of the present invention. [Figure 24]13 is a flowchart showing an example of processing by an angle model update unit 2202 according to the third embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, the embodiment of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiment. In each drawing, the same members or elements are given the same reference numerals, and duplicated descriptions are omitted or simplified.

[0016] Fig. 1 is a functional block diagram showing an example of a G-PCC encoding process according to an embodiment of the present invention. Note that some of the functional blocks shown in Fig. 1 are realized by causing a CPU or the like as a computer (not shown) included in the encoding device to execute a computer program stored in a memory (not shown) as a storage medium.

[0017] However, a part or all of these may be realized by hardware, which may be a dedicated circuit (ASIC) or a processor (a reconfigurable processor, DSP), etc.

[0018] In addition, the functional blocks shown in Fig. 1 do not have to be built in the same housing, and may be configured by separate devices connected to each other via signal paths. The above explanation regarding Fig. 1 also applies to Figs. 2, 5, 6, 8, 9, 11, 12, 17, 18, 21, and 22.

[0019] The encoding process of G-PCC will be described with reference to Fig. 1. Reference numeral 100 denotes a LiDAR that irradiates a target object with a laser as a measurement means, and acquires point cloud data consisting of a plurality of measurement points. When point cloud data 101 measured by the LiDAR 100 is input, it is first decomposed into position information 107 and attribute information 109 by a decomposition module 102. The position information 107 is converted into a position information bit stream 108 by a position information encoding module 103.

[0020] The attribute information 109, together with the position information 107, is converted into an attribute information bit stream 110 in an attribute information encoding module 104. Then, the position information bit stream 108 and the attribute information bit stream 110 are input to a synthesis module 105, where they are synthesized into encoded data 106 and output.

[0021] The location information encoding module 103 of the G-PCC uses predictive geometry coding as an encoding method suitable for point cloud data output by the LiDAR.

[0022] Predictive geometry coding is a technique in which the position of an input point is predicted based on the position information of multiple points input in the past, and the difference between the predicted position and the position information of the input point is encoded.

[0023] FIG. 2 is a functional block diagram for explaining the encoding process of predictive coding according to an embodiment of the present invention, and shows details of predictive geometry coding in position information encoding module 103 in FIG.

[0024] First, information for one point is extracted from the point cloud position information 107 to obtain point position information 201. This point position information 201 is three-dimensional coordinate information. Next, the point position information 201 is added as a node of a tree (prediction tree) in a tree update module 202. The tree can be updated arbitrarily based on the point position information 201, but typically the position of the point corresponding to the node is evaluated, and a branch indicating the encoding order is established between nodes that are close to each other.

[0025] After the tree is updated, a position prediction module 203 performs position prediction using the tree, and outputs a predicted position 207 and an identifier 208 of a parent node used for the prediction. A difference calculation module 204 calculates the difference (predicted position difference 209) between the predicted position 207 and point position information 201, and inputs the difference together with the parent node identifier 208 to an arithmetic coding module 205. In the arithmetic coding module 205, the predicted position difference 209 is arithmetically coded based on the parent node identifier 208, and is output as a bit stream 206 of point position information.

[0026] In addition, the smaller the predicted position difference 209, the higher the coding efficiency, and the larger the predicted position difference 209, the lower the coding efficiency. In other words, the closer the predicted position 207 is to the point position information 201, the higher the coding efficiency, and therefore it is important for the position prediction module 203 to perform accurate position prediction in order to increase the coding efficiency.

[0027] 3 is a diagram for explaining an example of processing performed by the position prediction module 203 and the difference calculation module 204 of predictive coding according to an embodiment of the present invention. Reference numeral 301 denotes a tree, in which n is a node to be predicted, the parent node of n is represented as n0, the parent node of n0 as n1, and the parent node of n1 as n2.

[0028] In the position prediction module 203, if an arbitrary node is x, the position of x is represented as Pos(x), and the predicted position is represented as Pred(x),

[0029] Pred(n)=Pos(n0)+Pos(n0)-Pos(n1) Or, Pred(n)=Pos(n1)+Pos(n0)-Pos(n2) Prediction is performed by one of the prediction methods. Pred(n) corresponds to the prediction position 207 in FIG.

[0030] Next, the difference calculation module 204 calculates the difference between the predicted position and the position of the point corresponding to node n. If an arbitrary node is x and the difference between the predicted position of x and the position is represented as Delta(x), then Delta(n) = Pred(n) - Pos(n) where Delta(n) corresponds to the predicted position difference 209 in FIG.

[0031] In this embodiment, the Angular Coding Mode is applied when applying the predictive geometry coding to the point cloud acquired by the rotational LiDAR. In the angular coding mode, three-dimensional information combining the ID for identifying the laser, the azimuth angle, and the distance is used as the position information of the points acquired by the rotational LiDAR.

[0032] Since rotating LiDAR has the characteristic that the elevation angle does not change if the same laser is used, the amount of information can be reduced by using an ID that identifies the laser instead of the elevation angle.

[0033] FIG. 4 is a diagram illustrating an example of a tree generated in the Angular Coding Mode according to an embodiment of the present invention.

[0034] As shown in 401, the upward direction in the figure represents the ID for identifying the laser, the right direction represents the azimuth angle, and the direction from the front to the back represents the distance. 1 to N in Fig. 4 represent the laser ID, and as shown in 402, a node is added to the points acquired by each laser so that the parent point is the point acquired by the laser with the same ID, generating a tree.

[0035] 5 is a functional block diagram for explaining the decoding process of the G-PCC according to the embodiment of the present invention. The input encoded data 106 is decomposed by a decomposition module 501, and a position information bit stream 108 and an attribute information bit stream 110 are output. The position information bit stream 108 is decoded by a position information decoding module 502 into the position information 107.

[0036] The attribute information bitstream 110 is input to an attribute information decoding module 503 together with the position information bitstream 108, and is decoded into attribute information 109. The position information 107 and the attribute information 109 are input to a synthesis module 504, and are output as synthesized point cloud data 101.

[0037] FIG. 6 is a functional block diagram for explaining the decoding process of predictive coding according to an embodiment of the present invention, and shows an example of the configuration of the position information decoding module 502 shown in FIG.

[0038] Data for one point extracted from the position information bit stream 108 is called a point position information bit stream 206. The position information bit stream 206 is input to an arithmetic decoding module 601, which restores a parent node identifier 208 and a predicted position difference 209.

[0039] In the tree update module 602, the parent node n0 is determined using the parent node identifier 208, and the node n is added as a child of n0. Next, the position prediction module 203 calculates the predicted position 207. After that, the position calculation module 603 calculates the position information 201 of the point from the predicted position 207 and the difference 209 between the predicted position.

[0040] If the predicted position 207 is represented as Pred(n), the difference between the predicted positions 209 is represented as Delta(n), and the position information of the point 201 is represented as Pos(n), then Pos(n)=Pred(n)+Delta(n) It is.

[0041] Example 1 7 is a diagram showing an example of a scanning path of a LiDAR for measuring an encoding target in the first embodiment of the present invention. In the first embodiment, the LiDAR 700 is controlled by, for example, a MEMS (Micro Electro Mechanical Systems) so that the azimuth angle and elevation angle for irradiating a laser 702 are set to a predetermined pattern of a scanning path as shown in FIG.

[0042] The scan path is not limited to the pattern shown in Fig. 7, and may be any pattern. In the first embodiment, a scan path 701 shown in Fig. 7 is set in advance by design values, and the azimuth angle and elevation angle at which the laser 702 is irradiated are adjusted, and the distance from the LiDAR 700 to the target object 703 is obtained based on information on the reflected light obtained from the target object 703.

[0043] The laser is irradiated in the order of unique numbers (hereafter referred to as "scan IDs") that correspond to the order in which the laser is irradiated, and when the scan of the last scan ID is completed, the scan of the first scan ID is performed again. For example, point 704 indicates a point whose scan ID is an arbitrary integer n, and after point 704 is scanned, point 705 whose scan ID is n+1 is scanned. Scanning is repeated until the LiDAR operation stops.

[0044] Each point in the point cloud obtained from the LiDAR includes information on distance, azimuth, elevation, and the scan ID of the point or the relative time when the scan was performed from scan ID=0 (hereinafter referred to as the "scan time").

[0045] 8 is a functional block diagram showing an example of a schematic configuration of a point cloud encoding device 800 according to the first embodiment of the present invention. The point cloud encoding device 800 has a CPU as a computer (not shown) and a memory as a storage medium storing a computer program.

[0046] The decomposition module 801 acquires point position information 201 and decomposes it into point angle information 802 and point distance information 805. Here, the decomposition module 801 functions as an acquisition means for acquiring point cloud data consisting of a plurality of measurement points from a LiDAR serving as a measurement means. The decomposition module 801 as an acquisition means also acquires point cloud data consisting of a plurality of measurement points generated by a LiDAR serving as a measurement means performing a scan by irradiating a laser.

[0047] The point angle information 802 includes the azimuth angle, elevation angle, and scan ID or scan time of the point. An angle information encoding device 803 encodes the point angle information 802 and outputs a bit stream of point angle information 804. The encoding method will be described later.

[0048] The point distance information 805 includes a distance value of the point. The distance information encoding device 806 performs encoding using, for example, the technique shown in Non-Patent Document 1, with the point distance information 805 as input, and outputs a bit stream 807 of the point distance information. The synthesis module 808 synthesizes the bit stream 804 of the angle information and the bit stream 807 of the point distance information, and outputs the bit stream 206 of the point position.

[0049] 9 is a functional block diagram showing an example of a schematic configuration of a point group decoding device 900 according to the first embodiment of the present invention. The point group decoding device 900 has a CPU as a computer (not shown) and a memory as a storage medium storing a computer program.

[0050] The decomposition module 901 decomposes the bit stream 206 of point position information into a bit stream 804 of point angle information and a bit stream 807 of point distance information. The angle information decoding device 902 decodes the angle information of the points and outputs angle information 802 of the points.

[0051] The decoding method will be described later. The distance information decoding device 903 receives the bit stream 807 of the distance information of the points, performs decoding using, for example, the method shown in Non-Patent Document 1, and outputs the distance information 805 of the points. The synthesis module 904 synthesizes the angle information 802 of the points and the distance information 805 of the points, and outputs the position information 201 of the points.

[0052] Next, the angle information encoding device 803 and the angle information decoding device 902 of this embodiment will be described in detail.

[0053] 10 is a diagram showing an example of an angle model according to the first embodiment of the present invention. The angle model (table) in this embodiment has a scan ID, an azimuth angle, and an elevation angle. Here, the angle model is information indicating a path (scan path) when a LiDAR, which is a point cloud measuring device, measures a point cloud.

[0054] More specifically, the angle model is an angle model that indicates the azimuth angle and elevation angle of each point. Note that the angle model may be information that indicates a two-dimensional trajectory that a laser beam emitted by a LiDAR, which is a point cloud measuring device, draws on the surface of a measurement object during a scanning operation.

[0055] The azimuth angle indicates the angle in the horizontal direction in Fig. 7, and the elevation angle indicates the angle in the vertical direction in Fig. 7. The scan ID, azimuth angle, and elevation angle are recorded in the same row, and by referring to the scan ID, the corresponding azimuth angle and elevation angle information can be obtained from the angle model. In the first embodiment, the angle information 802 of the input point includes the azimuth angle, elevation angle, and scan ID.

[0056] FIG. 11 is a functional block diagram showing an example of a schematic configuration of an angle information encoding device 803 according to the first embodiment of the present invention.

[0057] The angle information encoding device 803 shown in FIG. 11 includes an angle model storage unit 1101, an angle calculation unit 1102, an azimuth angle / elevation angle difference calculation unit 1103, and an arithmetic encoding unit 1104.

[0058] The angle model storage unit 1101 stores data of an angle model (measurement model information) based on the above-mentioned design value of the LiDAR. The measurement model information includes information on each point in the point cloud data, and is managed by a scan ID based on the laser scan position or timing.

[0059] That is, the measurement model information includes information on a plurality of points managed by IDs based on the laser irradiation position or laser irradiation timing during scanning by LiDAR as a measurement means. Note that "during scanning" does not mean the actual measurement time, but rather the data that is taken beforehand assuming a scan and used for modeling.

[0060] Furthermore, the information on the multiple measurement points stored in the angle model storage unit 1101 includes laser angle information, i.e., information on the azimuth angle and elevation angle of the laser. That is, the information on the multiple measurement points includes angle information related to the laser irradiation angle when the measurement means measures, and the measurement model information includes angle model information related to the laser irradiation angle when the measurement means scans.

[0061] The angle information includes the azimuth angle and elevation angle of the laser when the measuring means measures the object, and the angle model information includes the azimuth angle model and elevation angle model of the laser when the measuring means scans the object. The laser irradiation time information may include laser irradiation timing information or the like.

[0062] That is, the information on the multiple measurement points includes irradiation time information indicating the timing of laser irradiation when the measurement means is measuring, and the measurement model information includes irradiation time model information indicating the timing of laser irradiation when the measurement means is scanning.

[0063] The angle calculation unit 1102 extracts the azimuth angle and elevation angle values ​​corresponding to the scan ID of the input angle information 802 of the point from the angle model storage unit 1101 , and outputs them to the azimuth angle / elevation angle difference calculation unit 1103 .

[0064] That is, the angle calculation unit 1102 predicts the azimuth angle and elevation angle of the input point based on the angle model data, and calculates the predicted azimuth angle and the predicted elevation angle. Here, the angle calculation unit 1102 functions as a calculation means that calculates predicted points corresponding to multiple measurement points based on measurement model information related to the measurement pattern of the measurement means.

[0065] The azimuth angle / elevation angle difference calculation unit 1103 calculates the difference between the predicted azimuth angle and elevation angle values ​​obtained from the angle calculation unit 1102 and the measured azimuth angle and elevation angle values ​​obtained from the point cloud data, and outputs the obtained difference value to the arithmetic coding unit 1104. The azimuth angle / elevation angle difference calculation unit 1103 functions as a calculation means for calculating the difference between a measurement point and a predicted point corresponding to the measurement point.

[0066] The arithmetic coding unit 1104 obtains the difference value of the azimuth angle and the elevation angle output from the azimuth angle / elevation angle difference calculation unit 1103, and performs arithmetic coding. The obtained result is output as a bit stream 804 of angle information of the point. The arithmetic coding unit 1104 functions as a coding means for coding the difference.

[0067] 12 is a functional block diagram showing an example of a schematic configuration of an angle information decoding device 902 according to Example 1 of the present invention. The angle information decoding device 902 shown in FIG. 12 includes an arithmetic decoding unit 1201, an angle model storage unit 1202, an angle calculation unit 1203, and an azimuth angle / elevation angle calculation unit 1204.

[0068] The arithmetic decoding unit 1201 arithmetically decodes the bit stream 804 of angle information output by the angle information encoding device 803, outputs a scan ID to the angle calculation unit 1203, and outputs the difference values ​​of the azimuth angle and elevation angle to the azimuth angle / elevation angle calculation unit 1204.

[0069] The angle model storage unit 1202 stores angle model data based on the above-mentioned LiDAR design values.

[0070] The angle calculation unit 1203 identifies the scan ID of the difference value from the input order of the azimuth angle and elevation angle difference values, and outputs the azimuth angle and elevation angle values ​​of the corresponding scan ID from the angle model storage unit 1202 to the azimuth angle / elevation angle calculation unit 1204.

[0071] The azimuth angle / elevation angle calculation unit 1204 calculates the sum of the azimuth angle and elevation angle difference values ​​obtained from the arithmetic decoding unit 1201 and the angle prediction value obtained from the angle calculation unit 1203, and outputs the azimuth angle and elevation angle values ​​as angle information 802 of the point cloud.

[0072] In this way, the angle information decoding device 902 of the first embodiment has measurement model information related to the measurement pattern of the measurement means, and has a decoding means for decoding the information of the measurement point based on the comparison result between the measurement model information and the angle information included in the encoded data.

[0073] Fig. 13 is a flowchart showing a processing example of the angle calculation unit 1102 according to the first embodiment of the present invention, and a processing example of the angle calculation unit 1102 of the angle information encoding device 803 will be described with reference to Fig. 13. Note that the operation of each step of the flowchart in Fig. 13 is sequentially performed by a CPU or the like as a computer (not shown) in the point group encoding device 800 executing a computer program stored in a memory.

[0074] In step S1301, an angle model is obtained from the angle model storage unit 1101. Next, in step S1302, a scan ID is obtained from the angle information of the point. Furthermore, in step S1303, the values ​​of the azimuth angle and elevation angle corresponding to the scan ID are output from the angle model, and the process ends.

[0075] Fig. 14 is a flowchart showing a processing example of the angle calculation unit 1203 according to the first embodiment of the present invention, and a processing example of the angle calculation unit 1203 of the angle information decoding device 902 will be described with reference to Fig. 14. Note that the operation of each step of the flowchart in Fig. 14 is sequentially performed by a CPU or the like as a computer (not shown) in the point group decoding device 900 executing a computer program stored in a memory.

[0076] In step S1401, an angle model is obtained from the angle model storage unit 1202. Next, in step S1402, a scan ID is obtained from the input order of the bit stream of angle information of the point. In step S1403, the azimuth angle and elevation angle values ​​corresponding to the scan ID are output from the angle model value, and the process ends.

[0077] Fig. 15 is a flowchart showing a processing example of the azimuth angle / elevation angle difference calculation unit 1103 according to the first embodiment of the present invention, and a processing example of the azimuth angle / elevation angle difference calculation unit 1103 of the angle information encoding device 803 will be described with reference to Fig. 15. Note that the operation of each step of the flowchart in Fig. 15 is performed sequentially by a CPU or the like as a computer (not shown) in the point cloud encoding device 800 executing a computer program stored in a memory.

[0078] In step S1501, the measurement values ​​of the azimuth angle and elevation angle of the angle information of the input point are obtained. Next, in step S1502, the scan ID corresponding to the input azimuth angle and elevation angle is extracted from the point cloud data, and the values ​​of the azimuth angle and elevation angle corresponding to the scan ID are obtained as predicted values ​​from the angle model.

[0079] Next, in step S1503, the difference between the measured and predicted values ​​of the azimuth and elevation angles is calculated, and in step S1504, the difference value is output to the arithmetic coding unit 1104, and the process ends.

[0080] Fig. 16 is a flowchart showing an example of processing by the azimuth angle / elevation angle calculation unit 1204 according to the first embodiment of the present invention, and a processing example by the azimuth angle / elevation angle calculation unit 1204 of the angle information decoding device 902 will be described with reference to Fig. 16. Note that the operation of each step in the flowchart in Fig. 16 is performed sequentially by a CPU or the like as a computer (not shown) in the point cloud decoding device 900 executing a computer program stored in a memory.

[0081] In step S1601, the difference values ​​of the azimuth angle and the elevation angle are obtained from the bit stream of the decoded angle information. Next, in step S1602, the scan ID is obtained based on the input order of the bit stream of the angle information of the point, and the azimuth angle and the elevation angle values ​​corresponding to the scan ID are obtained as predicted values ​​from the angle model.

[0082] Next, in step S1603, the sum of the calculated values ​​and the predicted values ​​of the azimuth angle and the elevation angle is calculated. The obtained sum is output as point cloud data (azimuth angle and elevation angle values) in step S1604, and the process ends.

[0083] In this manner, according to this embodiment, the angle information encoding device 803 can generate an angle information bit stream of points while maintaining compression efficiency even if the scan path is complex, by utilizing the angle model (table) held by the angle model memory unit 1101.

[0084] Furthermore, the angle information decoding device 902 can decode the angle information bit stream of points generated by the angle information encoding device 803 by using the angle model (table) stored in the angle model storage unit 1202. Therefore, it is possible to obtain point group encoded data with high compression efficiency.

[0085] Example 2 Next, a second embodiment of the present invention will be described. The angle model in the second embodiment has displacement information of the azimuth angle and elevation angle direction of the scan obtained based on the design value of the LiDAR. Furthermore, the angle model in the second embodiment is a function that can output azimuth angle and elevation angle information of the point cloud using the scan time as an input. In the second embodiment, the angle information of the input points includes the azimuth angle, the elevation angle, and the scan time.

[0086] Fig. 17 is a functional block diagram showing a schematic configuration example of an angle information encoding device 1700 according to a second embodiment of the present invention, and Fig. 18 is a functional block diagram showing a schematic configuration example of an angle information decoding device 1800 according to the second embodiment of the present invention. The configurations of Fig. 17 and Fig. 18 are the same as those of Fig. 11 and Fig. 12, respectively, but differ in that they handle scan time.

[0087] A system configuration diagram according to the second embodiment of the present invention will be described below with reference to Fig. 17 and Fig. 18. An angle information encoding device 1700 shown in Fig. 17 includes an angle model storage unit 1701, an angle calculation unit 1702, an azimuth angle / elevation angle difference calculation unit 1103, and an arithmetic encoding unit 1104. In the second embodiment, it is assumed that input point cloud data includes distance, azimuth angle, elevation angle, and scan time.

[0088] The angle model storage unit 1701 has information on the displacement of the azimuth and elevation directions of the scan with respect to time based on the design of the LiDAR. The displacement information may be, for example, information indicating the displacement of the azimuth and elevation directions per predetermined time. The displacement information may also be, for example, information indicating the displacement of the azimuth and elevation directions that changes nonlinearly with respect to a predetermined time.

[0089] The angle calculation unit 1702 outputs azimuth angle and elevation angle information calculated from the scan time of the input point cloud data and the displacement information per unit time of the azimuth angle and elevation angle stored in the angle model storage unit 1701 to the azimuth angle / elevation angle difference calculation unit 1103.

[0090] The arithmetic coding unit 1703 obtains the difference value of the azimuth angle and the elevation angle output from the azimuth angle / elevation angle difference calculation unit 1103 and the scan time of the point angle information 802, and performs arithmetic coding. The obtained result is output as a bit stream 804 of the point angle information.

[0091] An angle information decoding device 1800 shown in FIG. 18 includes an arithmetic decoding unit 1801, an angle model storage unit 1802, an angle calculation unit 1803, and the azimuth angle / elevation angle calculation unit 1204.

[0092] The arithmetic decoding unit 1801 arithmetically decodes the bit stream 804 of the angle information of the points output by the angle information encoding device 1700, outputs the scan time to the angle calculation unit 1803, and outputs the azimuth angle and elevation angle difference values ​​to the azimuth angle / elevation angle calculation unit 1204.

[0093] The angle model storage unit 1802 has information on the displacement of the azimuth and elevation directions of the scan based on the design of the LiDAR with respect to time. The displacement information may be, for example, information indicating the displacement of the azimuth and elevation directions per predetermined time. The displacement information may also be, for example, information indicating the displacement of the azimuth and elevation directions that changes nonlinearly with respect to a predetermined time.

[0094] The angle calculation unit 1803 outputs azimuth angle and elevation angle information calculated from the scan time of the angle information bit stream of the point and the displacement information per unit time of the azimuth angle and elevation angle stored in the angle model storage unit 1802 to the azimuth angle / elevation angle calculation unit 1204.

[0095] Fig. 19 is a flowchart showing a processing example of the angle calculation unit 1702 according to the second embodiment of the present invention, and a processing example of the angle calculation unit 1702 of the angle information encoding device 1700 will be described with reference to Fig. 19. Note that the operation of each step of the flowchart in Fig. 19 is sequentially performed by a CPU or the like as a computer (not shown) in the point group encoding device 800 executing a computer program stored in a memory.

[0096] In step S1901, angle model data is obtained from the angle model storage unit 1701. Next, in step S1902, scan time data is obtained from the angle information of the input point. In step S1903, the scan time is input to the angle model, and the returned azimuth angle and elevation angle values ​​are output, and the process ends.

[0097] Fig. 20 is a flowchart showing a processing example of the angle calculation unit 1803 according to the second embodiment of the present invention, and a processing example of the angle calculation unit 1803 of the angle information decoding device 1800 will be described with reference to Fig. 20. Note that the operation of each step of the flowchart in Fig. 20 is sequentially performed by a CPU or the like as a computer (not shown) in the point group decoding device 900 executing a computer program stored in a memory.

[0098] In step S2001, angle model data is obtained from angle model storage unit 1802. Next, in step S2002, scan time is obtained from point angle information bit stream 804. In step S2003, the scan time is input to the angle model, and the obtained azimuth angle and elevation angle values ​​are output, and the process ends.

[0099] In this way, according to the second embodiment, the angle information encoding device 1700 can generate a bit stream of angle information of points while maintaining compression efficiency, regardless of the complexity of the scan path, by utilizing the angle model stored in the angle model storage unit 1701.

[0100] The aacc angle information decoding device 1800 can decode the bit stream of angle information of points created by the angle information encoding device 1700 by using the angle model stored in the angle model storage unit 1802. In addition, it becomes possible to handle point group encoded data with high compression efficiency.

[0101] Example 3 A third embodiment of the present invention will be described below. The LiDAR scan is performed according to a value determined in advance by a design value, but the scan path may change from the design value due to the temperature inside the device or aging of the drive system such as MEMS.

[0102] When the above situation occurs, the difference between the angle information in the angle model storage unit 1101 of the first embodiment and the measured angle value becomes large, and the encoding efficiency decreases.

[0103] Fig. 21 is a functional block diagram showing a schematic configuration example of an angle information encoding device 2100 according to a third embodiment of the present invention, and Fig. 22 is a functional block diagram showing a schematic configuration example of an angle information decoding device 2200 according to the third embodiment of the present invention. In the third embodiment, the input point cloud data has information on distance, azimuth angle, elevation angle, and scan ID.

[0104] The angle information encoding device 2100 shown in FIG. 21 includes an angle model storage unit 2101, an angle calculation unit 1102, an azimuth angle / elevation angle difference calculation unit 1103, an arithmetic encoding unit 1104, and an angle model update unit 2102.

[0105] The angle model storage unit 2101 stores the angle model 1000 based on the above-mentioned design values ​​of the LiDAR. Furthermore, when there is an input from the angle model update unit 2102, the angle model storage unit 2101 updates the target information of the angle model 1000.

[0106] The angle model update unit 2102 refers to the difference value obtained from the azimuth angle / elevation angle difference calculation unit 1103, and updates the target information in the angle model storage unit 2101 if the update judgment condition is satisfied. Here, the angle model update unit 2102 functions as a correction means for correcting the measurement model information based on the difference between the measurement model information and the angle of the measurement point. The update judgment process will be described later.

[0107] An angle information decoding device 2200 shown in FIG. 22 includes an arithmetic decoding unit 1201 , an angle model storage unit 2201 , an angle calculation unit 1203 , an azimuth angle / elevation angle calculation unit 1204 , and an angle model update unit 2202 .

[0108] The angle model storage unit 2201 stores the above-mentioned angle model 1000. Furthermore, when there is an input from the angle model update unit 2202, the angle model storage unit 2201 updates the target information of the angle model.

[0109] The angle model update unit 2202 refers to the azimuth angle and elevation angle values ​​obtained from the azimuth angle / elevation angle calculation unit 1204 and the azimuth angle and elevation angle values ​​stored in the angle model storage unit 2201, and updates the target information in the angle model storage unit 2201 if an update judgment condition is satisfied. Here, the angle model update unit 2202 functions as a correction means for correcting the measurement model information based on the difference between the measurement model information and the angle of the measurement point. Note that the update judgment will be described later.

[0110] Fig. 23 is a flowchart showing a processing example of the angle model update unit 2102 according to the third embodiment of the present invention, and the processing example of the angle model update unit 2102 will be described with reference to Fig. 23. Note that the operation of each step of the flowchart in Fig. 23 is sequentially performed by a CPU or the like as a computer (not shown) in the point cloud encoding device 800 executing a computer program stored in a memory.

[0111] In step S2301, the difference between the predicted and measured azimuth and elevation angles obtained from the azimuth angle / elevation angle difference calculation unit is acquired. In step S2302, it is determined whether the acquired difference value is equal to or greater than a predetermined threshold value. If it is determined in step S2302 that the difference value is less than the predetermined threshold value, the process in FIG. 23 is terminated.

[0112] If it is determined in step S2302 that the acquired difference value is equal to or greater than the predetermined threshold value, the process proceeds to step S2303. Then, the azimuth angle and elevation angle of the corresponding scan ID stored in the angle model storage unit are corrected according to the magnitude of the difference value, and the process in FIG. 23 is terminated.

[0113] In step S2303, the azimuth angle and elevation angle values ​​corresponding to the angle model storage unit are corrected by adding the difference value to them. The next time point cloud data of the same scan ID is input, the azimuth angle / elevation angle difference calculation unit makes a prediction using the corrected value.

[0114] Fig. 24 is a flowchart showing a processing example of the angle model update unit 2202 according to the third embodiment of the present invention, and the processing example of the angle model update unit 2202 will be described with reference to Fig. 24. Note that the operation of each step of the flowchart in Fig. 24 is sequentially performed by a CPU or the like as a computer (not shown) in the point cloud decoding device 900 executing a computer program stored in a memory.

[0115] In step S2401, the decoded azimuth and elevation angle values ​​obtained from the azimuth angle / elevation angle calculation unit and the azimuth and elevation angle stored in the angle model storage unit are obtained and a difference is calculated. In step S2402, it is determined whether the obtained difference value is equal to or greater than a predetermined threshold value.

[0116] If it is determined in step S2402 that the difference value is less than the predetermined threshold, the process in Fig. 24 is terminated. If it is determined in step S2402 that the difference value is equal to or greater than the predetermined threshold, the process proceeds to step S2403, where the azimuth angle and elevation angle of the scan ID stored in the angle model storage unit are corrected according to the magnitude of the difference value, and the process in Fig. 24 is terminated.

[0117] In step S2403, the azimuth angle and elevation angle values ​​corresponding to the angle model storage unit 2201 are corrected by adding half the difference value to them. Furthermore, the next time decoded data of the same scan ID is input, the azimuth angle and elevation angle prediction unit makes predictions using the corrected values.

[0118] In steps S2302 and S2402, the update determination is performed using the same predetermined threshold value, and in steps S2303 and S2403, the azimuth and elevation angle values ​​are corrected using the same correction procedure.

[0119] As described above, according to the third embodiment, the angle models in the angle model storage unit 2101 of the angle information encoding device 2100 and the angle model storage unit 2201 of the angle information decoding device 2200 are synchronously changed. Therefore, even if the scan path changes due to the influence of temperature or the like, it is possible to obtain encoded data with high compression efficiency.

[0120] In the third embodiment, for example, a temperature sensor may be provided, and the angle model storage unit may store a table or a function including temperature as a parameter, so that the angle model is changed according to the temperature detected by the temperature sensor.

[0121] In addition, in the first to third embodiments, an example of a table as shown in FIG. 10 has been described as the angle model, but the angle model may be a function, and the angle model storage unit may store the function.

[0122] In addition, for example, in the above-mentioned first to third embodiments, encoding is performed using an angle model, but for example, encoding may be performed by combining the encoding shown in Fig. 2 with the angle model. Also, decoding may be performed by combining the decoding shown in Fig. 6 with the angle model.

[0123] Although the present invention has been described in detail based on the preferred embodiments, the present invention is not limited to the above embodiments, and various modifications are possible based on the spirit of the present invention, and are not excluded from the scope of the present invention. The above embodiments include the following combinations.

[0124] (Configuration 1) An encoding device comprising: an acquisition means for acquiring point cloud data consisting of a plurality of measurement points from a measurement means; a calculation means for calculating predicted points corresponding to the plurality of measurement points based on measurement model information regarding a measurement pattern of the measurement means; a calculation means for calculating a difference between the measurement point and the predicted point corresponding to the measurement point; and an encoding means for encoding the difference.

[0125] (Configuration 2) The encoding device according to configuration 1, wherein the acquisition means acquires the point cloud data consisting of the plurality of measurement points generated by the measurement means performing a scan using a laser.

[0126] (Configuration 3) The encoding device described in Configuration 2, characterized in that the measurement model information includes information regarding a plurality of points managed by IDs based on the laser irradiation position or laser irradiation timing during scanning by the measurement means.

[0127] (Configuration 4) The encoding device according to configuration 2 or 3, characterized in that the information on the plurality of measurement points includes angle information regarding a laser irradiation angle when the measurement means measures, and the measurement model information includes angle model information regarding a laser irradiation angle when the measurement means scans.

[0128] (Configuration 5) 5. The encoding device according to configuration 4, wherein the angle information includes an azimuth angle and an elevation angle of a laser when the measurement means measures the image, and the angle model information includes an azimuth angle model and an elevation angle model of a laser when the measurement means measures the image.

[0129] (Configuration 6) An encoding device described in any one of configurations 2 to 5, characterized in that the information on the multiple measurement points includes irradiation time information indicating the timing of laser irradiation when the measurement means measures, and the measurement model information includes irradiation time model information indicating the timing of laser irradiation when the measurement means scans.

[0130] (Configuration 7) The encoding device according to any one of configurations 1 to 6, further comprising a correction means for correcting the measurement model information based on a difference between the measurement model information and the angle of the measurement point.

[0131] (Configuration 8) A decoding device characterized by having measurement model information regarding a measurement pattern of a measurement means, and comprising a decoding means for decodes information of measurement points based on a comparison result between the measurement model information and angle information included in the encoded data.

[0132] (Configuration 9) The decoding device according to configuration 8, further comprising an acquisition means for acquiring point cloud data consisting of the plurality of measurement points generated by the measurement means performing a scan using a laser.

[0133] (Configuration 10) The decoding device described in Configuration 9, characterized in that the information on the multiple measurement points includes angle information regarding a laser irradiation angle when the measurement means measures, and the measurement model information includes angle model information regarding a laser irradiation angle when the measurement means scans.

[0134] (Configuration 11) The decoding device described in Configuration 10, characterized in that the angle information includes an azimuth angle and an elevation angle of a laser at the time of measurement by the measurement means, and the angle model information includes an azimuth angle model and an elevation angle model of a laser at the time of scanning by the measurement means.

[0135] (Configuration 12) A decoding device described in any one of configurations 9 to 11, characterized in that the information on the multiple measurement points includes irradiation time information indicating the timing of laser irradiation when the measurement means measures, and the measurement model information includes irradiation time model information indicating the timing of laser irradiation when the measurement means scans.

[0136] (Configuration 13) The decoding device according to any one of configurations 8 to 12, further comprising a correcting means for correcting the measurement model information based on a difference between the measurement model information and the angle of the measurement point.

[0137] (Program) A computer program for controlling each means of the encoding device according to any one of configurations 1 to 7 or the decoding device according to any one of configurations 8 to 13 by a computer.

[0138] In order to realize a part or all of the control in the above-mentioned embodiment, a computer program for realizing the functions of the above-mentioned embodiment may be supplied to an encoding device or the like via a network or various storage media. Then, a computer (or a CPU, MPU, or the like) in the encoding or the like may read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention. [Explanation of symbols]

[0139] 100: LiDAR 800: Point cloud encoding device 900: Point cloud decoding device

Claims

1. An acquisition means for acquiring point cloud data consisting of a plurality of measurement points from a measurement means; a calculation means for calculating predicted points corresponding to the plurality of measurement points based on measurement model information related to a measurement pattern of the measurement means; A calculation means for calculating a difference between the measurement point and the prediction point corresponding to the measurement point; encoding means for encoding the difference; An encoding device comprising:

2. 2. The encoding device according to claim 1, wherein the acquisition means acquires the point cloud data made up of the plurality of measurement points generated by the measurement means performing a scan using laser irradiation.

3. 3. The encoding device according to claim 2, wherein the measurement model information includes information regarding a plurality of points managed by IDs based on the laser irradiation position or laser irradiation timing during scanning by the measurement means.

4. the information on the plurality of measurement points includes angle information on a laser irradiation angle at the time of measurement by the measuring means, 3. The encoding device according to claim 2, wherein the measurement model information includes angle model information relating to a laser irradiation angle during scanning by the measuring means.

5. the angle information includes an azimuth angle and an elevation angle of a laser at the time of measurement by the measurement means, 5. The encoding device according to claim 4, wherein the angle model information includes an azimuth angle model and an elevation angle model of a laser when the measuring means scans.

6. the information on the plurality of measurement points includes irradiation time information indicating a timing of laser irradiation during measurement by the measuring means, 3. The encoding device according to claim 2, wherein the measurement model information includes irradiation time model information indicating a timing of laser irradiation during scanning by the measuring means.

7. 2. The encoding device according to claim 1, further comprising a correction unit that corrects the measurement model information based on a difference between the measurement model information and an angle of the measurement point.

8. A decoding device comprising: a decoding means for decoding information of measurement points based on a comparison result between the measurement model information and angle information included in encoded data, the measurement model information having measurement model information relating to a measurement pattern of a measurement means.

9. 9. The decoding device according to claim 8, further comprising an acquisition unit that acquires point cloud data made up of the plurality of measurement points generated by the measurement unit performing a scan using a laser.

10. the information on the plurality of measurement points includes angle information on a laser irradiation angle at the time of measurement by the measuring means, 10. The decoding device according to claim 9, wherein the measurement model information includes angle model information relating to a laser irradiation angle during scanning by the measuring means.

11. the angle information includes an azimuth angle and an elevation angle of a laser at the time of measurement by the measurement means, 11. The decoding device according to claim 10, wherein the angle model information includes an azimuth angle model and an elevation angle model of a laser when the measuring means scans.

12. the information on the plurality of measurement points includes irradiation time information indicating a timing of laser irradiation during measurement by the measuring means, 10. The decoding device according to claim 9, wherein the measurement model information includes irradiation time model information indicating a timing of laser irradiation during scanning by the measuring means.

13. 9. The decoding device according to claim 8, further comprising a correction unit that corrects the measurement model information based on a difference between the measurement model information and the angle of the measurement point.

14. A computer program for controlling each means of the encoding device according to any one of claims 1 to 7 or the decoding device according to any one of claims 8 to 13 by a computer.

Citation Information

Patent Citations

  • Data compression apparatus, data decompressor, control method, program and storage medium

    JP2018116452A

Cited By

  • Data processing method, data processing device, and program

    WO2026197233A1