Point cloud data coupling device, point cloud data coupling system, point cloud data coupling method, and point cloud data coupling program
By employing low-density point cloud data as a framework to align high-density data sets through feature extraction and conditional alignment, the method addresses alignment errors in large structure modeling, resulting in precise and consistent three-dimensional models.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- MITSUBISHI ELECTRIC CORP
- Filing Date
- 2025-03-21
- Publication Date
- 2026-06-18
AI Technical Summary
Existing methods for generating three-dimensional models of large structures using high-density point cloud data suffer from accumulating alignment errors, leading to significant deviations in the final model due to the large number of data sets that need to be aligned.
Utilizing low-density point cloud data as a framework to assist in aligning high-density point cloud data sets by extracting feature points, setting a reference, determining a range of motion, and conditionally aligning adjacent data within this range to minimize errors.
This approach allows for the generation of highly accurate three-dimensional models with reduced overall distortion and fewer required measurements, maintaining consistency across the entire structure.
Smart Images
Figure JP2025010948_18062026_PF_FP_ABST
Abstract
Description
Point cloud data combining device, point cloud data combining system, point cloud data combining method, and point cloud data combining program
[0001] The present disclosure relates to a point cloud data combining device, a point cloud data combining system, a point cloud data combining method, and a point cloud data combining program.
[0002] In recent years, digital twin technology has been used as a simulation technology based on three-dimensional models of structures such as buildings or facilities. In order to apply this digital twin technology to existing buildings such as buildings without drawings, a technology for efficiently creating a detailed three-dimensional model of the building is required. The three-dimensional model of a building is called BIM. BIM is an abbreviation for Building Information Model.
[0003] Patent Document 1 discloses a technique for accurately generating a three-dimensional model of a staircase from high-density point cloud data measured at multiple positions. In the technique of Patent Document 1, high-density point cloud data is measured at multiple positions using a fixed-type LiDAR sensor, and features such as the ceiling or rails of the staircase are extracted from the high-density point cloud data. Then, in the technique of Patent Document 1, alignment of the high-density point cloud data is performed based on the extracted features.
[0004] Japanese Patent Application Laid-Open No. 2018-173392
[0005] In the technique of Patent Document 1, a three-dimensional model of the entire structure is generated by integrating, that is, aligning, a plurality of high-density point cloud data. In this case, the larger the structure, the greater the number of high-density point cloud data to be aligned. Therefore, there is a problem that the error in aligning the point cloud data accumulates as the structure becomes larger, resulting in a large deviation as a whole.
[0006] In the present disclosure, low-density point cloud data that can measure the entire structure at once is used as a framework to assist in aligning a group of high-density point cloud data. The purpose is to generate a three-dimensional model of the structure with high accuracy while maintaining the consistency of the entire structure.
[0007] The point cloud data merging device according to this disclosure includes: a feature extraction unit that acquires a plurality of first point cloud data, each representing a part of a structure, and second point cloud data representing the structure, wherein the second point cloud data has a lower density than each of the plurality of first point cloud data, and extracts a first feature point, which is a feature point of each of the plurality of first point cloud data, and a second feature point, which is a feature point of the second point cloud data; an initial alignment unit that performs alignment of each of the plurality of first point cloud data and the second point cloud data based on the first feature point and the second feature point; a reference setting unit that sets a reference point cloud data from the plurality of first point cloud data; a range of motion determination unit that selects first point cloud data adjacent to the reference point cloud data as adjacent point cloud data, and determines the range of motion of the adjacent point cloud data based on the difference between the first feature point and the second feature point of the adjacent point cloud data; The system includes a conditional alignment unit that determines the position of the adjacent point cloud data by aligning the adjacent point cloud data with the reference point cloud data, provided that the position of the adjacent point cloud data is within the range of motion, and then combines the adjacent point cloud data with the reference point cloud data.
[0008] The point cloud data merging device described herein uses a low-density second point cloud data set, capable of measuring the entire structure at once, as a framework to assist in the alignment of multiple high-density first point cloud data sets representing the structure. Therefore, the point cloud data merging device described herein can generate a highly accurate three-dimensional model of a structure while maintaining the consistency of the entire structure.
[0009] A diagram showing an example configuration of the point cloud data merging device according to Embodiment 1. A flowchart showing an example of operation of the point cloud data merging device according to Embodiment 1. A schematic diagram illustrating an example of first point cloud data according to Embodiment 1. A schematic diagram illustrating an example of first point cloud data and second point cloud data according to Embodiment 1. A schematic diagram showing each processing example of the point cloud data merging process according to Embodiment 1. A diagram showing an example configuration of the point cloud data merging device according to a modified example of Embodiment 1. A diagram showing the effect of the point cloud data merging device according to Embodiment 1. A diagram showing an example configuration of the point cloud data merging device according to Embodiment 2. A flowchart showing an example of operation of the point cloud data merging device according to Embodiment 2.
[0010] The following description of this embodiment will be illustrated with reference to the figures. In each figure, identical or corresponding parts are denoted by the same reference numerals. In the description of the embodiment, descriptions of identical or corresponding parts will be omitted or simplified as appropriate. The arrows in the figures mainly indicate the flow of data or processing. Also, the size relationships of the components in the following figures may differ from those in reality. In addition, in the description of the embodiment, directions or positions such as up, down, left, right, front, back, front, and back may be indicated. These notations are for the convenience of explanation and do not limit the arrangement, direction, and orientation of the devices, instruments, or parts.
[0011] Embodiment 1. ***Description of Configuration*** Figure 1 shows an example of the configuration of a point cloud data merging device 100 according to this embodiment. The point cloud data merging device 100 is a computer. The point cloud data merging device 100 includes a processor 910, as well as other hardware such as a memory 921, an auxiliary storage device 922, an input interface 930, an output interface 940, and a communication device 950. The processor 910 is connected to the other hardware via signal lines and controls this other hardware.
[0012] The point cloud data merging device 100 includes, as functional elements, a feature extraction unit 110, an initial alignment unit 120, a reference setting unit 130, a range of motion determination unit 140, a condition alignment unit 150, a model generation unit 160, and a storage unit 170. The storage unit 170 stores information such as the first point cloud data 71, the second point cloud data 72, the first feature point 711, the second feature point 721, and the range of motion 80. The storage unit 170 also stores multiple first point cloud data 71.
[0013] The functions of the feature extraction unit 110, the initial alignment unit 120, the reference setting unit 130, the range of motion determination unit 140, the condition alignment unit 150, and the model generation unit 160 are implemented by software. The storage unit 170 is provided in the memory 921. The storage unit 170 may also be provided in the auxiliary storage device 922, or it may be distributed between the memory 921 and the auxiliary storage device 922.
[0014] The processor 910 is a device that executes a point cloud data merging program. The point cloud data merging program is a program that realizes the functions of the feature extraction unit 110, the initial alignment unit 120, the reference setting unit 130, the range of motion determination unit 140, the condition alignment unit 150, and the model generation unit 160. The processor 910 is an IC that performs arithmetic processing. Specific examples of the processor 910 are a CPU, DSP, and GPU. IC is an abbreviation for Integrated Circuit. CPU is an abbreviation for Central Processing Unit. DSP is an abbreviation for Digital Signal Processor. GPU is an abbreviation for Graphics Processing Unit.
[0015] Memory 921 is a storage device that temporarily stores data. Specific examples of memory 921 include SRAM or DRAM. SRAM is an abbreviation for Static Random Access Memory. DRAM is an abbreviation for Dynamic Random Access Memory. Auxiliary storage device 922 is a storage device that stores data. Specific examples of auxiliary storage device 922 include HDD. Alternatively, auxiliary storage device 922 may be a portable storage medium such as an SD® memory card, CF, NAND flash, flexible disk, optical disk, compact disk, Blu-ray® disc, or DVD. HDD is an abbreviation for Hard Disk Drive. SD® is an abbreviation for Secure Digital. CF is an abbreviation for CompactFlash (registered trademark). DVD is an abbreviation for Digital Versatile Disk.
[0016] The input interface 930 is a port connected to an input device such as a mouse, keyboard, or touch panel. Specifically, the input interface 930 is a USB terminal. The input interface 930 may also be a port connected to a LAN. USB is an abbreviation for Universal Serial Bus. LAN is an abbreviation for Local Area Network. The input interface 930 may also be connected to a fixed LiDER 210 and a portable LiDER 220. The fixed LiDER 210 is also called a fixed LiDER sensor. The portable LiDER 220 is also called a portable LiDER sensor. The input interface 930 may also serve as a sensor interface for acquiring multiple first point cloud data 71 and second point cloud data 72 acquired by the fixed LiDER 210 and the portable LiDER 220.
[0017] The output interface 940 is a port to which the cable of an output device, such as a display, is connected. Specifically, the output interface 940 is a USB terminal or an HDMI® terminal. Specifically, the display is an LCD. The output interface 940 is also called the display interface. HDMI® is an abbreviation for High Definition Multimedia Interface. LCD is an abbreviation for Liquid Crystal Display.
[0018] The communication device 950 has a receiver and a transmitter. The communication device 950 is connected to a communication network such as a LAN, the Internet, a telephone line, or Wi-Fi (registered trademark). Specifically, the communication device 950 is a communication chip or NIC. The communication device 950 may also serve as a communication interface for acquiring multiple first point cloud data 71 and second point cloud data 72 acquired by a fixed LiDER 210 and a portable LiDER 220 wirelessly or via a wired connection. NIC is an abbreviation for Network Interface Card.
[0019] The point cloud data merging program is executed in the point cloud data merging device 100. The point cloud data merging program is loaded into the processor 910 and executed by the processor 910. Memory 921 stores not only the point cloud data merging program but also the OS. OS is an abbreviation for Operating System. The processor 910 executes the point cloud data merging program while executing the OS. The point cloud data merging program and the OS may also be stored in the auxiliary storage device 922. The point cloud data merging program and OS stored in the auxiliary storage device 922 are loaded into memory 921 and executed by the processor 910. Note that part or all of the point cloud data merging program may be incorporated into the OS.
[0020] The point cloud data merging device 100 may include multiple processors that replace the processor 910. These multiple processors share the task of executing the point cloud data merging program. Each processor is a device that executes the point cloud data merging program in the same way as the processor 910.
[0021] The data, information, signal values, and variable values used, processed, or output by the point cloud data merging program are stored in memory 921, auxiliary storage device 922, or registers or cache memory within the processor 910.
[0022] The term "part" in each of the feature extraction unit 110, initial alignment unit 120, reference setting unit 130, range of motion determination unit 140, condition alignment unit 150, and model generation unit 160 may be read as "circuit," "process," "procedure," "process," or "circuitry." The point cloud data merging program causes a computer to execute each of the following processes: feature extraction, initial alignment, reference setting, range of motion determination, condition alignment, and model generation. The term "process" in each of the above processes may be read as "program," "program product," "computer-readable storage medium storing the program," or "computer-readable recording medium recording the program." The point cloud data merging method is performed by the point cloud data merging device 100 executing the point cloud data merging program. The point cloud data merging program may be provided stored on a computer-readable recording medium. The point cloud data merging program may also be provided as a program product.
[0023] ***Explanation of Operation*** Next, the operation of the point cloud data merging device 100 according to this embodiment will be explained. The operation procedure of the point cloud data merging device 100 corresponds to the point cloud data merging method. Furthermore, the program that realizes the point cloud data merging process, which is the operation of the point cloud data merging device 100, corresponds to the point cloud data merging program.
[0024] Figure 2 is a flowchart showing an example of the operation of the point cloud data merging device 100 according to this embodiment.
[0025] <Feature extraction process: Steps S101, S102> In the feature extraction process, the feature extraction unit 110 acquires a plurality of first point cloud data 71, each representing a part of the structure 300, and second point cloud data 72, which represents the structure 300 and has a lower density than each of the plurality of first point cloud data 71. The feature extraction unit 110 extracts first feature points 711, which are feature points of each of the plurality of first point cloud data 71, and second feature points 721, which are feature points of the second point cloud data 72. Thus, the feature extraction process comprises a point cloud data acquisition process that acquires a plurality of first point cloud data 71 and second point cloud data 72, and a data feature point extraction process that extracts feature points from the plurality of first point cloud data 71 and second point cloud data 72.
[0026] <<Point Cloud Data Acquisition Process: Step S101>> In step S101, the feature extraction unit 110 acquires a plurality of first point cloud data 71, each representing a part of the structure 300, and a second point cloud data 72 representing the structure 300. The second point cloud data 72 is a point cloud data with a lower density than each of the plurality of first point cloud data 71. The first point cloud data 71 may be referred to as high-density point cloud data. The second point cloud data 72 may be referred to as low-density point cloud data.
[0027] Figure 3 is a schematic diagram illustrating an example of the first point cloud data 71 according to this embodiment. The first point cloud data 71 is obtained by measuring the structure 300 from multiple locations. The upper left figure of Figure 3 shows an example of an actual drawing of the structure 300. The pattern area in the upper right figure of Figure 3 represents the area represented by each individual first point cloud data 71. The solid lines indicate edges detected by each individual first point cloud data 71. The lower figure of Figure 3 shows the result of alignment of the first point cloud data 71. The dark pattern area in the lower figure of Figure 3 indicates the overlapping portion of the first point cloud data 71. The overlapping portion is the overlapping part of two point cloud data taken from the same object from different positions and angles. In alignment, for example, the data is merged so that this overlapping portion overlaps neatly. Generally, the wider and more complex the shape of the overlapping portion, the easier the alignment. If the overlapping portion has a simple shape, the error tends to be larger. Furthermore, in the lower diagram of Figure 3, if, for example, the first point cloud data is aligned from left to right, the accumulation of errors may increase as you move to the right.
[0028] As shown in the upper right diagram of Figure 3, the structure 300 is represented by multiple first point cloud data 71. Note that the structure 300 may include not only buildings such as office buildings or houses, but also equipment such as elevators, or infrastructure such as bridges.
[0029] Specifically, the first point cloud data 71 is measured by a fixed LiDAR 210. The fixed LiDAR 210 is mounted on a fixed structure such as a tripod and acquires data from its surroundings. Measurement with a fixed LiDAR 210 is highly accurate with high density, i.e., high resolution, but it is also expensive. Furthermore, because measurements with a fixed LiDAR 210 are taken from a fixed position, there is a high possibility of blind spots due to irregularities in the target shape. Therefore, in order to construct a 3D model of the structure 300 using measurements with a fixed LiDAR 210, multiple measurements are required by changing the position and angle to fill in the blind spots. Then, the point cloud data obtained from these multiple measurements needs to be integrated, i.e., aligned. For example, it may be necessary to take about three measurements for one room, and in total, more than 100 measurements may be required.
[0030] Figure 4 is a schematic diagram illustrating an example of the first point cloud data 71 and the second point cloud data 72 according to this embodiment. The second point cloud data 72 is point cloud data obtained by measuring the structure 300 while it is moving. Therefore, the second point cloud data 72 is acquired as point cloud data representing the entire structure 300 in a single measurement.
[0031] Specifically, the second point cloud data 72 is measured using a portable LiDAR 220. The portable LiDAR 220 is, for example, a LiDAR sensor mounted on a smartphone or tablet device. While the portable LiDAR 220 is low-cost, its measurement density is low, meaning it has low resolution and lower accuracy compared to a fixed type. On the other hand, because measurements using the portable LiDAR 220 can be taken while moving, it is possible to measure the entire target in a single measurement, filling in blind spots. Furthermore, with measurements using the portable LiDAR 220, there is no need to integrate point cloud data, i.e., align it, when constructing a 3D model of the structure 300.
[0032] The upper part of Figure 4 shows an example of an actual drawing of the structure 300. The middle part of Figure 4 shows an example of an edge obtained as a result of alignment using only multiple first point cloud data 71, with solid lines. As explained in Figure 3, the accumulation of errors increases to the right, showing distortion. The faint patterned area in the lower part of Figure 4 is an example of an area represented by the second point cloud data 72. The second point cloud data 72 is low-density point cloud data obtained by measuring the structure 300 in a single measurement. The second point cloud data 72 shows that it is coarse locally, but the overall distortion is small.
[0033] <<Data Feature Point Extraction Process: Step S102>> In step S102, the feature extraction unit 110 extracts first feature points 711, which are feature points of each of the multiple first point cloud data 71, and second feature points 721, which are feature points of the second point cloud data 72. Specifically, the feature extraction unit 110 extracts features such as lines, surfaces, and angles as first feature points 711 for each of the multiple first point cloud data 71. The feature extraction unit 110 also extracts features such as lines, surfaces, and angles as second feature points 721 for the second point cloud data 72.
[0034] <Initial alignment process: Step S103> In step S103, the initial alignment unit 120 aligns each of the first point cloud data 71 with the second point cloud data 72 based on the first feature point 711 and the second feature point 721.
[0035] Figure 5 is a schematic diagram showing examples of each process in the point cloud data merging process according to this embodiment. The upper part of Figure 5 shows the initial alignment process. The initial alignment unit 120 compares the first feature point 711 with the second feature point. Based on the comparison between the first feature point 711 and the second feature point, the initial alignment unit 120 aligns each of the multiple first point cloud data 71 with the second point cloud data 72.
[0036] Specific examples of alignment methods are as follows. For example, one method for aligning point clouds is ICP, which repeatedly updates coordinates by rotation or translation based on the distance between points in the point cloud. ICP is an abbreviation for Iterative Closest Point. There is also a method called feature-based alignment. In feature-based alignment, alignment is performed indirectly using the shape of characteristic points, lines, or surfaces (features), such as the columns or edges of roofs of buildings.
[0037] During the initial alignment process, each of the multiple first point cloud data 71 is placed in the second point cloud data 72. In other words, during the initial alignment process, each of the multiple first point cloud data 71 is aligned in the same coordinate system of a single second point cloud data 72.
[0038] <Reference Setting Process: Step S104> In step S104, the reference setting unit 130 sets a reference point cloud data 723 from a plurality of first point cloud data 71 that have been aligned with the second point cloud data 72. The reference point cloud data 723 is the point cloud data that is first fixed as a reference when aligning a plurality of first point cloud data 71 that are arranged in the same coordinate system as the second point cloud data 72. Specifically, it is as follows.
[0039] The reference setting unit 130 sets the first point cloud data 71 from among a plurality of first point cloud data 71 that minimizes the difference between the first feature point 711 and the second feature point 721 as the reference point cloud data 723. For example, when determining the reference point cloud data 723 for the first time, the reference setting unit 130 selects the one that has the most usable feature points for alignment between the first point cloud data and the highest alignment accuracy with the second point cloud data 72 at those feature points. As the alignment accuracy, NRMSD, i.e., normalized root mean squared error, is used. The reference setting unit 130 sets the first point cloud data 71 that minimizes the amount of deviation between corresponding feature points, such as lines or surfaces, between the first feature point 711 and the second feature point 721, i.e., the difference amount, as the reference point cloud data 723.
[0040] Alternatively, the reference setting unit 130 may set a first point cloud data 71 containing a feature whose position is known in advance as the reference point cloud data 723. For example, a feature whose dimension is known may be placed in advance, and the reference setting unit 130 may set the first point cloud data 71 containing that feature as the reference point cloud data 723.
[0041] <Movement Range Determination Process: Step S105> In step S105, the movement range determination unit 140 selects the first point cloud data 71 adjacent to the reference point cloud data 723 as the adjacent point cloud data 724. Then, the movement range determination unit 140 determines the movement range of the adjacent point cloud data 724 based on the difference between the first feature point 711 and the second feature point 721 of the adjacent point cloud data 724. Specifically, it is as follows.
[0042] The range of motion determination unit 140 determines the range of motion 80 as the range in which the difference between the first feature point 711 and the second feature point 721 of the adjacent point cloud data 724 does not exceed a threshold. The range of motion determination unit 140 sets a threshold for the difference and limits the distance that can be moved in each direction in the adjacent point cloud data 724, so that the adjacent point cloud data 724 does not deviate too far from the second point cloud data 72, which is low-density point cloud data. More specifically, it is as follows.
[0043] The middle section of Figure 5 shows the range of motion determination process for determining the range of motion 80 of the adjacent point cloud data 724. (1) The range of motion determination unit 140 selects one adjacent point cloud data 724 that is adjacent to the reference point cloud data 723 and has an overlap portion with the reference point cloud data 723. (2) The range of motion determination unit 140 records the change in the amount of difference when the adjacent point cloud data 724, which is high-density data, is shifted by a certain distance in any direction between the adjacent point cloud data 724 and the second point cloud data 72. This allows, for example, to determine which directions have a large change in the amount of difference and which directions have a small change. (3) The threshold for the amount of difference is set in advance. The range of motion determination unit 140 sets the range of motion 80 of the adjacent point cloud data 724 to the point where the threshold is not exceeded in each direction.
[0044] <Condition Alignment Process: Step S106, Step S107>In step S106, the condition alignment unit 150 performs alignment of the adjacent point group data 724 with respect to the reference point group data 723 on the condition that the position of the adjacent point group data 724 is within the range of the movable range 80. By this conditional alignment, the condition alignment unit 150 determines the position of the adjacent point group data 724. At this time, the condition alignment unit 150 determines the position of the adjacent point group data 724 within the range of the movable range 80 such that the alignment error between the adjacent point group data 724 and the reference point group data 723 is minimized.
[0045] In step S107, the condition alignment unit 150 combines the adjacent point group data 724 and the reference point group data 723. Then, the condition alignment unit 150 updates the combined point group data of the adjacent point group data 724 and the reference point group data 723 as new reference point group data 723.
[0046] In step S108, the condition alignment unit 150 determines whether all of the first point group data 71 has been combined as the reference point group data 723. If all of the first point group data 71 has been combined as the reference point group data 723, the process proceeds to step S109. If there is first point group data 71 that has not been combined with the reference point group data 723, the process returns to step S105 to select the next adjacent point group data 724.
[0047] <Model Generation Process: Step S109>When all of the first point group data of the plurality of first point group data 71 has been combined as the reference point group data 723, in step S109, the model generation unit 160 generates a three-dimensional model of the structure 300 based on the reference point group data 723.
[0048] ***Other configurations*** In this embodiment, the functions of the feature extraction unit 110, the initial alignment unit 120, the reference setting unit 130, the movable range determination unit 140, the conditional alignment unit 150, and the model generation unit 160 are realized by software. As a modification, the functions of the feature extraction unit 110, the initial alignment unit 120, the reference setting unit 130, the movable range determination unit 140, the conditional alignment unit 150, and the model generation unit 160 may be realized by hardware. Specifically, the point cloud data combining device 100 includes an electronic circuit 909 instead of the processor 910.
[0049] FIG. 6 is a diagram showing a configuration example of the point cloud data combining device 100 according to a modification of this embodiment. The electronic circuit 909 is a dedicated electronic circuit that realizes the functions of the feature extraction unit 110, the initial alignment unit 120, the reference setting unit 130, the movable range determination unit 140, the conditional alignment unit 150, and the model generation unit 160. Specifically, the electronic circuit 909 is a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, a logic IC, a GA, an ASIC, or an FPGA. GA is an abbreviation for Gate Array. ASIC is an abbreviation for Application Specific Integrated Circuit. FPGA is an abbreviation for Field-Programmable Gate Array.
[0050] The functions of the feature extraction unit 110, the initial alignment unit 120, the reference setting unit 130, the movable range determination unit 140, the conditional alignment unit 150, and the model generation unit 160 may be realized by one electronic circuit or may be realized by being distributed among a plurality of electronic circuits.
[0051] As another modification, some of the functions of the feature extraction unit 110, the initial alignment unit 120, the reference setting unit 130, the movable range determination unit 140, the conditional alignment unit 150, and the model generation unit 160 may be realized by an electronic circuit and the remaining functions may be realized by software. Also, some or all of the functions of the feature extraction unit 110, the initial alignment unit 120, the reference setting unit 130, the movable range determination unit 140, the conditional alignment unit 150, and the model generation unit 160 may be realized by firmware.
[0052] Each of the processor and electronic circuit is also called a processing circuit. In other words, the functions of the feature extraction unit 110, the initial alignment unit 120, the reference setting unit 130, the range of motion determination unit 140, the condition alignment unit 150, and the model generation unit 160 are realized by the processing circuit.
[0053] ***Explanation of the Effects of This Embodiment*** Figure 7 shows the effects of the point cloud data merging device 100 according to this embodiment. Normally, to reduce the accumulation of alignment errors, there were only two options: (A) reduce the number of alignments, that is, acquire high-density point cloud data over a wider area at once, or (B) increase the overlap portion between high-density point cloud data. However, due to the structure of the building or limitations of the measurement range of the LiDAR sensor, (A) is difficult to achieve, and (B) increases the number of measurements, resulting in an increase in both the number of alignments and the accumulation of alignment errors, as shown in the left figure of Figure 7.
[0054] In the point cloud data merging device according to this embodiment, a low-density second point cloud data, free from alignment errors, is used as the framework. This reduces the overall distortion caused by the accumulation of alignment errors between high-density first point cloud data, and allows for the acquisition of highly accurate point cloud data for the entire structure. Therefore, as shown in the right-hand figure of Figure 7, the point cloud data merging device according to this embodiment can generate a highly accurate and precise 3D model of a structure while maintaining the consistency of the entire structure.
[0055] Furthermore, the alignment accuracy of high-density first point cloud data typically depended on the amount of features in the overlapping areas. However, the point cloud data merging device according to this embodiment can maintain the accuracy of the entire structure even when the overlapping areas are narrow and lack distinctive shapes. Therefore, the point cloud data merging device according to this embodiment can reduce the number of measurements required for high-density first point cloud data.
[0056] Embodiment 2. This embodiment mainly describes the differences from Embodiment 1 and the additions to Embodiment 1. In this embodiment, components having the same function as in Embodiment 1 are denoted by the same reference numerals, and their descriptions are omitted.
[0057] ***Configuration Description*** Figure 8 shows an example of the configuration of the point cloud data merging device 100 according to this embodiment. In this embodiment, the configuration described in Embodiment 1 is further enhanced with a distribution calculation unit 180 and a distribution presentation unit 190. The other configurations are the same as in Embodiment 1.
[0058] The distribution calculation unit 180 visualizes the distribution of the second feature points 721 in the second point cloud data 72. The distribution presentation unit 190 presents the visualized distribution of the second feature points 721 to the user by displaying it on an output device via the output interface 940. The feature extraction unit 110 acquires multiple first point cloud data 71, including markers that are placed according to the distribution of the second feature points 721.
[0059] ***Explanation of Operation*** Figure 9 is a flowchart showing an example of the operation of the point cloud data merging device 100 according to this embodiment. The data merging process according to this embodiment includes steps 111 to S114 instead of steps S101 and S102 of Embodiment 1. The processing from step S103 onwards is the same as in Embodiment 1.
[0060] In step S111, the distribution calculation unit 180 acquires the second point cloud data 72 and extracts the second feature points 721 from the second point cloud data 72. The distribution calculation unit 180 then calculates the distribution of the second feature points 721 in the structure 300. The distribution calculation unit 180 visualizes the distribution of the second feature points 721. In step S112, the distribution presentation unit 190 displays the visualized distribution of the second feature points 721 on a display device via the output interface 940 and presents it to the user. Specifically, it is as follows.
[0061] First, the distribution calculation unit 180 extracts recognizable features such as surfaces or lines from the second point cloud data 72 as second feature points 721. Next, the distribution calculation unit 180 divides the entire structure 300 into a three-dimensional grid. The distribution calculation unit 180 measures the amount of second feature points 721 detected within each grid and calculates the feature distribution density at each point of the entire structure 300. Feature distribution density is the amount of features per unit volume. Then, the distribution presentation unit 190 displays the visualized feature distribution density on a display device. For example, the distribution presentation unit 190 highlights areas with low feature distribution density to show the user areas of the structure 300 with few features.
[0062] The user places markers in areas with few displayed features to serve as alignment guides. These markers are, for example, objects whose three-dimensional shape details are known and which can be fixed to the floor or wall.
[0063] Step S113 is performed when it is confirmed by the user that the marker has been installed. In step S113, the feature extraction unit 110 acquires a plurality of first point cloud data 71 obtained after the marker has been installed. In step S114, the feature extraction unit 110 extracts the first feature point 711 from each of the plurality of first point cloud data 71. The subsequent steps are the same as in Embodiment 1.
[0064] ***Explanation of the Effects of This Embodiment*** In addition to the effects of Embodiment 1, the point cloud data merging device according to this embodiment has the effect of further improving the accuracy of alignment between high-density point cloud data.
[0065] In Embodiments 1 and 2 described above, each part of the point cloud data merging device was described as an independent functional block. However, the configuration of the point cloud data merging device does not have to be as described in the embodiments above. The functional blocks of the point cloud data merging device can be configured in any way as long as they can realize the functions described in the embodiments above. Also, the point cloud data merging device does not have to be a single device, but a system composed of multiple devices. Furthermore, multiple parts of Embodiments 1 and 2 may be combined and implemented. Alternatively, only one part of these embodiments may be implemented. In addition, these embodiments may be combined and implemented in any way, either as a whole or in part. That is to say, in Embodiments 1 and 2, it is possible to freely combine each embodiment, modify any component of each embodiment, or omit any component in each embodiment.
[0066] The embodiments described above are essentially preferred examples and are not intended to limit the scope of the Disclosure, the scope of the Applications of the Disclosure, or the scope of Uses of the Disclosure. The embodiments described above can be modified in various ways as needed. For example, the procedures described using flowcharts or sequence diagrams may be modified as appropriate.
[0067] The various aspects of this disclosure are summarized below as appendices. (Appendix 1) A feature extraction unit that acquires a plurality of first point cloud data, each representing a part of a structure, and second point cloud data representing the structure, which has a lower density than each of the plurality of first point cloud data, and extracts a first feature point, which is a feature point of each of the plurality of first point cloud data, and a second feature point, which is a feature point of the second point cloud data; an initial alignment unit that performs alignment of each of the plurality of first point cloud data and the second point cloud data based on the first feature point and the second feature point; a reference setting unit that sets a reference point cloud data from the plurality of first point cloud data; a range of motion determination unit that selects first point cloud data adjacent to the reference point cloud data as adjacent point cloud data, and determines the range of motion of the adjacent point cloud data based on the difference between the first feature point and the second feature point of the adjacent point cloud data, A point cloud data merging device comprising a conditional alignment unit that determines the position of the adjacent point cloud data by performing alignment of the adjacent point cloud data with respect to the reference point cloud data, provided that the position of the adjacent point cloud data is within the range of motion, and then merges the adjacent point cloud data with the reference point cloud data. (Note 2) The point cloud data merging device according to Note 1, wherein the reference setting unit sets the first point cloud data among the plurality of first point cloud data in which the difference between the first feature point and the second feature point is minimized as the reference point cloud data. (Note 3) The point cloud data merging device according to Note 1, wherein the reference setting unit sets the first point cloud data among the plurality of first point cloud data in which a feature object whose position is known in advance is included as the reference point cloud data. (Note 4) The point cloud data merging device according to any one of Notes 1 to 3, wherein the range of motion determination unit determines the range of motion as the range in which the difference between the first feature point and the second feature point of the adjacent point cloud data does not exceed a threshold. (Note 5) The point cloud data merging device according to any one of Notes 1 to 4, wherein the condition alignment unit determines the position of the adjacent point cloud data within the range of motion so as to minimize the error in alignment between the adjacent point cloud data and the reference point cloud data.(Note 6) The point cloud data merging device according to any one of Notes 1 to 5, wherein the condition alignment unit updates the point cloud data obtained by combining the adjacent point cloud data and the reference point cloud data as the reference point cloud data, and the point cloud data merging device further comprises a model generation unit that generates a three-dimensional model of the structure based on the reference point cloud data when all of the plurality of first point cloud data are merged as the reference point cloud data. (Note 7) The point cloud data merging device according to any one of Notes 1 to 6, wherein the feature extraction unit acquires the plurality of first point cloud data obtained by measuring the structure from multiple positions and the second point cloud data obtained by measuring the structure while moving. (Note 8) The point cloud data merging device according to Note 7, wherein the feature extraction unit acquires the plurality of first point cloud data measured by a fixed LiDAR sensor and the second point cloud data acquired by a portable LiDAR sensor. (Note 9) The point cloud data merging device comprises a distribution calculation unit for visualizing the distribution of the second feature points in the second point cloud data, and the feature extraction unit acquires the plurality of first point cloud data including markers installed according to the distribution of the second feature points, as described in any one of Notes 1 to 8.(Note 10) In a point cloud data merging system for merging point cloud data used to generate a three-dimensional model of a structure, the system includes: a feature extraction unit that acquires a plurality of first point cloud data, each representing a part of the structure, and a second point cloud data representing the structure, the second point cloud data having a lower density than each of the plurality of first point cloud data, and extracts a first feature point, which is a feature point of each of the plurality of first point cloud data, and a second feature point, which is a feature point of the second point cloud data; an initial alignment unit that performs alignment of each of the plurality of first point cloud data and the second point cloud data based on the first feature point and the second feature point; a reference setting unit that sets a reference point cloud data from the plurality of first point cloud data; a range of motion determination unit that selects first point cloud data adjacent to the reference point cloud data as adjacent point cloud data, and determines the range of motion of the adjacent point cloud data based on the difference between the first feature point and the second feature point of the adjacent point cloud data; A point cloud data merging system comprising a conditional alignment unit that determines the position of adjacent point cloud data by performing alignment of adjacent point cloud data with respect to the reference point cloud data, provided that the position of the adjacent point cloud data is within the range of motion, and then merges the adjacent point cloud data with the reference point cloud data.(Note 11) The computer acquires a plurality of first point cloud data, each representing a part of a structure, and a second point cloud data representing the structure, which is less dense than each of the plurality of first point cloud data. The computer extracts a first feature point, which is a feature point of each of the plurality of first point cloud data, and a second feature point, which is a feature point of the second point cloud data. The computer aligns each of the plurality of first point cloud data with the second point cloud data based on the first and second feature points. The computer sets a reference point cloud data from the plurality of first point cloud data. The computer selects first point cloud data adjacent to the reference point cloud data as adjacent point cloud data, and determines the range of motion of the adjacent point cloud data based on the difference between the first and second feature points of the adjacent point cloud data. A point cloud data merging method comprising a computer determining the position of adjacent point cloud data by aligning the adjacent point cloud data with the reference point cloud data, provided that the position of the adjacent point cloud data is within the range of motion, and merging the adjacent point cloud data with the reference point cloud data.(Note 12) A feature extraction process that obtains a plurality of first point cloud data, each representing a part of a structure, and a second point cloud data representing the structure, which has a lower density than each of the plurality of first point cloud data, and extracts a first feature point, which is a feature point of each of the plurality of first point cloud data, and a second feature point, which is a feature point of the second point cloud data; an initial alignment process that aligns each of the plurality of first point cloud data with the second point cloud data based on the first feature point and the second feature point; a reference setting process that sets a reference point cloud data from the plurality of first point cloud data; a range of motion determination process that selects first point cloud data adjacent to the reference point cloud data as adjacent point cloud data, and determines the range of motion of the adjacent point cloud data based on the difference between the first feature point and the second feature point of the adjacent point cloud data; A point cloud data merging program that causes a computer to perform a conditional alignment process to merge the adjacent point cloud data with the reference point cloud data, by performing alignment of the adjacent point cloud data with respect to the reference point cloud data, under the condition that the position of the adjacent point cloud data is within the range of motion, thereby determining the position of the adjacent point cloud data.
[0068] 71 First point cloud data, 72 Second point cloud data, 711 First feature point, 721 Second feature point, 723 Reference point cloud data, 724 Adjacent point cloud data, 80 Range of motion, 100 Point cloud data merging device, 110 Feature extraction unit, 120 Initial alignment unit, 130 Reference setting unit, 140 Range of motion determination unit, 150 Condition alignment unit, 160 Model generation unit, 170 Storage unit, 180 Distribution calculation unit, 190 Distribution presentation unit, 210 Fixed LiDAR, 220 Portable LiDAR, 300 Structure, 909 Electronic circuit, 910 Processor, 921 Memory, 922 Auxiliary storage device, 930 Input interface, 940 Output interface, 950 Communication device.
Claims
1. A feature extraction unit that acquires a plurality of first point cloud data, each representing a part of a structure, and a second point cloud data representing the structure, which has a lower density than each of the plurality of first point cloud data, and extracts a first feature point, which is a feature point of each of the plurality of first point cloud data, and a second feature point, which is a feature point of the second point cloud data; an initial alignment unit that performs alignment of each of the plurality of first point cloud data and the second point cloud data based on the first feature point and the second feature point; a reference setting unit that sets a reference point cloud data from the plurality of first point cloud data; a range of motion determination unit that selects first point cloud data adjacent to the reference point cloud data as adjacent point cloud data, and determines the range of motion of the adjacent point cloud data based on the difference between the first feature point and the second feature point of the adjacent point cloud data; A point cloud data merging device comprising a conditional alignment unit that determines the position of adjacent point cloud data by performing alignment of adjacent point cloud data with respect to the reference point cloud data, provided that the position of the adjacent point cloud data is within the range of motion, and then merges the adjacent point cloud data with the reference point cloud data.
2. The point cloud data merging device according to claim 1, wherein the reference setting unit sets the first point cloud data from among the plurality of first point cloud data that minimizes the difference between the first feature point and the second feature point as the reference point cloud data.
3. The point cloud data merging device according to claim 1, wherein the reference setting unit sets a first point cloud data set that includes a feature whose position is known in advance from among the plurality of first point cloud data as the reference point cloud data.
4. The point cloud data merging device according to any one of claims 1 to 3, wherein the range of motion determination unit determines the range of motion as the range in which the difference between the first feature point and the second feature point of the adjacent point cloud data does not exceed a threshold.
5. The point cloud data merging device according to any one of claims 1 to 4, wherein the condition alignment unit determines the position of the adjacent point cloud data within the range of the movable range so as to minimize the error in alignment between the adjacent point cloud data and the reference point cloud data.
6. The point cloud data merging device according to any one of claims 1 to 5, wherein the condition alignment unit updates the point cloud data obtained by combining the adjacent point cloud data and the reference point cloud data as the reference point cloud data, and the point cloud data merging device comprises a model generation unit that generates a three-dimensional model of the structure based on the reference point cloud data when all of the plurality of first point cloud data are merged as the reference point cloud data.
7. The point cloud data merging device according to any one of claims 1 to 6, wherein the feature extraction unit acquires a plurality of first point cloud data obtained by measuring the structure from a plurality of positions and a second point cloud data obtained by measuring the structure while it is moving.
8. The point cloud data merging device according to claim 7, wherein the feature extraction unit acquires the plurality of first point cloud data measured by a fixed LiDAR sensor and the second point cloud data acquired by a portable LiDAR sensor.
9. The point cloud data merging device comprises a distribution calculation unit for visualizing the distribution of the second feature points in the second point cloud data, and the feature extraction unit acquires the plurality of first point cloud data including markers installed according to the distribution of the second feature points, according to any one of claims 1 to 8.
10. A point cloud data merging system for merging point cloud data used to generate a three-dimensional model of a structure, comprising: a feature extraction unit that acquires a plurality of first point cloud data, each representing a part of the structure, and a second point cloud data representing the structure, wherein the second point cloud data has a lower density than each of the plurality of first point cloud data, and extracts a first feature point, which is a feature point of each of the plurality of first point cloud data, and a second feature point, which is a feature point of the second point cloud data; an initial alignment unit that performs alignment of each of the plurality of first point cloud data and the second point cloud data based on the first feature point and the second feature point; a reference setting unit that sets a reference point cloud data from the plurality of first point cloud data; a range of motion determination unit that selects first point cloud data adjacent to the reference point cloud data as adjacent point cloud data, and determines the range of motion of the adjacent point cloud data based on the difference between the first feature point and the second feature point of the adjacent point cloud data; A point cloud data merging system comprising a conditional alignment unit that determines the position of adjacent point cloud data by performing alignment of adjacent point cloud data with respect to the reference point cloud data, provided that the position of the adjacent point cloud data is within the range of motion, and then merges the adjacent point cloud data with the reference point cloud data.
11. The computer acquires a plurality of first point cloud data, each representing a part of a structure, and a second point cloud data representing the structure, which has a lower density than each of the plurality of first point cloud data. The computer extracts a first feature point, which is a feature point of each of the plurality of first point cloud data, and a second feature point, which is a feature point of the second point cloud data. The computer aligns each of the plurality of first point cloud data with the second point cloud data based on the first and second feature points. The computer sets a reference point cloud data from the plurality of first point cloud data. The computer selects first point cloud data adjacent to the reference point cloud data as adjacent point cloud data, and determines the range of motion of the adjacent point cloud data based on the difference between the first and second feature points of the adjacent point cloud data. A point cloud data merging method comprising a computer determining the position of adjacent point cloud data by aligning the adjacent point cloud data with the reference point cloud data, provided that the position of the adjacent point cloud data is within the range of motion, and merging the adjacent point cloud data with the reference point cloud data.
12. A feature extraction process that obtains a plurality of first point cloud data, each representing a part of a structure, and a second point cloud data representing the structure, which has a lower density than each of the plurality of first point cloud data, and extracts a first feature point, which is a feature point of each of the plurality of first point cloud data, and a second feature point, which is a feature point of the second point cloud data; an initial alignment process that aligns each of the plurality of first point cloud data with the second point cloud data based on the first feature point and the second feature point; a reference setting process that sets a reference point cloud data from the plurality of first point cloud data; a range of motion determination process that selects first point cloud data adjacent to the reference point cloud data as adjacent point cloud data, and determines the range of motion of the adjacent point cloud data based on the difference between the first feature point and the second feature point of the adjacent point cloud data; A point cloud data merging program that causes a computer to perform a conditional alignment process to merge the adjacent point cloud data with the reference point cloud data, by performing alignment of the adjacent point cloud data with respect to the reference point cloud data, under the condition that the position of the adjacent point cloud data is within the range of motion, thereby determining the position of the adjacent point cloud data.