Low cost portable indoor three-dimensional mapping device and method

Abstract

The application provides a low-cost portable indoor three-dimensional mapping method and device, which performs multi-sensor data synchronous acquisition, comprises setting two linear scanning laser sensors with different axial positions, a vertical downward single-point laser ranging module and a MEMS inertial measurement unit, simultaneously performing data acquisition on the state and indoor space; sensor space relative relationship calibration is performed to determine the fusion of horizontal linear scanning laser data and the MEMS inertial measurement unit in two-dimensional position and posture; two-dimensional position and posture and laser ranging joint three-dimensional position and posture calculation comprises the following steps: performing optimal estimation on three-dimensional position and posture by utilizing extended Kalman filtering to fuse two-dimensional position and posture, MEMS inertial measurement unit data and vertical laser ranging information; projecting the three-dimensional position and posture estimated by the extended Kalman filtering to a two-dimensional plane, and using the three-dimensional position and posture as initial estimation for scanning matching; and generating indoor three-dimensional point cloud by fusing the three-dimensional position and posture and the linear scanning laser data of the inclined linear scanning laser sensor.

Description

Technical Field

[0001] This invention belongs to the field of multi-sensor integration and mobile measurement. Through a complete set of device integration schemes and data processing procedures, it realizes real-time estimation of the device's own three-dimensional pose during use, and enables rapid acquisition and updating of indoor three-dimensional geospatial data. Background Technology

[0002] Mobile surveying technology is a measurement technique that integrates positioning, attitude determination, and measurement. It enables measurement while in motion, avoiding the waste of human and time resources caused by moving instruments between multiple stations, greatly improving the freedom of the measurement platform, and solving the problem of blind spots that may exist in traditional surveying. Accurate indoor 3D spatial information can be applied to many fields such as refined management of public places, indoor navigation, ancient heritage protection, and indoor disaster emergency avoidance. Rapidly creating accurate indoor maps has become a prerequisite for applications such as building information modeling management, indoor location-based services, augmented reality, and virtual reality. However, in indoor environments, effective GNSS satellite signals are often unavailable, making it impossible to use GNSS to determine the position and extract the trajectory of sensors and carriers. The continuous maturation of SLAM technology provides strong technical support for indoor mobile surveying, solving the problem of how mobile surveying devices can achieve "self-localization" in the absence of GNSS signals.

[0003] Currently, laser SLAM methods using single-line LiDAR as the primary sensor are relatively mature. The Hector SLAM scheme proposed in the literature (Kohlbrecher S, Von Stryk O, Meyer J, et al. "A flexible and scalable SLAM system with full 3D motion estimation". Proc of IEEE International Symposium on Safety, Security, and Rescue Robotics. 2011.) is an algorithm that only performs front-end scan matching without back-end optimization. This scheme uses the Gauss-Newton method to solve the front-end scan matching problem, matching the LiDAR data acquired in each frame with the map. By using fast approximation of map gradients and multi-resolution grids, it achieves reliable localization and mapping capabilities in different environments. This method is suitable for single-line LiDAR and can estimate the pose (including planar coordinates and orientation angles) with relatively low computational cost, constructing a two-dimensional grid map. However, two-dimensional grid maps have drawbacks such as low resolution and a single viewpoint, failing to provide a detailed and comprehensive representation of the scene. The Cartographer open-source solution proposed in the literature (Hess W, Kohler D, Rapp H, et al. “Real-time loop closure in 2D LIDARSLAM”. 2016 IEEE International Conference on Robotics and Automation (ICRA). 2016.) focuses on creating local sub-maps that fuse multi-sensor data and a scan matching strategy for loop closure detection. It optimizes local errors through extended Kalman filtering and distributes global errors through graph optimization. This method can be used for 2D localization and mapping with single-line LiDAR, and can also be used for multi-line LiDAR to estimate 3D coordinates and pose (six degrees of freedom), and reconstruct a 3D point cloud map. Currently, representative 3D laser SLAM mobile mapping devices on the market mainly include cart-type devices such as the NavVis M6, backpack-type devices such as the Leica Pegasus backpack, and handheld devices such as the GeoSLAM ZEB-REVO. Among them, the GeoSLAM ZEB-REVO, the most portable handheld laser SLAM mapping device, still weighs 3.5kg, making it inconvenient for mapping personnel to conduct large-scale, long-term mapping. Furthermore, the commercial prices of these three devices are generally high, making them unsuitable for use in research and production projects with limited budgets.Therefore, there is an urgent need for an indoor 3D mapping solution that can reduce the difficulty of data collection and the cost of equipment purchase.

[0004] In recent years, with the maturation of laser scanner design principles and the reduction in production costs, laser SLAM has gradually become the most popular research topic in the SLAM field. Based on this, designing a low-cost, portable multi-sensor integrated indoor 3D mapping device and method is not only technically feasible but will also provide the market with a new option, possessing significant promotional value. Summary of the Invention

[0005] This invention proposes a low-cost, portable, multi-sensor integrated indoor 3D mapping solution. By synchronously integrating multiple low-cost, consumer-grade line-scan laser sensors and single-point laser ranging modules, a lightweight, portable multi-sensor integrated indoor 3D mapping device is constructed, enabling rapid acquisition and updating of indoor 3D geospatial data.

[0006] To achieve the above objectives, the present invention provides a low-cost, portable indoor 3D mapping method, comprising the following steps:

[0007] Step 1, multi-sensor data synchronous acquisition, includes setting up two line-scan laser sensors with different orientations, a vertically downward single-point laser ranging module and a MEMS inertial measurement unit. These sensors simultaneously collect data on their own status and the indoor space to achieve comprehensive perception of indoor three-dimensional spatial information.

[0008] Step 2, sensor spatial relative relationship calibration, includes calibrating the external parameters between different sensors as a prerequisite for multi-sensor data fusion, and determining the spatial relative relationship between horizontally mounted line scan laser sensors and tilted line scan laser sensors;

[0009] Step 3: Determine the two-dimensional pose of the horizontally positioned linear scanning laser data fused with the MEMS inertial measurement unit;

[0010] Step 4: Joint 3D pose calculation using 2D pose and laser ranging, including using extended Kalman filter to fuse 2D pose, MEMS inertial measurement unit data and vertical laser ranging information to perform optimal estimation of 3D pose; projecting the 3D pose estimated by extended Kalman filter onto a 2D plane as the initial estimate for scan matching;

[0011] Step 5: The three-dimensional pose and the tilted line scan laser data are fused to generate an indoor three-dimensional point cloud.

[0012] Furthermore, step 2 is implemented by including the following sub-steps:

[0013] Step 2.1: In a regular rectangular corridor, the integrated device is rotated along different axes to extract straight line segments from two sets of point cloud data collected by the dual-axis scanning sensor.

[0014] Step 2.2: By determining the relationship between the line and the corridor surface, find the valid corridor observation information in each set of data.

[0015] Step 2.3: Construct coplanar and orthogonal constraints between line segments.

[0016] Step 2.4 transforms the calibration problem into a nonlinear optimization problem based on coplanar and orthogonal constraints, solves it multiple times and performs error processing to obtain the calibration result.

[0017] Furthermore, step 3 is implemented by including the following sub-steps:

[0018] Step 3.1 involves preprocessing the scanned point cloud data, including initial value correction, downsampling, and outlier removal, to improve data quality.

[0019] Step 3.2: Construct a map based on the representation of the grid occupancy probability map, describe the environment using laser data, and estimate the grid occupancy probability and gradient using bilinear interpolation.

[0020] Step 3.3: Match the data obtained by the laser sensor with each other or with the generated map to find the optimal two-dimensional pose of the device that minimizes the measurement error.

[0021] Furthermore, step 5 is implemented by including the following sub-steps:

[0022] Step 5.1: Based on the sampling frequency of the tilted line scan laser data, perform linear time interpolation on the three-dimensional pose to obtain a pose output that is aligned with the timestamp of the tilted line scan laser data.

[0023] Step 5.2: Using the calibration extrinsic parameters of the tilted line scan sensor in the integrated device obtained in Step 2, and the three-dimensional pose of the device in the world reference frame obtained in Step 4, the three-dimensional point cloud of the indoor geographic space is reconstructed.

[0024] On the other hand, the present invention also provides a low-cost portable indoor three-dimensional mapping device for implementing the low-cost portable indoor three-dimensional mapping method described in any of the above claims.

[0025] Furthermore, it includes an industrial control computer and multiple sensors, including two line-scan laser sensors with different axes, a vertically downward single-point laser ranging module, and a MEMS inertial measurement unit. These sensors simultaneously collect data on their own status and the indoor space, and transmit the data to the industrial control computer for data processing and storage, in order to achieve comprehensive perception of indoor three-dimensional spatial information.

[0026] Furthermore, the two line-scan laser sensors with different axial orientations include a horizontal line-scan laser sensor and a tilted line-scan laser sensor.

[0027] Moreover, the tilted line scan laser sensor is mounted above the horizontal line scan laser sensor, and the plane in which it is located makes an angle of approximately 30 degrees with the horizontal plane.

[0028] The present invention has the following positive effects:

[0029] 1) This invention proposes a low-cost, portable, multi-sensor integrated indoor 3D mapping method and technical process. By synchronously integrating multiple low-cost, consumer-grade line-scan laser sensors and laser rangefinders, a lightweight, portable multi-sensor integrated indoor 3D mapping device is constructed to achieve rapid acquisition and updating of indoor 3D geospatial data.

[0030] 2) This invention solves the problem that traditional line scan laser sensors can only generate low-resolution, single-view grid maps through two-dimensional SLAM. By using two line scan laser sensors with different mounting axes, a detailed and comprehensive representation of the scene can be achieved.

[0031] 3) The indoor three-dimensional mapping method proposed in this invention has good versatility and does not have specific requirements for the model of the line scan laser sensor and the integrated external parameters. The external parameter calibration site used is a common rectangular corridor in reality. The calibration principle is clear and easy to understand, the operation is simple, and no additional calibration device is required.

[0032] 4) This invention utilizes the integration of advanced laser scanning autonomous positioning and attitude determination technology with MEMS technology to achieve positioning and attitude determination of mobile carriers indoors without GNSS signals.

[0033] This invention employs multi-sensor integration and mobile measurement technology to significantly improve the efficiency and accuracy of indoor 3D spatial information acquisition, reduce the workload and cost of indoor 3D spatial information acquisition and updating, and provide a low-cost solution for indoor 3D mapping and modeling. It has broad application prospects in fields such as indoor 3D mapping, 3D BIM, indoor location services, and indoor space safety operation and maintenance. Attached Figure Description

[0034] Figure 1This is a flowchart of a method according to an embodiment of the present invention;

[0035] Figure 2 This diagram illustrates the placement and operating mode of the position sensor according to an embodiment of the present invention.

[0036] Figure 3 This is a schematic diagram of the bilinear interpolation estimation occupancy probability and gradient in an embodiment of the present invention. Detailed Implementation

[0037] The technical solution of the present invention will be described below with reference to the accompanying drawings and embodiments.

[0038] like Figure 1 As shown, this embodiment of the invention provides a low-cost, portable, multi-sensor integrated indoor 3D mapping device and method. The indoor 3D mapping method implemented based on the device includes the following steps:

[0039] Step 1), Synchronous Acquisition of Multi-Sensor Data. The low-cost portable multi-sensor integrated indoor 3D mapping device proposed in this invention includes multiple different types of sensors, at least including: two line-scan laser sensors with different axes of orientation, a vertically downward single-point laser ranging module, and a MEMS inertial measurement unit. These sensors simultaneously collect data on their own status and the indoor space, and transmit the data to a built-in industrial control computer for processing and storage, thereby achieving comprehensive perception of indoor 3D spatial information. The embodiment uses a built-in industrial control computer equipped with an Intel Pentium N4200 quad-core processor and 8GB of memory. Existing products can be used for each sensor, such as:

[0040] Line scan laser sensors: SLAMTEC Rplidar A3, SLAMTEC Rplidar S1

[0041] Vertical downward single-point laser ranging module: Motian RF LDM_L1

[0042] MEMS Inertial Measurement Unit: SuperNuclear Electronics HI226

[0043] In practice, each sensor can be connected to an industrial control computer. Alternatively, the collected sensor data can be quantized and transmitted to a mobile client for visualization.

[0044] The preferred arrangement and operating mode of the four sensors used in the embodiment are as follows: Figure 2 As shown.

[0045] The coordinate system of the MEMS inertial measurement unit is taken as the carrier coordinate system of the device, with the direction directly in front of the device as the x-axis and the plane on which the device is located as the xoy plane, thus constructing an xyz right-handed coordinate system. The MEMS inertial measurement unit calculates high-frequency attitude angles and other information by integrating acceleration and angular velocity; a horizontal line-scan laser sensor is mounted on the xoy plane and estimates the pose of the device by horizontal scanning; a single-point laser ranging module is mounted directly below the MEMS inertial measurement unit and emits laser pulses to the ground for ranging; an inclined line-scan laser sensor is mounted above the horizontal line-scan laser sensor, with its plane making an angle of approximately 30 degrees with the xoy plane (which is also the horizontal plane of the horizontal line-scan laser sensor).

[0046] Step 2), sensor spatial relative relationship calibration. Calibrating the external parameters of different sensors is a prerequisite for multi-sensor data fusion. The spatial relative relationship between horizontally mounted line-scan laser sensors and tilted line-scan laser sensors directly affects the accuracy of indoor 3D point clouds.

[0047] The present invention further proposes that the implementation process includes the following sub-steps:

[0048] Step 2.1: In a regular rectangular corridor, the integrated device is rotated along different axes to extract straight line segments from two sets of point cloud data collected by the dual-axis scanning sensor.

[0049] Step 2.2: By determining the relationship between the line and the corridor surface, find the valid corridor observation information in each set of data.

[0050] Step 2.3: Construct coplanar and orthogonal constraints between line segments.

[0051] Step 2.4 transforms the calibration problem into a nonlinear optimization problem based on coplanar and orthogonal constraints, solves it multiple times and performs error processing to obtain the calibration result.

[0052] For ease of implementation and reference, the preferred implementation method adopted in the embodiments is provided as follows:

[0053] First, a regular rectangular corridor is selected as the calibration site. The integrated device is rotated along different axes to collect observation data. For example, it is rotated 90° around the z-axis of the carrier coordinate system, then 90° around the y-axis, and finally 90° around the x-axis, forming a rotation sequence containing the scanning results of the rectangular corridor in multiple directions. S1 and S2 represent the coordinate systems of the horizontal and tilted lidars, respectively. [1|1] and [2|2] represent the attitudes of the two lidars relative to a certain reference coordinate system, respectively. R represents the rotation matrix, and T represents the translation vector. That is, R1 and R2 represent the corresponding rotation matrices of the horizontal and tilted lidars, respectively, and T1 and T2 represent the corresponding translation vectors of the horizontal and tilted lidars, respectively. For ease of calculation, S1 is selected as the reference coordinate system. Then R1 is the identity matrix, T1 is the zero vector, and the final calibration result is [2|2]. In an ideal case, two non-coplanar line-scanning laser sensors are used to scan a regular rectangular corridor, and two rectangles falling on the surface of the corridor can be obtained at each moment. Straight line segments were extracted from the cuboid corridor contours acquired by different line-scan laser sensors based on the RANSAC algorithm.

[0054] Secondly, based on the relationship between the direction vectors of the line segments, the set of line segments for each group of observation data is sorted and numbered. This invention further proposes an observation evaluation method based on coplanarity, which finds the correct arrangement and transformation from line segment numbers to corridor plane numbers, determines the subordinate relationship between line segments and corridor planes, and ensures that lines with the same corridor plane number are coplanar line pairs, and lines with adjacent corridor plane numbers lie on two mutually perpendicular planes.

[0055] Let i (i = 1, 2) represent the number of the line scan laser sensor. The sorted set of lines scanned by the i-th line scan laser sensor can be represented as:

[0056]

[0057] in, It is the straight line numbered j scanned by the i-th line-scanning laser sensor.

[0058] A corridor observation can be defined as a set containing at most four straight lines:

[0059] CO={S1,S2,S3,S4} (2)

[0060] Among them, S a (a = 1, 2, 3, 4) represents the set of lines belonging to corridor plane a.

[0061]

[0062] For any two line segments extracted from the scanning results of a line-scan laser sensor, their four endpoints can form a tetrahedron. Let... and Let each be a vector drawn from one vertex of the tetrahedron to the other three vertices. Then, the volume of the tetrahedron can be calculated using the following formula:

[0063]

[0064] When two line segments are coplanar, the volume V of the tetrahedron tetrahedron It is 0. Therefore, it can be determined according to V. tetrahedron The positional relationship between two line segments is determined by whether their values ​​are close to zero. For all previously generated possible corridor observations, each observation can be evaluated using the sum of the volumes of tetrahedra:

[0065]

[0066] Where, n v V represents the number of straight lines on the corridor surface v, and the function Φ() represents the sum of the volumes of the tetrahedrons formed by all pairs of line segments. CO The final score for the corridor observations is determined by the corridor observation with the smallest sum of tetrahedral volumes.

[0067] Then, coplanar and orthogonal constraints are constructed between the line segments. Coplanar constraints mean that when two scanned line segments lie on the same surface of the cuboid corridor, these two line segments should be coplanar in three-dimensional space; orthogonal constraints mean that adjacent surfaces of the cuboid corridor containing the scanned line segments should be perpendicular to each other. Finally, the calibration problem is transformed into a nonlinear optimization problem based on coplanar and orthogonal constraints. Four line segments falling on the corridor surface are extracted from the scanning results of the line-scan laser sensor, and L... i C i and I i (i = 1, 2) represent the line segment, its midpoint, and its direction vector extracted from the scanning result of the i-th line-scan laser sensor, respectively. Then, the coplanar constraint and orthogonal constraint can be expressed as:

[0068]

[0069] and

[0070]

[0071] The superscripts a and a+1 are used to distinguish the two adjacent corridor surfaces to which the line segment belongs (a = 1, 2, 3, 4), and n a and n a+1 Let these represent the normal vectors of the planes containing corridors a and a+1, respectively.

[0072] Finally, the calibration problem is transformed into a nonlinear optimization problem based on coplanar and orthogonal constraints, expressed as:

[0073]

[0074] Where N is the number of corridor observations, and a and a+1 are used to distinguish the two adjacent corridor surfaces to which the line segment belongs. (u = a or u = {a, a+1}) is the corridor observation value CO r The weights corresponding to the residuals are represented by [R²|T²], which indicates the calibration result of the extrinsic parameters to be optimized. It is recommended to use the Levenberg-Marquardt method to iteratively solve this nonlinear least squares problem, performing multiple solutions and error processing to obtain the calibration result.

[0075] In practice, multiple sets of observations can be performed based on the required accuracy before determining the final calibration result. In this example, five independent sets of observations are conducted at the calibration site. For each set of independent observations, 100 repeated calibration calculations are performed. Values ​​exceeding three times the standard deviation are removed from each calibration result, and the average is taken as the final calibration result for that set.

[0076] The present invention further proposes that the implementation process includes the following sub-steps:

[0077] Step 3) Horizontally place line scan laser data and fuse it with MEMS inertial measurement unit to calculate two-dimensional pose.

[0078] Step 3.1 involves preprocessing the scanned point cloud data, including initial value correction, downsampling, and outlier removal, to improve data quality.

[0079] Step 3.2: Construct a map based on the representation of the grid occupancy probability map, describe the environment using laser data, and estimate the grid occupancy probability and gradient using bilinear interpolation.

[0080] Step 3.3: Match the data obtained by the laser sensor with each other or with the generated map to find the optimal two-dimensional pose of the device that minimizes the measurement error.

[0081] For ease of implementation and reference, the preferred implementation method adopted in the embodiments is provided as follows:

[0082] First, the horizontally positioned line-scan laser data is preprocessed. Initial value correction is performed on the laser scan based on the attitude information calculated by the MEMS inertial measurement unit. Dense scan point clouds are downsampled and noise filtered to remove outliers, improving data quality and facilitating subsequent scan matching.

[0083] Secondly, the environment is described using laser data, expressed as a discrete raster occupancy probability map. Bilinear interpolation is used to estimate P at any raster location. m The occupancy probability M(P) at (x,y) is... m and gradients along the x and y axes. and like Figure 3 As shown on the left, for any point on the map, we can determine its relationship with the four surrounding grid vertices P. 00 (x0,y0),P 01 (x0,y1),P 10 (x1,y0) and P 11 The distance between (x1, y1) and the occupancy probability M(P) at the vertex. 00 M(P) 01 M(P) 10 ) and M(P 11 The occupancy probability of a point is linearly interpolated along the x and y axes.

[0084]

[0085] Similarly, since the four interpolation points are on the same grid with a unit distance of 1, it can also be done as follows: Figure 3 As shown on the right, the gradients in two directions are calculated:

[0086]

[0087] Finally, the data obtained by the laser sensor is matched with the generated map to find the optimal 2D pose of the device that achieves the best match between the line scan laser data and the map. For any 2D pose of the device, the sum of the occupancy probabilities of the endpoints of all laser beams on the map is calculated. When this value is maximized, the device pose at the current moment is the desired pose. An objective function is constructed to solve for the optimal estimate ε. * :

[0088]

[0089] Where n is the total number of scan points in the current frame, ε = (P x ,P y ,φ) T P represents the two-dimensional pose of the device on the map. x ,P y φ represents the x-coordinate, y-coordinate, and horizontal orientation angle of the device, respectively; S k (ε) represents mapping the k-th laser scanning point to the map coordinate system, M(S k (ε) represents S kThe map occupancy probability of position (ε). The Gauss-Newton method is used to solve the nonlinear optimization problem. For an initial pose estimate ε, Δε is estimated according to equation (12).

[0090]

[0091] For M(S) in equation (12) k Perform a first-order Taylor expansion of (ε+Δε):

[0092]

[0093] Differentiating equation (13) with respect to Δε, we get:

[0094]

[0095] Therefore, Δε can be calculated:

[0096]

[0097] Where H is an approximate second-order Hessian matrix:

[0098]

[0099] The gradient of the map can be calculated using equation (10). The optimal estimate of ε can be obtained through the above steps. * That is, the optimal pose estimate of the device at the current moment.

[0100] Step 4) Combine 2D pose and laser ranging to calculate 3D pose. Since the mapping device has six degrees of freedom in 3D space, representing its state using only 2D pose is incomplete. Therefore, this invention proposes to fuse 2D pose and laser ranging information to obtain the device's 3D pose.

[0101] The present invention further proposes that the implementation process includes the following sub-steps:

[0102] Step 4.1: Optimal estimation of three-dimensional pose is performed by fusing two-dimensional pose, MEMS inertial measurement unit data, and vertical laser ranging information using extended Kalman filtering.

[0103] Step 4.2: Project the 3D pose estimated by the extended Kalman filter onto the 2D plane as the initial estimate for scan matching.

[0104] For ease of implementation and reference, the preferred implementation method adopted in the embodiments is provided as follows:

[0105] First, the two-dimensional pose estimated using horizontally mounted line-scan laser data and the three-dimensional pose information obtained by the MEMS inertial measurement unit are fused using an extended Kalman filter. Then, the device elevation is calculated based on the ranging information obtained by the vertical single-point laser ranging module, resulting in the device's three-dimensional pose information with six degrees of freedom.

[0106] The state of the device in three-dimensional space is represented as x = (p x ,p y ,p z ,φ,θ,ψ,v x ,v y ,v z ) T , where (p x ,p y ,p z ) T Represents three-dimensional position coordinates, (φ,θ,ψ) T For roll, pitch, and yaw angles, (v x ,v y ,v z ) T Represent the velocity vector. The state estimate is expressed as... Its covariance matrix is ​​P, and the two-dimensional pose estimated in step 3 is ε. * Its corresponding covariance matrix R = H -1 The fusion process can then be represented as:

[0107] P + =P-(1-w) -1 KAP (17)

[0108]

[0109] in, and P + These are the state estimates and their corresponding covariance matrices obtained after one fusion update, respectively. Matrix A transforms the entire state space into a three-dimensional subspace of the SLAM system. w∈(0,1) are adjustable weight parameters, and K is the Kalman gain, which is calculated as follows:

[0110]

[0111] By adding vertical single-point laser ranging information to the optimized three-dimensional spatial state vector, the absolute height information of the device in the indoor space is obtained.

[0112] Then, the 3D pose of the device estimated by the extended Kalman filter is projected onto a horizontal plane and used as the initial estimate for the SLAM scan-match optimization process. A suitable initial estimate helps the scan-match process find the rigid body transformation parameters (translation and rotation parameters) of the system more quickly, effectively shortening the optimization process.

[0113] Step 5) Use the three-dimensional pose and the tilted line scan laser data to generate an indoor three-dimensional point cloud.

[0114] The present invention further proposes that the implementation process includes the following sub-steps:

[0115] Step 5.1: Based on the sampling frequency of the tilted line scan laser data, perform linear time interpolation on the three-dimensional pose to obtain a pose output that is aligned with the timestamp of the tilted line scan laser data.

[0116] Step 5.2: Using the calibration extrinsic parameters of the tilted line scan sensor in the integrated device obtained in Step 2, and the three-dimensional pose of the device in the world reference frame obtained in Step 4, the three-dimensional point cloud of the indoor geographic space is reconstructed.

[0117] For ease of implementation and reference, the preferred implementation method adopted in the embodiments is provided as follows:

[0118] First, due to differences in the power-on sequence and sampling frequency of different sensors, it cannot be guaranteed that every frame of data acquired by different sensors will have perfectly aligned timestamps. Therefore, the 3D pose of the device estimated jointly by the horizontal line scan data, MEMS inertial measurement unit data, and vertical single-point laser ranging data is not completely synchronized with the tilted line scan laser data. Linear time interpolation of the 3D pose is required based on the sampling frequency of the tilted line scan laser data to obtain a pose output that is aligned with the timestamps of the tilted line scan laser data.

[0119] Then, based on the calibration extrinsic parameters of the tilted line-scan laser sensor in the integrated device, and the three-dimensional pose of the device in the world coordinate system after time synchronization processing, the coordinates of the tilted line-scan laser point cloud can be transformed from its own sensor coordinate system to the world coordinate system, thus reconstructing the three-dimensional point cloud of the indoor geographic space.

[0120] In specific implementation, the method proposed in the technical solution of this invention can be automatically executed by those skilled in the art using computer software technology. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of this invention and computer equipment (e.g., industrial control computer) that includes the computer program running the corresponding computer program, should also be within the protection scope of this invention.

[0121] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A low-cost, portable indoor 3D mapping method, characterized in that: Includes the following steps, Step 1, multi-sensor data synchronous acquisition, includes setting up two line-scan laser sensors with different orientations, a vertically downward single-point laser ranging module and a MEMS inertial measurement unit. These sensors simultaneously collect data on their own status and the indoor space to achieve comprehensive perception of indoor three-dimensional spatial information. Step 2, sensor spatial relative relationship calibration, includes calibrating the external parameters between different sensors as a prerequisite for multi-sensor data fusion, and determining the spatial relative relationship between horizontally mounted line scan laser sensors and tilted line scan laser sensors; Step 3: Determine the two-dimensional pose of the horizontally positioned linear scanning laser data fused with the MEMS inertial measurement unit; Step 4: Joint 3D pose calculation using 2D pose and laser ranging, including using extended Kalman filter to fuse 2D pose, MEMS inertial measurement unit data and vertical laser ranging information to perform optimal estimation of 3D pose; projecting the 3D pose estimated by extended Kalman filter onto a 2D plane as the initial estimate for scan matching; Step 5: The three-dimensional pose and the tilted line scan laser data are fused to generate an indoor three-dimensional point cloud.

2. The low-cost portable indoor three-dimensional mapping method according to claim 1, characterized in that: Step 2 is implemented by including the following sub-steps: Step 2.1: In a regular rectangular corridor, rotate the integrated device along different axes to extract straight line segments from two sets of point cloud data collected by the dual-axis scanning sensor. Step 2.2: By determining the relationship between the line and the corridor surface, find the valid corridor observation information in each set of data; Step 2.3: Construct coplanar and orthogonal constraints between line segments; Step 2.4 transforms the calibration problem into a nonlinear optimization problem based on coplanar and orthogonal constraints, solves it multiple times and performs error processing to obtain the calibration result.

3. The low-cost portable indoor three-dimensional mapping method according to claim 1, characterized in that: Step 3 is implemented by including the following sub-steps: Step 3.1: Perform initial value correction, downsampling, and outlier removal on the scanned point cloud data to improve data quality; Step 3.2: Construct a map based on the representation of the grid occupancy probability map, describe the environment using laser data, and estimate the grid occupancy probability and gradient using bilinear interpolation. Step 3.3: Match the data obtained by the laser sensor with each other or with the generated map to find the optimal two-dimensional pose of the device that minimizes the measurement error.

4. The low-cost portable indoor three-dimensional mapping method according to claim 1, 2, or 3, characterized in that: Step 5 is implemented by including the following sub-steps: Step 5.1: Based on the sampling frequency of the tilted line scan laser data, perform linear time interpolation on the three-dimensional pose to obtain a pose output that is aligned with the timestamp of the tilted line scan laser data. Step 5.2: Using the calibration extrinsic parameters of the tilted line scan sensor in the integrated device obtained in Step 2, and the three-dimensional pose of the device in the world reference frame obtained in Step 4, the three-dimensional point cloud of the indoor geographic space is reconstructed.

5. A low-cost portable indoor three-dimensional mapping device, characterized in that: This method is used to implement a low-cost, portable indoor three-dimensional mapping method as described in any one of claims 1-4.

6. The low-cost portable indoor three-dimensional mapping device according to claim 5, characterized in that: It includes an industrial control computer and multiple sensors, including two line-scan laser sensors with different axes, a vertically downward single-point laser ranging module, and a MEMS inertial measurement unit. These sensors simultaneously collect data on their own status and the indoor space, and transmit the data to the industrial control computer for data processing and storage, so as to realize all-round perception of indoor three-dimensional spatial information.

7. The low-cost portable indoor three-dimensional mapping device according to claim 6, characterized in that: The two line-scan laser sensors with different orientations include a horizontal line-scan laser sensor and an inclined line-scan laser sensor.

8. The low-cost portable indoor three-dimensional mapping device according to claim 7, characterized in that: The tilted line-scan laser sensor is mounted above the horizontal line-scan laser sensor, with its plane at an angle of approximately 30 degrees to the horizontal plane.

Images