A method and apparatus for dynamic measurement of three-dimensional data acquisition and processing
By combining the time hard synchronization of IMU sensors and lidar with deep learning networks and filtering methods, the problems of motion distortion and noise reduction in dynamic 3D measurement are solved, achieving high-precision 3D data acquisition and processing, which is applicable to multiple engineering fields.
Patent Information
- Application Number
- CN202610970627.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-01
- Publication Date
- 2026-11-17
AI Technical Summary
Existing 3D measurement technologies lack sufficient accuracy in motion distortion correction during dynamic acquisition, have insufficient robustness in noise reduction methods, and lack a systematic processing flow, thus failing to meet the application needs of large-scale engineering machinery production and complex scenarios.
We employ hard synchronization between the IMU sensor and the lidar, eliminate point cloud distortion by mapping attitude angle changes, and combine kd-tree and K-means algorithms for denoising. Subsequently, we use the PointCleanNet deep learning network and improved bilateral filtering for accurate denoising.
It achieves high-precision motion distortion correction and comprehensive noise reduction, improving measurement accuracy and processing efficiency, adapting to complex scenarios, and is applicable to fields such as aerospace, automobile manufacturing, energy engineering, and infrastructure construction.
Smart Images

Figure CN123111065A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to a method and apparatus for dynamically measured three-dimensional data acquisition and processing. Background Technology
[0002] 3D measurement technology is a core means of acquiring 3D shape and size information of objects, and is widely used in aerospace, automobile manufacturing, energy engineering, infrastructure construction and other fields. With the advancement of intelligent construction, the market has placed higher demands on 3D measurement, urgently requiring high-speed dynamic response and millimeter-level accuracy data acquisition and processing in large-scale scenarios.
[0003] Current technologies still face the following bottlenecks:
[0004] (i) Insufficient accuracy of motion distortion correction. During dynamic acquisition, the rotation of the lidar platform causes points at different times within the same frame to be located in different coordinate systems, resulting in point cloud ghosting and positional deviation. In existing IMU-assisted correction methods, the raw IMU data contains zero-point offset and noise, and direct integration leads to rapid error accumulation. Furthermore, the coordinate system mapping between the IMU and lidar is coarse, resulting in limited correction accuracy.
[0005] (ii) The denoising methods lack robustness. Traditional filtering methods (statistical filtering, bilateral filtering, etc.) rely on local surface fitting or noise model assumptions, and the parameters need to be manually adjusted, making it difficult to adapt to different noise levels. Although deep learning methods do not require a pre-set model, a single network cannot simultaneously handle large-amplitude noise and small-amplitude high-frequency noise, and existing methods are mostly designed for static point clouds, with poor adaptability to dynamic scenes.
[0006] (iii) Lack of a systematic processing flow. Existing technologies are mostly optimized for single links and lack a closed-loop solution for the entire process from synchronous acquisition and distortion correction to precise noise reduction. This makes it impossible to meet the application needs of complex scenarios such as large-scale engineering machinery production, deep foundation pit monitoring, and irregular building measurement. Summary of the Invention
[0007] Purpose of the invention: The purpose of this invention is to solve the technical problems in the prior art and provide a method and apparatus for dynamic measurement of three-dimensional data acquisition and processing.
[0008] Technical solution: In the first aspect, this application proposes a method for dynamically measured three-dimensional data acquisition and processing, including the following steps:
[0009] Step 1: Acquire pose data from the IMU sensor and point cloud data from the LiDAR. The IMU sensor is mounted on the 3D laser scanner, and hard synchronization of time between the LiDAR and the IMU sensor is achieved.
[0010] Step 2: Preprocess the angular velocity data in the pose data, calculate the attitude angle change of the lidar during the acquisition process, map the attitude angle change to the lidar coordinate system, and use the mapping result to correct the point cloud coordinates of the original point cloud data in real time to eliminate the point cloud distortion caused by dynamic movement.
[0011] Step 3: For the corrected point cloud data, the kd-tree algorithm is used to remove isolated outlier noise points, and the K-means algorithm is used for clustering to remove edge noise points, thus completing the first denoising step.
[0012] Step 4: For the point cloud data after the first denoising, first use the PointCleanNet deep learning network to remove outliers and perform displacement projection denoising, and then use an improved bilateral filter for smoothing to complete the second denoising.
[0013] Preferably, the preprocessing includes zero-point offset correction and sliding window filtering;
[0014] The zero-point offset correction is as follows: During the static period after system startup, the angular velocity output of the IMU sensor in a stationary state is collected and its average value is calculated as the zero-point offset. In subsequent dynamic acquisitions, this offset is subtracted from the original measurement value in real time, as shown in the following formula:
[0015] ;
[0016] The sliding window filtering is performed by smoothing the corrected angular velocity data using a moving average filter with a window length of 5, as shown in the following formula:
[0017] ;
[0018] in, This represents the raw angular velocity data output by the IMU sensor at time t. The corrected angular velocity data, This is the filtered angular velocity data.
[0019] The preferred method for calculating attitude angle changes is as follows:
[0020] After obtaining the preprocessed angular velocity data, the angular velocity components along the x, y, and z axes in the IMU coordinate system are respectively... , , ;
[0021] The start time of each frame of point cloud from the lidar As the starting point for integration, during the point cloud acquisition time period of this frame... Integrating the preprocessed angular velocity, we obtain the attitude angle change at the end of the frame relative to the start time, as shown in the following formula:
[0022] ;
[0023] The integration employs the Euler numerical integration method, using discrete sampled values from the IMU for accumulation.
[0024] ;
[0025] in, The attitude angle at the initial moment is usually set as the zero vector, or determined by the initial alignment process; The pre-processed triaxial angular velocities; The time span for acquiring a single frame of point cloud data;
[0026] The index of the discrete sampling point. N is the total number of IMU sampling points within the single-frame point cloud acquisition time window; Let be the attitude angle at the k-th sampling time. The attitude angle at the (k-1)th sampling time, with an initial value. ; The preprocessed angular velocity at the k-th sampling time; The sampling time interval of the IMU is determined by the IMU sampling frequency.
[0027] Preferably, mapping the attitude angle change to the lidar coordinate system includes:
[0028] The fixed extrinsic rotation matrix from the IMU coordinate system to the lidar coordinate system is obtained through offline calibration. ;
[0029] Using the ZYX Euler angle sequence, i.e., the heading-pitch-roll sequence, the attitude change rotation matrix of the IMU coordinate system at time t relative to the initial time is constructed. ;
[0030] ;
[0031] The attitude changes in the IMU coordinate system are mapped to the lidar coordinate system using the following formula:
[0032] ;
[0033] in, , , These are the basic rotation matrices for rotations about the X, Y, and Z axes, respectively:
[0034] ;
[0035] Let be the rotation matrix of the IMU coordinate system at time t relative to the initial time, derived from the attitude angles. Calculated;
[0036] The fixed extrinsic parameter rotation matrix from the IMU coordinate system to the lidar coordinate system is obtained through calibration and satisfies... ;
[0037] for The inverse matrix of the rotation matrix is, since the rotation matrix is an orthogonal matrix, its inverse is equal to its transpose, i.e. ;
[0038] Let t be the rotation transformation matrix of the lidar coordinate system at time t relative to the initial time.
[0039] Preferably, the point cloud coordinates of the original point cloud data are corrected in real time using the mapping results, including:
[0040] Let the coordinates of the original point cloud collected by the lidar at time t be... The corrected point cloud coordinates are:
[0041] ;in, Let t be the rotation transformation matrix of the lidar coordinate system at time t relative to the initial time.
[0042] Preferably, the kd-tree algorithm for removing isolated outliers includes:
[0043] Building point cloud data The three-dimensional kd-tree index structure, This represents the three-dimensional coordinates of the i-th point in the point cloud, where n is the total number of points in the point cloud;
[0044] For each point, search its neighborhood using a kd-tree. For any point in the point cloud, the nearest neighbor is given by the nearest neighbor. ,calculate To its k neighborhood points Euclidean distance:
[0045] ;
[0046] Calculate the distance from this point to its k-th ... Calculate the Euclidean distance between each neighboring point and the average distance. The formula is as follows:
[0047] ;
[0048] Calculate the mean μ and standard deviation σ of the average distances between all points, and set a decision threshold. , where λ is the proportionality coefficient.
[0049] ;
[0050] ;
[0051] like If the point is isolated, it is considered an isolated outlier and is removed.
[0052] Preferably, before using the K-means algorithm for clustering, the optimal number of clusters K is determined using the elbow rule based on the point cloud data after kd-tree removal.
[0053] Let the point cloud data after removing isolated outliers be... The total number of point clouds is ;
[0054] The elbow rule is as follows: calculate the sum of squared clustering errors (SSE) under different K values, and select the inflection point where the rate of decrease of SSE slows down as the optimal K value.
[0055] The formula for calculating SSE is:
[0056] ;
[0057] in, This represents the m-th cluster; The center point of the m-th cluster is calculated using the following formula: ; For point To its respective cluster center The square of the Euclidean distance.
[0058] Preferably, K-means clustering is used to remove edge noise points, including:
[0059] Point cloud data after removing outliers Perform K-means clustering, iteratively assigning points and updating cluster centers until convergence;
[0060] Step 301: From point cloud data K points are randomly selected as the initial cluster centers. ;
[0061] Step 302: For each point in the point cloud Calculate the Euclidean distance from each cluster center and assign it to the nearest cluster, as shown in the following formula:
[0062] ;
[0063] in, The midpoint of the t-th iteration The cluster number to which it belongs;
[0064] Step 303: Recalculate the centroid of each cluster:
[0065] ;
[0066] This represents the total number of points that this cluster has in the current round. This represents the center point of the m-th cluster in the t-th iteration. This represents the center point of the m-th cluster in the (t+1)-th iteration;
[0067] Step 304: Repeat steps 302 and 303 until the cluster centers no longer change or the preset maximum number of iterations is reached;
[0068] Step 305: After clustering is completed, for each cluster... Calculate the distance from each point within a cluster to its cluster center. The Euclidean distance is used to obtain the distance set, as shown in the following formula:
[0069] ;
[0070] For the specific point p of the m-th cluster i The corresponding distance value;
[0071] Calculate the mean of intra-cluster distances. and standard deviation The formula is as follows:
[0072] ;
[0073] ;
[0074] Step 306: Set the decision threshold within the cluster. :
[0075] ;
[0076] Step 307: If the distance from a point to the cluster center is greater than... If a point is identified as a cluster noise point, it will be removed, as shown in the following formula:
[0077] .
[0078] Preferably, the PointCleanNet deep learning network is first used for outlier removal and displacement projection denoising, including:
[0079] The PointCleanNet deep learning network includes a four-element spatial transfer network, a multilayer perceptron with shared weights, a feature fusion module, and a regression module.
[0080] (1) The four-element spatial transfer network transforms the input point cloud to a normalized pose, including:
[0081] Let the input point cloud be ,in QSTN outputs quaternion transformation parameters The corresponding rotation matrix is:
[0082] ;
[0083] The transformed point cloud is as follows:
[0084] ;
[0085] in, Translation parameters
[0086] (2) The multilayer perceptron transforms the point cloud Extracting point-by-point features and then reconstructing the original pose space through inverse transformation includes:
[0087] The extracted feature vector is:
[0088] ;
[0089] Where d is the feature dimension;
[0090] After inverse transformation, the original pose space is restored as follows:
[0091] ;
[0092] in It is the inverse of the high-dimensional extended matrix, used to rotate the eigenvector field back to its original orientation;
[0093] (3) The feature fusion module obtains permutation-invariant fused features by summing the feature vectors of neighboring points, including:
[0094] Setting points The local neighborhood point set is The corresponding set of feature vectors is The feature fusion operation is as follows:
[0095]
[0096] (4) The regression module first outputs a binary classification probability based on the fused features, determines and removes outliers; then it outputs a correction displacement vector for the retained valid points, projecting the noise points onto a clean surface, including:
[0097] Phase 1: Outlier Classification and Removal
[0098] For each point The regression module outputs the network's predicted probabilities:
[0099] ;
[0100] in It is the Sigmoid activation function. To output a fully connected layer with a dimension of 2, If the point is an outlier, it is determined to be an outlier and removed.
[0101] like If the value is 0, it is determined to be a valid point and retained for subsequent noise reduction;
[0102] Second stage: Displacement vector estimation and projection:
[0103] For the retained valid points, the regression module estimates the corrected displacement vector:
[0104] ;
[0105] in For a fully connected layer with an output dimension of 3, the denoised point cloud coordinates are:
[0106]
[0107] This displacement vector accurately projects the noisy points onto the original clean point cloud surface, thus achieving the noise reduction function.
[0108] Preferably, an improved bilateral filter is then used for smoothing, including:
[0109] The core formula for improving bilateral filtering is:
[0110]
[0111] in For point The normal vector, The weighting factor for the bilateral filter is calculated using the following formula:
[0112]
[0113] in, For point The neighborhood point set is determined by kd-tree search;
[0114] For point With neighboring points The Euclidean distance between them;
[0115] , Let i be the point and its neighboring points. Intensity value or curvature characteristics;
[0116] The dot product of the normal vector and the direction vector of the neighboring point represents the projection of the neighboring point onto the normal direction.
[0117] Spatial domain weighting function, using Gaussian kernel form:
[0118]
[0119] The formula for calculating d is as follows:
[0120] ;
[0121] in This is a spatial bandwidth parameter that controls the smoothing range of the spatial neighborhood.
[0122] The feature domain weight function also adopts the Gaussian kernel form:
[0123]
[0124] in This is the feature bandwidth parameter, which controls the sensitivity of feature similarity.
[0125] Spatial bandwidth and characteristic bandwidth Based on adaptive determination of point cloud neighborhood density:
[0126] ;
[0127] ;
[0128] in, , It is an empirical coefficient, and , These are used to adjust the scaling ratios of spatial bandwidth and feature bandwidth, respectively.
[0129] The number of neighboring points is as follows:
[0130] in Based on the number of basic neighborhood points, For local point cloud density, For global point cloud density, , This represents the upper and lower limits of the number of neighboring points.
[0131] Preferably, the training loss function of the PointCleanNet network includes outlier detection loss and denoising loss;
[0132] The outlier detection loss is the cross-entropy loss:
[0133]
[0134] in These are the true labels (1 indicates an outlier, 0 indicates a valid point). Predict probabilities for the network.
[0135] Denoising loss (Chamfer distance variant):
[0136] ;
[0137] in For truly clean point clouds, For the denoised point cloud, this loss function simultaneously measures the proximity of the denoised points to the real surface and the uniformity of the real surface coverage.
[0138] n represents the point cloud after denoising. The number of points, m represents the true clean point cloud. The number of points in the middle;
[0139] The total loss function is:
[0140] ;
[0141] in The balance coefficient is set to 1.0.
[0142] Secondly, this application proposes a dynamic measurement three-dimensional data acquisition and processing device, comprising:
[0143] The acquisition unit is used to acquire pose data from the IMU sensor and point cloud data from the lidar. The IMU sensor is mounted on the 3D laser scanner and achieves hard time synchronization between the lidar and the IMU sensor.
[0144] The preprocessing unit is used to preprocess the angular velocity data in the pose data, calculate the attitude angle change of the lidar during the acquisition process, map the attitude angle change to the lidar coordinate system, and use the mapping result to correct the point cloud coordinates of the original point cloud data in real time to eliminate the point cloud distortion caused by dynamic movement.
[0145] The first noise reduction unit is used to remove isolated outlier noise points from the corrected point cloud data by sequentially using the kd-tree algorithm and the K-means algorithm to cluster and remove edge noise points, thus completing the first noise reduction.
[0146] The second denoising unit is used to process the point cloud data after the first denoising. First, it uses the PointCleanNet deep learning network to remove outliers and perform displacement projection denoising. Then, it uses an improved bilateral filter for smoothing to complete the second denoising.
[0147] Thirdly, this application proposes an electronic device including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and when the computer program is executed by the processor, the processor causes the processor to perform the method described in the above embodiments.
[0148] Fourthly, this application proposes a computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, it implements the method described in the above embodiments.
[0149] Beneficial effects:
[0150] 1. A comprehensive collaborative processing solution to the challenges of dynamic measurement. This invention constructs a closed-loop processing system encompassing "synchronous acquisition—coordinate correction—hybrid filtering and denoising—deep learning joint denoising," with each step progressing and working synergistically. Step 1 addresses data time synchronization, step 2 addresses motion distortion, and steps 3 and 4 address noise of different natures and amplitudes in stages, forming a complete technical chain from data acquisition to high-quality point cloud output. Compared to existing static or quasi-static measurement methods, this invention offers greater adaptability to various scenarios and significantly improves measurement accuracy and processing efficiency.
[0151] 2. High-precision motion distortion correction effectively eliminates point cloud ghosting and position deviation. This invention ensures precise matching of point cloud data and pose data in the time dimension through the IEEE 1588 clock synchronization protocol (PTP hard time synchronization). In step 2, zero-point offset correction and sliding window filtering preprocessing are performed on the raw IMU angular velocity data, effectively suppressing zero-point offset and measurement noise in the raw IMU data; then, the attitude angle change is calculated using the Euler numerical integration method, and the attitude change is accurately mapped to the lidar coordinate system through the extrinsic parameter rotation matrix; finally, the raw point cloud coordinates are corrected in real time, and all point cloud data collected during dynamic movement are uniformly transformed to the reference coordinate system at the initial moment.
[0152] 3. High-efficiency hybrid filtering for noise reduction, comprehensively eliminating outlier noise. A sequential complementary strategy of first using kd-tree and then K-means is adopted. The kd-tree algorithm utilizes its efficient neighborhood search capability to identify and remove isolated outliers based on statistical features, reducing the interference of outliers on subsequent cluster analysis. Building on this, the elbow rule is used to optimize the selection of the K value in the K-means algorithm, avoiding the subjectivity and blindness of K value selection in the traditional K-means algorithm, using the features of the point cloud data after kd-tree removal, thus eliminating noise points at the cluster edges. The two algorithms complement each other—kd-tree excels at handling isolated noise far from the main cluster, while K-means excels at handling noise at the cluster edges, achieving comprehensive and efficient removal of outlier noise points.
[0153] 4. Deep learning and improved bilateral filtering are used together for denoising, achieving accurate denoising and surface reconstruction in noisy environments. The PointCleanNet deep learning network adopts a two-stage cascaded architecture: the first stage classifies and removes outliers, and the second stage estimates and projects the displacement vectors of the remaining point cloud. This network does not require a pre-set noise model, and no parameters need to be provided by the user during testing. It can automatically discover and maintain high curvature features and is robust to noise of different levels and types. The network internally includes a four-element spatial transfer network (transforming the point cloud to a normal pose), a multilayer perceptron with shared weights (for point-by-point feature extraction), a feature fusion module (ensuring permutation invariance), and a regression module (for classification and displacement estimation). These modules work together to achieve accurate denoising.
[0154] The improved bilateral filtering method incorporates several improvements over the classic bilateral filtering method: it automatically calculates spatial bandwidth and feature bandwidth parameters based on the neighborhood density and curvature features of the point cloud, avoiding manual parameter tuning; it introduces a normal outlier factor (NOF) to remove outliers in the neighborhood during normal vector calculation, thereby improving the accuracy of normal vector estimation; and it uses a kd-tree to establish the topological relationship of the point cloud and adaptively determines the number of neighborhood points based on the neighborhood density, avoiding neighborhood selection bias caused by a fixed radius.
[0155] PointCleanNet handles large-amplitude noise and outliers, while the improved bilateral filter handles small-amplitude high-frequency noise. The two processes are executed sequentially and complement each other, achieving accurate denoising and surface reconstruction in noisy environments.
[0156] 5. Extensive experimental verification and significant results. This invention combines simulation experiments with real-world dataset testing, using the EuRoC and TUM-VI datasets as the testing foundation. A real-world dynamic measurement hardware platform combining LiDAR and IMU was built for on-site data acquisition experiments. The experiments selected absolute azimuth error (AOE) and relative azimuth error (ROE) as the core evaluation indicators for point cloud coordinate correction, and point cloud denoising accuracy, feature retention rate, and processing efficiency as evaluation indicators for denoising effect. Median error was selected as the overall dynamic measurement accuracy indicator. The experimental results fully verify the effectiveness, accuracy, and robustness of the method under dynamic measurement conditions. Attached Figure Description
[0157] Figure 1 This is a schematic diagram of the method according to an embodiment of the present invention;
[0158] Figure 2 This is a schematic diagram of the system structure provided in one embodiment.
[0159] Figure 3 This is a block diagram of an electronic device structure provided in one embodiment of this application. Detailed Implementation
[0160] To make the technical solution of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0161] Example 1:
[0162] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions in the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention. Unless otherwise defined, the technical or scientific terms used herein should have the ordinary meaning understood by those skilled in the art. The terms "including" and similar expressions used herein mean that the element or object preceding the term covers the element or object listed after the term and its equivalents, but do not exclude other elements or objects.
[0163] In response to the problems existing in the current technology, such as Figure 1As shown, a method for dynamic measurement of three-dimensional data acquisition and processing is proposed, including the following steps:
[0164] Step 1: Acquire pose data from the IMU sensor and point cloud data from the LiDAR. The IMU sensor is mounted on the 3D laser scanner, and hard synchronization of time between the LiDAR and the IMU sensor is achieved.
[0165] By mounting an IMU sensor on a 3D laser scanner (LiDAR), during the 3D data acquisition process, PTP time hard synchronization between the LiDAR and the IMU sensor is achieved through the Ethernet port based on the IEEE 1588 clock synchronization protocol, ensuring that the point cloud data of the LiDAR and the pose data of the IMU sensor are accurately matched in the time dimension.
[0166] Specifically, the IMU sensor is rigidly fixed to the housing of the 3D laser scanner, ensuring that their relative position and attitude remain unchanged during movement. After system startup, the lidar and IMU sensor are connected to the same time synchronization network via Ethernet, and clock synchronization is performed based on the IEEE 1588 protocol, achieving microsecond-level synchronization accuracy. During data acquisition, the lidar records the start and end timestamps of each frame of point cloud data acquired; the IMU sensor outputs angular velocity data at a fixed frequency (e.g., 200Hz), with each data point carrying a high-precision timestamp. Through time synchronization, it is ensured that each point cloud data point can find its corresponding IMU pose data on the timeline.
[0167] PTP hard synchronization enables precise matching of the point cloud data of the LiDAR and the pose data of the IMU sensor in the time dimension, providing the basic data for subsequent point cloud coordinate correction and avoiding correction errors caused by time misalignment.
[0168] Step 2: Preprocess the angular velocity data in the pose data, calculate the attitude angle change of the lidar during the acquisition process, map the attitude angle change to the lidar coordinate system, and use the mapping result to correct the point cloud coordinates of the original point cloud data in real time to eliminate the point cloud distortion caused by dynamic movement.
[0169] The raw angular velocity data acquired by the IMU sensor includes zero-point offset and measurement noise. If integration is performed directly, the error will accumulate over time, leading to a decrease in the accuracy of attitude angle estimation. Therefore, the raw IMU data must be preprocessed before integration:
[0170] In some specific embodiments, the preprocessing includes zero-point offset correction and sliding window filtering;
[0171] The zero-point offset correction is as follows: During the static period after system startup, the angular velocity output of the IMU sensor in a stationary state is collected and its average value is calculated as the zero-point offset b0. In subsequent dynamic acquisitions, this offset is subtracted from the original measurement value in real time, as shown in the following formula:
[0172] ;
[0173] The sliding window filtering is performed by smoothing the corrected angular velocity data using a moving average filter with a window length of 5, as shown in the following formula:
[0174] ;
[0175] in, This represents the raw angular velocity data output by the IMU sensor at time t. The corrected angular velocity data, This is the filtered angular velocity data.
[0176] By employing dual preprocessing techniques of zero-point offset correction and sliding window filtering, systematic errors and random noise in the raw IMU data are effectively suppressed, providing high-quality angular velocity data for subsequent attitude angle calculations. Zero-point offset correction ensures that static errors are not amplified by integration, while sliding window filtering smooths out high-frequency noise. The combination of these two techniques significantly improves the accuracy and stability of attitude angle estimation.
[0177] In some specific embodiments, the method for calculating attitude angle changes is as follows:
[0178] The preprocessed angular velocity data is integrated over time to obtain the attitude angle change of the lidar during the acquisition process. The specific calculation formula is as follows: The angular velocity components along the x, y, and z axes in the IMU coordinate system are obtained after obtaining the preprocessed angular velocity. , , ;
[0179] The start time of each frame of point cloud from the lidar As the starting point for integration, during the point cloud acquisition time period of this frame... Integrating the preprocessed angular velocity, we obtain the attitude angle change at the end of the frame relative to the start time, as shown in the following formula:
[0180] ;
[0181] In practical engineering implementation, since the angular velocity output by the IMU is a discrete sampled value, the integration method uses Euler's numerical integration method, which accumulates the discrete sampled values from the IMU.
[0182] ;
[0183] in, The attitude angle at the initial moment is usually set as the zero vector, or determined by the initial alignment process; The pre-processed triaxial angular velocities; The time span for acquiring a single frame of point cloud data;
[0184] The index of the discrete sampling point. N is the total number of IMU sampling points within the single-frame point cloud acquisition time window; Let be the attitude angle at the k-th sampling time. The attitude angle at the (k-1)th sampling time, with an initial value. ; The preprocessed angular velocity at the k-th sampling time; The sampling time interval of the IMU (in seconds) is determined by the IMU sampling frequency.
[0185] By accumulating and integrating the IMU sampling points within the acquisition time window of each frame of point cloud, the attitude angle change trajectory of the point cloud at each moment from the start to the end of acquisition can be obtained, and finally the total attitude angle change at the end of acquisition relative to the start time can be obtained.
[0186] This step uses the Euler numerical integration method to transform continuous integration into discrete accumulation, enabling the algorithm to be implemented efficiently on embedded systems or computers. By calculating the attitude angle change frame by frame, the attitude change trajectory of the lidar during each frame acquisition can be accurately captured, providing fine attitude information for subsequent point cloud coordinate point-by-point correction.
[0187] In some specific embodiments, the attitude angle change is mapped to the lidar coordinate system, including:
[0188] Because the lidar and IMU sensor are rigidly connected on the hardware platform, their relative position and attitude relationship is constant.
[0189] Let I be the IMU coordinate system and L be the lidar coordinate system. What is the extrinsic rotation matrix between them? Obtained through offline calibration, it represents the rotational transformation from the IMU coordinate system to the lidar coordinate system.
[0190] The fixed extrinsic rotation matrix from the IMU coordinate system to the lidar coordinate system is obtained through offline calibration. ;
[0191] Using the ZYX Euler angle sequence, i.e., the heading-pitch-roll sequence, the attitude change rotation matrix of the IMU coordinate system at time t relative to the initial time is constructed. ;
[0192] ;
[0193] The attitude changes in the IMU coordinate system are mapped to the lidar coordinate system using the following formula:
[0194] ;
[0195] in, , , These are the basic rotation matrices for rotations about the X, Y, and Z axes, respectively:
[0196] ;
[0197] Let t be the rotation matrix of the IMU coordinate system relative to the initial time, derived from the attitude angles. Calculated; , , These refer to the rotation angles around the X, Y, and Z axes, respectively.
[0198] The fixed extrinsic parameter rotation matrix from the IMU coordinate system to the lidar coordinate system is obtained through calibration and satisfies... ;
[0199] for The inverse matrix of the rotation matrix is, since the rotation matrix is an orthogonal matrix, its inverse is equal to its transpose, i.e. ;
[0200] Let be the rotation transformation matrix of the lidar coordinate system at time t relative to the initial time.
[0201] This step precisely maps the attitude changes in the IMU coordinate system to the LiDAR coordinate system using an extrinsic rotation matrix. Since the LiDAR and IMU are rigidly connected, their relative attitude relationship is fixed, and the extrinsic matrix obtained through offline calibration ensures the accurate transmission of attitude change data. The rotation matrix is constructed using ZYX Euler angles, which conforms to common attitude description practices in engineering, facilitating understanding and implementation.
[0202] In some specific embodiments, the point cloud coordinates of the original point cloud data are corrected in real time using the mapping results, including:
[0203] Based on the rotation transformation matrix of the lidar coordinate system obtained by mapping The coordinates of the original point cloud collected by the lidar are corrected in real time.
[0204] Let the coordinates of the original point cloud collected by the lidar at time t be... The point is located in the instantaneous coordinate system of the lidar at the current moment. Due to dynamic movement, the point has rotational distortion relative to the initial reference coordinate system, which needs to be corrected to a unified reference coordinate system. The corrected point cloud coordinates are:
[0205] ;in, Let t be the rotation transformation matrix of the lidar coordinate system at time t relative to the initial time.
[0206] For all original points within a single frame of point cloud, corrections are performed based on the rotation transformation matrix corresponding to the acquisition time t. Since the acquisition time span of a single frame of point cloud is relatively short (usually tens of milliseconds), it is possible to choose to uniformly use the correction matrix at the end time for each frame for overall correction, or to perform point-by-point correction after interpolation for each point within the frame—the latter has higher accuracy but requires more computation.
[0207] Through the above correction process, all point cloud data collected by the lidar during dynamic movement are uniformly transformed to the reference coordinate system at the initial moment, effectively eliminating point cloud distortion caused by platform movement and solving problems such as point cloud ghosting and positional deviation.
[0208] In some specific embodiments, this is a data structure for storing and organizing points in a k-dimensional space, enabling efficient nearest neighbor search. This method uses the kd-tree algorithm to establish the spatial topological relationships of point cloud data and identifies and removes isolated outliers based on statistical features. The kd-tree algorithm removes isolated outliers by including:
[0209] Building point cloud data (Taken from the previous text) The three-dimensional kd-tree index structure, This represents the three-dimensional coordinates of the i-th point in the point cloud, where n is the total number of points in the point cloud;
[0210] For each point, search its neighborhood using a kd-tree. For any point in the point cloud, the nearest neighbor is given by the nearest neighbor. ,calculate To its k neighborhood points Euclidean distance:
[0211] ;
[0212] Calculate the Euclidean distance from the point to its k neighbors and find the average distance. The formula is as follows:
[0213] ;
[0214] Calculate the mean μ and standard deviation σ of the average distances between all points, and set a decision threshold. , where λ is the proportionality coefficient.
[0215] ;
[0216] ;
[0217] like If the point is isolated, it is considered an isolated outlier and is removed.
[0218] It can effectively identify and remove isolated noise points far from the main point cloud, reducing the impact of outliers on subsequent clustering analysis. The efficient neighborhood search capability of the kd-tree algorithm allows this step to maintain a relatively fast processing speed even on large-scale point clouds.
[0219] Step 3: For the corrected point cloud data, the kd-tree algorithm is used to remove isolated outlier noise points, and the K-means algorithm is used for clustering to remove edge noise points, thus completing the first denoising step.
[0220] In some specific embodiments, before using the K-means algorithm for clustering, the optimal number of clusters K is determined using the elbow rule based on the point cloud data after kd-tree removal.
[0221] Let the point cloud data after removing isolated outliers be... The total number of point clouds is ;
[0222] The elbow rule is as follows: calculate the sum of squared clustering errors (SSE) under different K values, and select the inflection point where the rate of decrease of SSE slows down as the optimal K value.
[0223] After removing isolated outliers using a kd-tree, the total amount of point cloud data decreases, and the number of clusters in the point cloud decreases accordingly. The K-means algorithm requires a pre-defined number of clusters K, and in traditional methods, the value of K is difficult to determine accurately. This invention utilizes the characteristics of the point cloud data after kd-tree removal to indirectly optimize the selection of the K value in the K-means algorithm.
[0224] The formula for calculating SSE is:
[0225] ;
[0226] in, This represents the m-th cluster; The center point of the m-th cluster is calculated using the following formula: ; For point To its respective cluster center The square of the Euclidean distance.
[0227] Since the kd-tree removal process reduces the interference of outliers, the clustering structure of point cloud data is clearer, and the elbow rule can more accurately determine the optimal K value, avoiding the subjectivity and blindness of K value selection in the traditional K-means algorithm.
[0228] In some specific embodiments, the K-means algorithm is used to cluster and remove edge noise points, including:
[0229] Point cloud data after removing outliers Perform K-means clustering, iteratively assigning points and updating cluster centers until convergence; the total number of point clouds is n′.
[0230] Step 301: From point cloud data K points are randomly selected as the initial cluster centers. ;
[0231] Step 302: For each point in the point cloud Calculate the Euclidean distance from each cluster center and assign it to the nearest cluster, as shown in the following formula:
[0232] ;
[0233] in, The midpoint of the t-th iteration The cluster number to which it belongs;
[0234] Step 303: Recalculate the centroid of each cluster:
[0235] ;
[0236] This represents the total number of points that this cluster has in the current round. This represents the center point of the m-th cluster in the t-th iteration. This represents the center point of the m-th cluster in the (t+1)-th iteration;
[0237] Step 304: Repeat steps 302 and 303 until the cluster centers no longer change or the preset maximum number of iterations is reached;
[0238] Step 305: After clustering is completed, for each cluster... Calculate the distance from each point within a cluster to its cluster center. The Euclidean distance is used to obtain the distance set, as shown in the following formula:
[0239] ;
[0240] For the specific point p of the m-th cluster i The corresponding distance value;
[0241] Calculate the mean of intra-cluster distances. and standard deviation The formula is as follows:
[0242] ;
[0243] ;
[0244] Step 306: Set the decision threshold within the cluster. :
[0245] ;
[0246] Step 307: If the distance from a point to the cluster center is greater than... If a point is identified as a cluster noise point, it will be removed, as shown in the following formula:
[0247] .
[0248] The K-means clustering analysis described above can effectively identify and remove noise points located at the edges of clusters. These points are usually non-isolated outliers generated during dynamic measurements and cannot be effectively identified by the kd-tree method. Removing these noise points completes the initial filtering of outliers.
[0249] In this method, the kd-tree algorithm and the K-means algorithm are executed sequentially and their order cannot be changed. Their logical relationship is as follows:
[0250] First, the kd-tree algorithm is executed: This utilizes the kd-tree's neighborhood search capability to identify and remove isolated outliers. These outliers are far from the main point cloud; retaining them would severely interfere with the initialization of K-means cluster centers, leading to distorted clustering results. Removing outliers using the kd-tree provides more structurally clear point cloud data for subsequent K-means algorithms.
[0251] Then, the K-means algorithm is executed: based on the removal of outliers using the kd-tree algorithm, the optimized K-means algorithm is used to perform cluster analysis on the remaining point cloud, removing noise points at the edges of clusters. Although these noise points are not isolated outliers, they still affect the surface quality of the point cloud and need to be further filtered out through cluster analysis.
[0252] The two algorithms complement each other: kd-tree excels at handling isolated noise far from the main cluster, while K-means excels at handling noise at the edges of clusters. Executing them sequentially achieves comprehensive and efficient removal of outlier noise points.
[0253] Step 4: For the point cloud data after the first denoising, the PointCleanNet deep learning network is first used for outlier removal and displacement projection denoising, and then an improved bilateral filter is used for smoothing to complete the second denoising. This achieves point cloud denoising and surface reconstruction in a noisy environment.
[0254] PointCleanNet employs a two-stage cascaded architecture: the first stage classifies and removes outliers, while the second stage estimates the displacement vectors and denoises the remaining point cloud. This architecture achieves accurate removal of point cloud noise through the synergistic effect of these three modules.
[0255] PointCleanNet is a point cloud denoising and outlier removal network proposed by Rakotosaona et al. in 2019 and published in the top computer graphics journal *Computer Graphics Forum*. This network is an improvement on the PCPNet (Point Cloud Properties Network) architecture and is specifically optimized for point cloud denoising tasks.
[0256] The main difference between PointCleanNet and existing denoising methods is:
[0257] 1. Limitations of traditional denoising methods: Traditional methods (such as MLS surface fitting, JETS fitting, bilateral filtering, etc.) heavily rely on local surface fitting or statistical noise model assumptions, require manual adjustment of multiple parameters (such as neighborhood size, smoothing intensity, etc.), and are difficult to adapt to different noise levels and surface types.
[0258] 2. Advantages of PointCleanNet: No need to pre-set noise model, no need for users to provide any parameters during testing, can automatically discover and maintain high curvature features, is robust to noise of different levels and types, and can handle large-scale dense point clouds.
[0259] In some specific embodiments, the PointCleanNet deep learning network is first used for outlier removal and displacement projection denoising, including:
[0260] The PointCleanNet deep learning network includes a four-element spatial transfer network, a multilayer perceptron with shared weights, a feature fusion module, and a regression module.
[0261] The Quaternion Spatial Transformer Network (QSTN) transforms the input point cloud to a canonical pose. After the point cloud is input, it is first added to the network. QSTN is a quaternion version of the standard Spatial Transformer Network, and its role is to learn the optimal pose transformation of the point cloud, transferring the original point cloud to a canonical pose that facilitates network learning. It includes:
[0262] Let the input point cloud be ,in QSTN outputs quaternion transformation parameters The corresponding rotation matrix is:
[0263] ; The transformed point cloud is as follows:
[0264] ;
[0265] in, Translation parameters
[0266] The multilayer perceptron processes the transformed point cloud. Extracting point-by-point features and then reconstructing the original pose space through inverse transformation includes:
[0267] The extracted feature vector is:
[0268] ;
[0269] Where d is the feature dimension;
[0270] After inverse transformation, the original pose space is restored as follows:
[0271] ;
[0272] in It is the inverse of the high-dimensional extended matrix, used to rotate the eigenvector field back to its original orientation;
[0273] The feature fusion module obtains permutation-invariant fused features by summing the feature vectors of neighboring points. Since point cloud data is unordered, the network needs to ensure that the output is insensitive to the order of the input points. The feature fusion module solves the problem of order stability in point cloud data by summing the feature vectors point-by-point along the point dimension and merging them into a unified feature vector. This includes:
[0274] Setting points The local neighborhood point set is The corresponding set of feature vectors is The feature fusion operation is as follows:
[0275]
[0276] This symmetric aggregation function ensures that the output feature Fi remains unchanged regardless of the input order of the neighboring points, thus satisfying the point cloud permutation invariance requirement.
[0277] The regression module first outputs a binary classification probability based on the fused features, identifies and removes outliers; then, it outputs a correction displacement vector for the retained valid points, projecting the noisy points onto a clean surface, including:
[0278] Phase 1: Outlier Classification and Removal
[0279] For each point The regression module outputs the network's predicted probabilities:
[0280] ;
[0281] in It is the Sigmoid activation function. For a fully connected layer with an output dimension of 2, if If the point is an outlier, it is determined to be an outlier and removed.
[0282] like If the value is 0, it is determined to be a valid point and retained for subsequent noise reduction;
[0283] Second stage: Displacement vector estimation and projection:
[0284] For the retained valid points, the regression module estimates the corrected displacement vector:
[0285] ;in For a fully connected layer with an output dimension of 3, the denoised point cloud coordinates are:
[0286] ;
[0287] This displacement vector accurately projects the noisy points onto the original clean point cloud surface, thus achieving the noise reduction function.
[0288] In some specific embodiments, improved bilateral filtering is used for smoothing. After PointCleanNet denoising, the point cloud may still contain slight high-frequency residual noise, requiring further smoothing through improved bilateral filtering while preserving geometric features. This includes:
[0289] The core formula for improving bilateral filtering is:
[0290] ;
[0291] in For point The normal vector, The weighting factor for the bilateral filter is calculated using the following formula:
[0292] in, For point The neighborhood point set is determined by kd-tree search;
[0293] For point With neighboring points The Euclidean distance between them;
[0294] , The intensity value or curvature feature at point i;
[0295] The dot product of the normal vector and the direction vector of the neighboring point represents the projection of the neighboring point onto the normal direction.
[0296] Spatial domain weighting function, using Gaussian kernel form:
[0297]
[0298] The formula for calculating d is as follows:
[0299] ;
[0300] in This is a spatial bandwidth parameter that controls the smoothing range of the spatial neighborhood.
[0301] The feature domain weight function also adopts the Gaussian kernel form:
[0302]
[0303] in This is the feature bandwidth parameter, which controls the sensitivity of feature similarity.
[0304] Spatial bandwidth and characteristic bandwidth Based on adaptive determination of point cloud neighborhood density:
[0305] ;
[0306] ;
[0307] in, , It is an empirical coefficient, and , These are used to adjust the scaling ratios of spatial bandwidth and feature bandwidth, respectively.
[0308] The number of neighboring points is as follows:
[0309] in Based on the number of basic neighborhood points, For local point cloud density, For global point cloud density, , This represents the upper and lower limits of the number of neighboring points.
[0310] In the normal vector calculation stage, the normal outlier factor (NOF) is introduced to determine whether neighborhood points should participate in the weight calculation:
[0311] ;
[0312] The normal vector of the query point. Let be the normal vector of the neighboring point j. The normal vector of the neighborhood point k;
[0313] like ( If the preset threshold is 2.0, then the point is determined to be a normal outlier, and the contribution of the neighboring point is removed in the bilateral filtering calculation.
[0314] In some specific implementations, PointCleanNet employs a two-stage joint training method, with the loss function consisting of two parts:
[0315] Outlier detection loss (cross-entropy loss):
[0316] ; in These are the true labels (1 indicates an outlier, 0 indicates a valid label). Predict probabilities for the network.
[0317] Denoising loss (Chamfer distance variant):
[0318] ; in For truly clean point clouds, This is the denoised point cloud. The loss function simultaneously measures the proximity of the denoised points to the real surface and the uniformity of the real surface coverage.
[0319] The total loss function is:
[0320] in This is the balance coefficient, usually taken as 1.0.
[0321] PointCleanNet's deep learning denoising and improved bilateral filtering are jointly applied in a sequential manner, forming a complementary enhancement mechanism:
[0322] First, PointCleanNet deep learning denoising is performed: it utilizes the learning ability of deep neural networks to identify and remove outliers, while estimating the displacement vector to project the noise points onto the clean surface, thus achieving significant noise suppression and outlier removal.
[0323] Improved bilateral filtering smoothing is then performed: Based on PointCleanNet denoising, improved bilateral filtering is used to smooth the residual high-frequency noise. At the same time, the geometric features of the point cloud are preserved through normal outlier removal and adaptive parameter selection mechanisms.
[0324] The two algorithms complement each other: PointCleanNet excels at handling large-amplitude noise and outliers, while the improved bilateral filter excels at handling small-amplitude high-frequency noise; the former requires no preset parameters but involves a large amount of computation, while the latter has adaptive parameters and high computational efficiency. Joint filtering fully leverages the advantages of both to achieve accurate denoising and surface reconstruction in noisy environments.
[0325] The experiment verified the following:
[0326] To verify the effectiveness, accuracy, and robustness of the three-dimensional data dynamic acquisition and processing method described in this invention, a combination of simulation experiments and actual dataset testing was used to verify each step of the method and its overall effect. The EuRoC dataset and TUM-VI dataset were used as the testing basis. Simultaneously, a real-world dynamic measurement hardware platform using a lidar + IMU was built for on-site data acquisition experiments. Specific experimental verification results are as follows:
[0327] (a) Experimental Data Set and Test Metrics
[0328] The EuRoC and TUM-VI datasets were selected as the foundation for algorithm training and testing. The EuRoC dataset contains 200Hz images and inertial data of a micro-aircraft, including 11 flight trajectories, and uses an uncalibrated ADIS16448 IMU. A laser tracker and motion capture system provide accurate ground truth data. The TUM-VI dataset consists of visual-inertial sequences from a handheld device in different scenarios, using an inexpensive BMI160 IMU (200Hz). After proper calibration, the ground truth data and IMU data are accurately time-synchronized. Simultaneously, an industrial-grade LiDAR + IMU dynamic measurement platform was built to collect dynamic point cloud data on-site at a large steel structure building, serving as real-world test data.
[0329] The test metrics selected were absolute azimuth error (AOE) and relative azimuth error (ROE) as the core evaluation metrics for point cloud coordinate correction. The mean square error, three-dimensional azimuth error, and yaw error between the true value and the algorithm estimate were calculated. Point cloud denoising accuracy, feature retention rate, and processing efficiency were selected as evaluation metrics for denoising effect. The denoising effect of traditional denoising algorithms and the algorithm of this invention were compared. The median error was selected as the dynamic measurement accuracy metric of the overall method to evaluate the measurement stability of the method under dynamic conditions.
[0330] (II) Verification Results of IMU Coordinate Correction
[0331] Azimuth error analysis was performed on the point cloud data after IMU coordinate correction according to this invention. The results show that the estimated value of the original uncalibrated IMU deviates from the true value in less than 10 seconds, which cannot meet the requirements of dynamic measurement. The IMU calibrated by the method of this invention, combined with the point cloud correction strategy of radar coordinate system mapping, achieved median errors of 1.34 degrees / minute and 0.68 degrees / minute on the EuRoC and TUM-VI datasets, respectively, which is 2-4 times more accurate than the traditional inertial correction method. Even in high motion dynamics scenarios, the corrected point cloud data can still maintain a high degree of matching with the true value, effectively eliminating point cloud distortion caused by dynamic movement. The problems of point cloud ghosting and position deviation are completely solved, proving that the coordinate correction method of this invention can accurately compensate for dynamic movement errors and is suitable for high dynamic measurement scenarios.
[0332] This application also proposes a dynamic measurement three-dimensional data acquisition and processing device, comprising:
[0333] The acquisition unit 201 is used to acquire pose data from the IMU sensor and point cloud data from the lidar. The IMU sensor is mounted on the 3D laser scanner and achieves hard time synchronization between the lidar and the IMU sensor.
[0334] The preprocessing unit 202 is used to preprocess the angular velocity data in the pose data, calculate the attitude angle change of the lidar during the acquisition process, map the attitude angle change to the lidar coordinate system, and use the mapping result to correct the point cloud coordinates of the original point cloud data in real time to eliminate the point cloud distortion caused by dynamic movement.
[0335] The first noise reduction unit 203 is used to sequentially remove isolated outlier noise points from the corrected point cloud data using the kd-tree algorithm and remove edge noise points using the K-means algorithm for clustering, thus completing the first noise reduction.
[0336] The second denoising unit 204 is used to perform outlier removal and displacement projection denoising on the point cloud data after the first denoising. Then, an improved bilateral filter is used for smoothing to complete the second denoising.
[0337] In other embodiments of the present invention, an electronic device 400 is disclosed, such as... Figure 3 As shown, the electronic device may include: one or more processors 401; a memory 402; a display 403; one or more application programs (not shown); and one or more computer programs 404. These devices can be connected via one or more communication buses 405. The one or more computer programs 404 are stored in the memory 402 and configured to be executed by the one or more processors 401. The one or more computer programs 404 include instructions that can be used to perform actions such as... Figure 1 And the steps in the corresponding embodiments.
[0338] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0339] In the various embodiments of the present invention, the functional units can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0340] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as flash memory, portable hard disk, read-only memory, random access memory, magnetic disk, or optical disk.
[0341] The above description is merely a specific implementation of the embodiments of the present invention, but the protection scope of the embodiments of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed in the embodiments of the present invention should be covered within the protection scope of the embodiments of the present invention. Therefore, the protection scope of the embodiments of the present invention should be determined by the protection scope of the claims.
Claims
1. A method for dynamically measuring three-dimensional data acquisition and processing, characterized in that, Includes the following steps: Step 1: Acquire pose data from the IMU sensor and point cloud data from the LiDAR. The IMU sensor is mounted on the 3D laser scanner, and hard synchronization of time between the LiDAR and the IMU sensor is achieved. Step 2: Preprocess the angular velocity data in the pose data, calculate the attitude angle change of the lidar during the acquisition process, map the attitude angle change to the lidar coordinate system, and use the mapping result to correct the point cloud coordinates of the original point cloud data in real time to eliminate the point cloud distortion caused by dynamic movement. Step 3: For the corrected point cloud data, the kd-tree algorithm is used to remove isolated outlier noise points, and the K-means algorithm is used for clustering to remove edge noise points, thus completing the first denoising step. Step 4: For the point cloud data after the first denoising, first use the PointCleanNet deep learning network to remove outliers and perform displacement projection denoising, and then use an improved bilateral filter for smoothing to complete the second denoising.
2. The method for dynamically measuring three-dimensional data acquisition and processing according to claim 1, characterized in that, The preprocessing includes zero-point offset correction and sliding window filtering; The zero-point offset correction is as follows: During the static period after system startup, the angular velocity output of the IMU sensor in a stationary state is collected and its average value is calculated as the zero-point offset. In subsequent dynamic acquisitions, this offset is subtracted from the original measurement value in real time, as shown in the following formula: ; The sliding window filtering is performed by smoothing the corrected angular velocity data using a moving average filter with a window length of 5, as shown in the following formula: ; in, This represents the raw angular velocity data output by the IMU sensor at time t. The corrected angular velocity data, This is the filtered angular velocity data.
3. The method for dynamically measuring three-dimensional data acquisition and processing according to claim 1, characterized in that, The method for calculating attitude angle changes is as follows: After obtaining the preprocessed angular velocity data, the angular velocity components along the x, y, and z axes in the IMU coordinate system are respectively... , , ; The start time of each frame of point cloud from the lidar As the starting point for integration, during the point cloud acquisition time frame of this frame... Integrating the preprocessed angular velocity, we obtain the attitude angle change at the end of the frame relative to the start time, as shown in the following formula: ; The integration employs the Euler numerical integration method, using discrete sampled values from the IMU for accumulation. ; in, The attitude angle at the initial moment is usually set as the zero vector, or determined by the initial alignment process; The pre-processed triaxial angular velocities; The time span for acquiring a single frame of point cloud data; The index of the discrete sampling point. N is the total number of IMU sampling points within the single-frame point cloud acquisition time window; Let be the attitude angle at the k-th sampling time. The attitude angle at the (k-1)th sampling time, with an initial value. ; The preprocessed angular velocity at the k-th sampling time; The sampling time interval of the IMU is determined by the IMU sampling frequency.
4. The method for dynamic measurement of three-dimensional data acquisition and processing according to claim 1, characterized in that, And mapping the attitude angle change to the lidar coordinate system, including: The fixed extrinsic rotation matrix from the IMU coordinate system to the lidar coordinate system is obtained through offline calibration. ; Using the ZYX Euler angle sequence, i.e., the heading-pitch-roll sequence, the attitude change rotation matrix of the IMU coordinate system at time t relative to the initial time is constructed. ; ; The attitude changes in the IMU coordinate system are mapped to the lidar coordinate system using the following formula: ; in, , , These are the basic rotation matrices for rotations about the X, Y, and Z axes, respectively: ; Let be the rotation matrix of the IMU coordinate system at time t relative to the initial time, derived from the attitude angles. Calculated; The fixed extrinsic parameter rotation matrix from the IMU coordinate system to the lidar coordinate system is obtained through calibration and satisfies... ; for The inverse matrix of the rotation matrix is, since the rotation matrix is an orthogonal matrix, its inverse is equal to its transpose, i.e. ; Let t be the rotation transformation matrix of the lidar coordinate system at time t relative to the initial time.
5. The method for dynamically measuring three-dimensional data acquisition and processing according to claim 4, characterized in that, The point cloud coordinates of the original point cloud data are corrected in real time using the mapping results, including: Let the coordinates of the original point cloud collected by the lidar at time t be... The corrected point cloud coordinates are: ;in, Let t be the rotation transformation matrix of the lidar coordinate system at time t relative to the initial time.
6. The method for dynamically measuring three-dimensional data acquisition and processing according to claim 1, characterized in that, The kd-tree algorithm removes isolated outliers by including: Building point cloud data The three-dimensional kd-tree index structure, This represents the three-dimensional coordinates of the i-th point in the point cloud, where n is the total number of points in the point cloud; For each point, search its neighborhood using a kd-tree. For any point in the point cloud, the nearest neighbor is given by the nearest neighbor. ,calculate To its k neighborhood points Euclidean distance: ; Calculate the distance from this point to its k-th point. Calculate the Euclidean distance between each neighboring point and the average distance. The formula is as follows: ; Calculate the mean μ and standard deviation σ of the average distances between all points, and set a decision threshold. , where λ is the proportionality coefficient. ; ; like If the point is isolated, it is considered an isolated outlier and is removed.
7. The method for dynamic measurement of three-dimensional data acquisition and processing according to claim 1, characterized in that, Before using the K-means algorithm for clustering, the optimal number of clusters K is determined using the elbow rule based on the point cloud data after kd-tree removal. Let the point cloud data after removing isolated outliers be... The total number of point clouds is ; The elbow rule is as follows: calculate the sum of squared clustering errors (SSE) under different K values, and select the inflection point where the rate of decrease of SSE slows down as the optimal K value. The formula for calculating SSE is: ; in, This represents the m-th cluster; The center point of the m-th cluster is calculated using the following formula: ; For point To its respective cluster center The square of the Euclidean distance.
8. The method for dynamically measuring three-dimensional data acquisition and processing according to claim 1, characterized in that, K-means clustering algorithm is used to remove edge noise points, including: Point cloud data after removing outliers Perform K-means clustering, iteratively assigning points and updating cluster centers until convergence; Step 301: From point cloud data K points are randomly selected as the initial cluster centers. ; Step 302: For each point in the point cloud Calculate the Euclidean distance from each cluster center and assign it to the nearest cluster, as shown in the following formula: ; in, The midpoint of the t-th iteration The cluster number to which it belongs; Step 303: Recalculate the centroid of each cluster: ; This represents the total number of points that this cluster has in the current round. This represents the center point of the m-th cluster in the t-th iteration. This represents the center point of the m-th cluster in the (t+1)-th iteration; Step 304: Repeat steps 302 and 303 until the cluster centers no longer change or the preset maximum number of iterations is reached; Step 305: After clustering is completed, for each cluster... Calculate the distance from each point within a cluster to its cluster center. The Euclidean distance is used to obtain the distance set, as shown in the following formula: ; For the specific point p of the m-th cluster i The corresponding distance value; Calculate the mean of intra-cluster distances. and standard deviation The formula is as follows: ; ; Step 306: Set the decision threshold within the cluster. : ; Step 307: If the distance from a point to the cluster center is greater than... If a point is identified as a cluster noise point, it will be removed, as shown in the following formula: 。 9. The method for dynamic measurement of three-dimensional data acquisition and processing according to claim 1, characterized in that, First, the PointCleanNet deep learning network is used for outlier removal and displacement projection denoising, including: The PointCleanNet deep learning network includes a four-element spatial transfer network, a multilayer perceptron with shared weights, a feature fusion module, and a regression module. (1) The four-element spatial transfer network transforms the input point cloud to a normalized pose, including: Let the input point cloud be ,in QSTN outputs quaternion transformation parameters The corresponding rotation matrix is: ; The transformed point cloud is as follows: ; in, Translation parameters (2) The multilayer perceptron transforms the point cloud Extracting point-by-point features and then recovering the original pose space through inverse transformation includes: The extracted feature vector is: ; Where d is the feature dimension; After inverse transformation, the original pose space is restored as follows: ; in It is the inverse of the high-dimensional extended matrix, used to rotate the eigenvector field back to its original orientation; (3) The feature fusion module obtains permutation-invariant fused features by summing the feature vectors of neighboring points, including: Setting points The local neighborhood point set is The corresponding feature vector set is The feature fusion operation is as follows: ; (4) The regression module first outputs a binary classification probability based on the fused features, determines and removes outliers; then it outputs a correction displacement vector for the retained valid points, projecting the noise points onto a clean surface, including: Phase 1: Outlier Classification and Removal For each point The regression module outputs the network's predicted probabilities: ; in It is the Sigmoid activation function. To output a fully connected layer with a dimension of 2, If the point is an outlier, it is determined to be an outlier and removed. like If the value is 0, it is determined to be a valid point and retained for subsequent noise reduction; Second stage: Displacement vector estimation and projection: For the retained valid points, the regression module estimates the corrected displacement vector: ; in For a fully connected layer with an output dimension of 3, the denoised point cloud coordinates are: ; This displacement vector accurately projects the noisy points onto the original clean point cloud surface, thus achieving the noise reduction function.
10. A method for dynamically measuring three-dimensional data acquisition and processing according to claim 9, characterized in that, Then, an improved bilateral filter is used for smoothing, including: The core formula for improving bilateral filtering is: ; in For point The normal vector, The weighting factor for the bilateral filter is calculated using the following formula: ; in, For point The neighborhood point set is determined by kd-tree search; For point With neighboring points The Euclidean distance between them; , Let i be the point and its neighboring points. Intensity value or curvature characteristics; The dot product of the normal vector and the direction vector of the neighboring point represents the projection of the neighboring point onto the normal direction. Spatial domain weighting function, using Gaussian kernel form: ; The formula for calculating d is as follows: ; in This is a spatial bandwidth parameter that controls the smoothing range of the spatial neighborhood. The feature domain weight function also adopts the Gaussian kernel form: ; in This is the feature bandwidth parameter, which controls the sensitivity of feature similarity. Spatial bandwidth and characteristic bandwidth Based on adaptive determination of point cloud neighborhood density: ; ; in, , It is an empirical coefficient, and , These are used to adjust the scaling ratios of spatial bandwidth and feature bandwidth, respectively. The number of neighboring points is as follows: in Based on the number of basic neighborhood points, For local point cloud density, For global point cloud density, , This represents the upper and lower limits of the number of neighboring points.
11. The method for dynamic measurement of three-dimensional data acquisition and processing according to claim 9, characterized in that, The training loss function of the PointCleanNet network includes outlier detection loss and denoising loss; The outlier detection loss is the cross-entropy loss: ; in These are the true labels (1 indicates an outlier, 0 indicates a valid point). Predict probabilities for the network. Denoising loss (Chamfer distance variant): ; in For truly clean point clouds, For the denoised point cloud, this loss function simultaneously measures the proximity of the denoised points to the real surface and the uniformity of the real surface coverage. n represents the point cloud after denoising. The number of points, m represents the true clean point cloud. The number of points in the middle; The total loss function is: ; in The balance coefficient is set to 1.
0.
12. A three-dimensional data acquisition and processing device for dynamic measurement, characterized in that, include: The acquisition unit is used to acquire pose data from the IMU sensor and point cloud data from the lidar. The IMU sensor is mounted on the 3D laser scanner and achieves hard time synchronization between the lidar and the IMU sensor. The preprocessing unit is used to preprocess the angular velocity data in the pose data, calculate the attitude angle change of the lidar during the acquisition process, map the attitude angle change to the lidar coordinate system, and use the mapping result to correct the point cloud coordinates of the original point cloud data in real time to eliminate the point cloud distortion caused by dynamic movement. The first noise reduction unit is used to remove isolated outlier noise points from the corrected point cloud data by sequentially using the kd-tree algorithm and the K-means algorithm to cluster and remove edge noise points, thus completing the first noise reduction. The second denoising unit is used to process the point cloud data after the first denoising. First, it uses the PointCleanNet deep learning network to remove outliers and perform displacement projection denoising. Then, it uses an improved bilateral filter for smoothing to complete the second denoising.
