Geometric registration method and system based on BIM (Building Information Modeling) and AR (Augmented Reality) real scene

Abstract

The invention relates to the technical field of highway engineering informatization, in particular to a geometric registration method and system based on a BIM model and an AR real scene. The method comprises the following steps: obtaining multi-source data of a construction site, and generating an AR real scene of the construction site according to the multi-source data; fusing multi-source data to obtain accurate parameters of the construction equipment; calculating a spatial transformation matrix of a construction site through accurate parameters of construction equipment, and obtaining an initial registration result of the BIM model and the AR real scene by combining engineering characteristics and the spatial transformation matrix; a dynamic error compensation mechanism is introduced, the initial registration result is corrected through the dynamic error compensation mechanism, and a local registration result is obtained; and constructing a global and local coordinate system mapping optimization model, and mapping the local registration result to a global coordinate system by using the global and local coordinate system mapping optimization model to obtain a global registration result. According to the method, a reliable spatial data basis is provided for highway engineering quality inspection, construction guidance and defect identification.

Description

Technical Field

[0001] The present invention relates to the technical field of informatization of highway engineering, and particularly relates to a geometric registration method and system based on a BIM model and AR reality. Background Technique

[0002] BIM technology has become an important tool for the whole life cycle management of highway engineering. Its functions such as three-dimensional visualization modeling, collision detection, and engineering quantity calculation have significantly improved the design rationality and construction efficiency. However, in the construction stage, the real-time matching of the BIM model and the actual on-site scene still faces challenges. In traditional methods, construction workers need to use two-dimensional drawings or static BIM models for spatial imagination, which is prone to construction errors due to understanding deviations and is difficult to adapt to the dynamic changes in complex construction environments, resulting in a large spatial deviation between the model and the reality. Although the existing AR (augmented reality) technology can superimpose virtual models onto the real scene, the registration accuracy is limited by sensor errors, environmental interference, and dynamic construction conditions, and it is difficult to meet the quality inspection requirements in terms of accuracy. In addition, the existing collaborative applications of AR and BIM mostly focus on indoor or simple-structured scenes, and the geometric registration effect is relatively poor for the requirements of highway engineering scenarios with a large scope, linear distribution, and variable geological conditions. Therefore, finding a method and system that are applicable to highway engineering scenarios with a large scope, linear distribution, and variable geological conditions and can quickly and accurately achieve the geometric registration of the BIM model and AR reality is an urgent problem to be solved at present. Summary of the Invention

[0003] In view of the deficiencies of the existing methods and the requirements of practical applications, in order to solve the problem of quickly and accurately realizing the geometric registration of the BIM model and AR reality. On the one hand, the present invention provides a geometric registration method based on a BIM model and AR reality, including the following steps: obtaining multi-source data of the construction site, generating an AR reality of the construction site according to the multi-source data; fusing the multi-source data to obtain accurate parameters of construction equipment; calculating a spatial transformation matrix of the construction site through the accurate parameters of the construction equipment, and combining engineering characteristics and the spatial transformation matrix to obtain an initial registration result of the BIM model and the AR reality; introducing a dynamic error compensation mechanism to correct the initial registration result through the dynamic error compensation mechanism to obtain a local registration result; constructing a global and local coordinate system mapping optimization model, and using the global and local coordinate system mapping optimization model to map the local registration result to the global coordinate system to obtain a global registration result.

[0004] The present invention optimizes the mapping relationship between the local coordinate system and the global geographic information system by integrating multi-source sensor data, constructing a dynamic spatial reference framework, introducing an adaptive feature matching and error compensation mechanism, and combining with the linear distribution characteristics of highway engineering, supports real-time dynamic calibration during construction, effectively reduces the influence of environmental interference, and realizes high-precision geometric registration between the BIM model and the AR real scene on the construction site, providing a reliable spatial data basis for quality inspection, construction guidance and defect identification in construction scenarios such as highway engineering with large scope, linear distribution and variable geological conditions.

[0005] Optionally, the step of fusing the multi-source data to obtain accurate parameters of the construction equipment includes the following steps: Obtain the predicted parameters of the construction equipment according to the monitoring data of the inertial measurement unit; correct the prediction error of the predicted parameters through the Kalman gain to obtain the accurate parameters of the construction equipment.

[0006] Optionally, the obtaining of the predicted parameters of the construction equipment according to the monitoring data of the inertial measurement unit satisfies the following formula: , where represents the predicted value of the system state of the construction equipment at time for k time, represents the three-dimensional position of the construction equipment at time, represents the three-dimensional velocity of the construction equipment at represents the time difference, represents the rotation matrix corresponding to the quaternion, represents the attitude of the construction equipment at represents k the acceleration at represents the gravitational acceleration, represents quaternion multiplication, represents the mapping from angular velocity to quaternion increment, represents k the angular velocity at

[0007] Optionally, the step of combining the engineering characteristics and the spatial transformation matrix to obtain the initial registration result between the BIM model and the AR real scene includes the following steps: Extract the key geometric features from the BIM model; perform an initial matching on the BIM model and the AR real scene according to the key geometric features.

[0008] Optionally, the initial matching of the BIM model and the AR real scene according to the key geometric features satisfies the following formula: , where represents minimizing the objective function, represents the rotation matrix, represents the translation vector, represents the number of all matching point pairs, represents the weight factor of the curvature similarity of the point pair and satisfies: , represents the curvature value of the th feature point of the BIM model, represents the curvature value of the th matching point of the AR real scene, represents the weight decay rate adjustment parameter, represents the feature point of the BIM model, represents the th matching point of the AR real scene. Based on the spatial characteristics of the linear structure of highway engineering, the present invention extracts key geometric features from the BIM model and performs multi-scale matching with the real scene point cloud at the construction site. At the same time, combined with dynamic disturbance monitoring and prediction, the adaptive correction of the registration parameters is realized, which is beneficial to improving the accuracy of the present invention.

[0009] Optionally, the introduction of a dynamic error compensation mechanism corrects the initial registration result through the dynamic error compensation mechanism to obtain a local registration result, including the following steps: Based on the monitoring data of the inertial measurement unit, obtain the instantaneous displacement of the construction equipment; construct a prediction model for the future displacement trend of the construction equipment, and combine the prediction model for the future displacement trend, the instantaneous displacement and the monitoring data to obtain the pose increment; adjust the transformation matrix according to the pose increment, and use the adjusted transformation matrix to obtain the local registration result.

[0010] Optionally, adjusting the transformation matrix according to the pose increment satisfies the following formula: , where represents the adjusted transformation matrix, represents the transformation matrix before adjustment, represents the 3×3 identity matrix, represents the rotation vector, represents that it is the rotation vector corresponding skew-symmetric matrix, represents the translation vector.

[0011] The present invention realizes error compensation through a real-time monitoring and prediction mechanism, eliminating dynamic interferences in the construction scenario such as mechanical vibrations and displacements of temporary facilities.

[0012] Optionally, the construction of the global and local coordinate system mapping optimization model satisfies the following formula: , where represents the minimization of the objective function, represents the set of rotation, translation, and scaling parameters to be optimized, represents the preset number of control point groups, represents the coordinate transformation function that maps points in the local coordinate system to the global coordinate system, represents the three-dimensional coordinates of the i-th control point in the local construction coordinate system, represents the three-dimensional coordinates of the i-th control point in the global geographic coordinate system, represents the square of the Euclidean distance, which is used to quantify the deviation between the transformed local point and the global point.

[0013] Optionally, using the global and local coordinate system mapping optimization model to map the local registration result to the global coordinate system to obtain the global registration result satisfies the following formula:

[0014] where represents the local coordinates, represents the inverse transformation matrix, represents all coordinates, represents the coordinates of the origin of the local coordinate system of the n-th segment in the global coordinate system. The present invention constructs a bidirectional mapping model and iteratively optimizes the transformation parameters, eliminating the cumulative error of segment registration, solving the non-linear mapping deviation problem between the global geographic coordinate system and the local construction coordinate system in large-scale highway projects, and achieving the geometric alignment consistency of the overall scenario.

[0015] Second aspect, to efficiently execute a geometric registration method based on a BIM model and AR reality provided by the present invention, the present invention further provides a geometric registration system based on a BIM model and AR reality, including a processor, an input device, an output device, and a memory. The processor, the input device, the output device, and the memory are interconnected. Among them, the memory is used to store a computer program, and the computer program contains program instructions. The processor is configured to call the program instructions to execute a geometric registration method based on a BIM model and AR reality as described in the first aspect of the present invention. The geometric registration system based on a BIM model and AR reality of the present invention has a compact structure and stable performance, and can stably execute a geometric registration method based on a BIM model and AR reality provided by the present invention, further improving the overall applicability and practical application ability of the present invention. Description of the Drawings

[0016] Figure 1 It is a flowchart of a geometric registration method based on a BIM model and AR reality provided by an embodiment of the present invention; Figure 2 It is a framework diagram of a geometric registration system based on a BIM model and AR reality provided by an embodiment of the present invention. Detailed Embodiments

[0017] The specific embodiments of the present invention will be described in detail below. It should be noted that the embodiments described here are only for illustrative purposes and are not used to limit the present invention. In the following description, in order to provide a thorough understanding of the present invention, a large number of specific details are elaborated. However, it is obvious to those of ordinary skill in the art that the present invention does not have to adopt these specific details. In other instances, well-known circuits, software, or methods are not specifically described to avoid obscuring the present invention.

[0018] Throughout the specification, the mention of "one embodiment", "embodiment", "one example", or "example" means that the specific features, structures, or characteristics described in connection with that embodiment or example are included in at least one embodiment of the present invention. Thus, the phrases "in one embodiment", "in an embodiment", "one example", or "example" appearing throughout the specification do not necessarily all refer to the same embodiment or example. In addition, specific features, structures, or characteristics can be combined in any appropriate combination and / or sub-combination in one or more embodiments or examples. In addition, those of ordinary skill in the art should understand that the drawings provided herein are for illustrative purposes only, and the drawings are not necessarily drawn to scale.

[0019] Please refer to Figure 1, to solve the problem of quickly and accurately achieving the geometric registration between the BIM model and the AR real scene. The present invention provides a geometric registration method based on the BIM model and the AR real scene. As Figure 1 shown, in one embodiment, the method includes the following steps: S1. Obtain multi-source data of the construction site, and generate an AR real scene of the construction site according to the multi-source data.

[0020] In the embodiment, the three-dimensional point cloud, the real scene model and the dynamic pose data of the construction site are synchronously collected by a lidar, an unmanned aerial vehicle photogrammetry and an inertial measurement unit (IMU). The multi-source data is denoised, coordinate-normalized and timestamp-aligned to generate a standardized input data set.

[0021] Specifically, a lidar (LiDAR) is used to obtain high-density point cloud data of the construction site, the scanning frequency ≥ 20 Hz, and the point cloud resolution ≤ 5 cm; an unmanned aerial vehicle is equipped with a multispectral camera and an RTK-GNSS module to generate an AR real scene of the construction site, and the positioning accuracy is ±2 cm; an inertial measurement unit (IMU) collects the angular velocity and acceleration pose data of the construction equipment at a frequency of 100 Hz.

[0022] In some other embodiments, when obtaining the multi-source data of the construction site, data preprocessing is also required, including point cloud denoising, coordinate normalization and timestamp alignment.

[0023] Point cloud denoising means that, based on statistical outlier removal (SOR), the abnormal points in the point cloud are removed. The average distance between the i-th point in the point cloud and its k neighborhood points is calculated, and the noise points deviating more than 3 times the standard deviation from the mean value are removed by setting a threshold.

[0024] , , the removal condition: , where is the average Euclidean distance between the i-th point in the point cloud and k points in its neighborhood, and N is the total number of points in the point cloud.

[0025] Coordinate normalization means that the BIM model (WGS84 coordinate system), the point cloud (local coordinate system) and the IMU data (device coordinate system) are uniformly converted to the construction global coordinate system (CGCS2000).

[0026] Timestamp alignment means that the PTP (Precision Time Protocol) protocol is used to realize the synchronization of multi-sensor data, and the time deviation ≤ 1 ms to ensure the data time sequence consistency.

[0027] Data preprocessing eliminates the heterogeneity of multi-source data through denoising, coordinate unification, and time series synchronization, providing accurate and low-noise standardized input for subsequent registration.

[0028] S2. Integrate the multi-source data to obtain accurate parameters of the construction equipment.

[0029] In the embodiment, the step of integrating the multi-source data in step S2 to obtain accurate parameters of the construction equipment includes the following steps: S21. Obtain the predicted parameters of the construction equipment according to the monitoring data of the inertial measurement unit.

[0030] First, define the global geographic coordinate system, the local construction coordinate system, and the equipment movement coordinate system.

[0031] The global geographic coordinate system (GIS) refers to integrating the BIM model design parameters (road centerline coordinates, elevation data) based on the CGCS2000 coordinate system; The local construction coordinate system refers to defining the tangent direction along the road centerline as the x-axis, the normal direction as the y-axis, and the direction perpendicular to the ground as the z-axis based on the highway mileage section, forming a linear reference system; The equipment movement coordinate system refers to the IMU sensor bound to the construction machinery, with the origin at the centroid of the equipment and the x-axis pointing in the forward direction of the equipment.

[0032] Next, establish a piecewise linear mapping relationship, extract the parameter equation of the highway centerline from the BIM model, and express the three-dimensional coordinates of the centerline with the arc length s as the independent variable as: , where s is the arc length value corresponding to the mileage. Divide the construction scene into several mileage intervals, and the mapping relationship between the local coordinate system and the global coordinate system within each interval is described by the homogeneous transformation matrix : , where is the rotation matrix, is the position of the origin of the i-th local coordinate system in the global coordinate system.

[0033] Then, perform short-term state speculation based on the monitoring data of the inertial measurement unit.

[0034] The construction equipment is constantly moving during the construction process. The IMU measures the acceleration and angular velocity data 100 times per second. These data are used to predict the "next" state of the equipment, and the state content needs to predict the three-dimensional position of the construction equipment, the three-dimensional velocity , and the attitude (the attitude is represented by a quaternion, similar to the three-dimensional rotation angle).

[0035] The state of the construction equipment can be expressed as:

[0036] Furthermore, the acceleration measured by the inertial measurement unit and the angular velocity .

[0037] Furthermore, the position change is obtained by multiplying the current velocity by the time interval to estimate the position at the next moment. For example, if the device moves eastward at a speed of 1 meter per second, the position increases by 0.1 meter eastward after 0.1 second. The velocity change is estimated by multiplying the acceleration measured by the IMU by the time interval. For example, if the acceleration is 2 meters per second squared, the velocity increases by 0.2 meters per second after 0.1 second. The attitude change calculates the change in the quaternion attitude through the angular velocity of the IMU (how fast the device rotates) and updates the orientation of the device. For example, if the device turns left, the attitude parameters are adjusted accordingly.

[0038] Specifically, obtaining the prediction parameters of the construction equipment according to the monitoring data of the inertial measurement unit satisfies the following formula: , where represents the predicted value of the system state of the construction equipment at time for k time, represents the three-dimensional position of the construction equipment at time, represents the three-dimensional velocity of the construction equipment at represents the time difference, represents the rotation matrix corresponding to the quaternion, which is used to transform the acceleration from the device coordinate system to the global coordinate system, represents the attitude of the construction equipment at k time, represents the acceleration at time, represents k the gravitational acceleration, .

[0039] Furthermore, the mapping from angular velocity to quaternion increment satisfies:

[0040] Furthermore, the observation data includes the position after LiDAR matching and GNSS positioning , and the observation equation is: , where Represents zero-mean Gaussian observation noise.

[0041] Furthermore, the state transition Jacobian matrix satisfies: , and The partial derivatives of position and velocity with respect to the quaternion attitude need to be calculated through the Lie algebra perturbation model.

[0042] Among them, Represents the skew-symmetric matrix of angular velocity.

[0043] The observation Jacobian matrix satisfies: , The process noise covariance satisfies: , where and Are the noise standard deviations of the accelerometer and gyroscope respectively.

[0044] S22. Correct the prediction error of the predicted parameters through the Kalman gain to obtain the accurate parameters of the construction equipment.

[0045] The prediction result will deviate due to IMU noise and needs to be corrected through the actual measurement data of LiDAR and GNSS: First, scan the surrounding environment through LiDAR, match the feature points of the BIM model, and calculate the actual position of the equipment (accuracy about 5 cm). Secondly, the drone provides high-precision global positioning (horizontal accuracy ±2 cm), but it may be temporarily invalid due to weather or occlusion. Finally, directly provide the real-time orientation of the equipment (such as pitch angle, roll angle).

[0046] Compare the predicted position and attitude with the actual measurement values of LiDAR and GNSS to obtain the difference (for example, the predicted position is 1.2 meters eastward, the actual measurement is 1.15 meters, and the residual is 0.05 meters). Determine whether to trust the predicted value or the observed value through the Kalman gain. For example: if the GNSS signal is stable (small error), then trust the GNSS data more and greatly correct the prediction result; if LiDAR fails to match due to dust interference, then reduce its weight and mainly rely on IMU prediction and GNSS data.

[0047] Adjust the position, speed and attitude parameters of the construction equipment according to the residual and the Kalman gain to obtain a better estimated value, that is, the accurate parameters of the construction equipment.

[0048] Furthermore, if GNSS suddenly fails, the Extended Kalman Filter (EKF) automatically increases the weights of LiDAR and IMU to ensure positioning continuity. In response to the above abnormal situation, then update the transformation matrix 100 times per second to reflect the movement and attitude changes of the equipment in real time, providing an accurate spatial reference for the AR model superposition.

[0049] The Extended Kalman Filter quickly senses the motion trend of the device through the IMU, and then combines the field measurement data of LiDAR and GNSS to dynamically balance the errors between prediction and measurement, and finally outputs reliable position, velocity, and attitude information. This process effectively solves the problem of insufficient accuracy of a single sensor in the construction environment and lays a real-time and adaptive spatial reference for geometric registration.

[0050] S3. Calculate the spatial transformation matrix of the construction site through the precise parameters of the construction equipment, and combine the engineering characteristics and the spatial transformation matrix to obtain the initial registration result of the BIM model and the AR real scene.

[0051] In the embodiment, calculating the spatial transformation matrix of the construction site through the precise parameters of the construction equipment means obtaining the spatial transformation matrix based on the dynamic spatial reference framework of the geographic coordinate system (GIS), the local construction coordinate system, and the equipment movement coordinate system, and fusing the precise parameters of the construction equipment through the Extended Kalman Filter.

[0052] Furthermore, the step of combining the engineering characteristics and the spatial transformation matrix to obtain the initial registration result of the BIM model and the AR real scene includes the following steps: S31. Extract the key geometric features from the BIM model.

[0053] Specifically, the key geometric features include the highway center line, the slope contour, the stake number marking points, and the curvature-normal vector features of the slope contour.

[0054] Enhance the feature distinctiveness by calculating the curvature C of the local feature points of the point cloud and the normal vector n, where the curvature is determined by the ratio of the eigenvalues of the covariance matrix, and the normal vector is solved by the principal component analysis (PCA) for the direction corresponding to the minimum eigenvalue.

[0055] S32. Perform an initial matching of the BIM model and the AR real scene according to the key geometric features.

[0056] Voxelize and downsample the AR real scene point cloud (voxel size 0.1m), extract the curvature-normal vector features corresponding to the BIM, and perform an initial registration of the BIM features and the downsampled AR real scene point cloud by the iterative closest point (ICP).

[0057] Furthermore, the initial matching of the BIM model and the AR real scene according to the key geometric features satisfies the following formula: , where represents the minimization of the objective function, represents the rotation matrix, represents the translation vector, represents the number of all matching point pairs, Represents the weight factor of the point pair curvature similarity and satisfies: , represents the curvature value of the th feature point of the BIM model, represents the curvature value of the th matching point of the AR real scene, represents the th feature point of the BIM model, represents the th matching point of the AR real scene.

[0058] S4. Introduce a dynamic error compensation mechanism to correct the initial registration result through the dynamic error compensation mechanism and obtain a local registration result.

[0059] After the initial matching of local features is completed, dynamic interferences in the construction scene, such as mechanical vibrations and displacements of temporary facilities, may still cause continuous deviation of the registration parameters, and error compensation needs to be achieved through a real-time monitoring and prediction mechanism.

[0060] Specifically, the step of introducing a dynamic error compensation mechanism to correct the initial registration result through the dynamic error compensation mechanism and obtain a local registration result includes the following steps: S41. Obtain the instantaneous displacement of the construction equipment based on the monitoring data of the inertial measurement unit.

[0061] The IMU collects the angular velocity ω and acceleration a of the construction equipment at a frequency of 100 Hz. After separating the gravity component g through integral operation, the instantaneous displacement of the equipment is calculated : , and this value reflects the pose offset caused by vibration or collision of the equipment.

[0062] S42. Construct a future displacement trend prediction model of the construction equipment, and obtain a pose increment by combining the future displacement trend prediction model, the instantaneous displacement, and the monitoring data.

[0063] Specifically, the future displacement trend prediction model is an LSTM network model.

[0064] Input the pose sequence within the past 10 seconds , and output the displacement prediction value for the next 3 seconds . The network structure is 3-layer LSTM, with 64 nodes in the hidden layer, and the training uses the mean squared error (MSE) loss function.

[0065] Generate a pose increment through the weighted fusion of the prediction result and the measured value of the IMU, satisfying: , where represents the pose increment, represents the rotation vector, Indicates the translation vector.

[0066] S43. Adjust the transformation matrix according to the pose increment, and obtain the local registration result by using the adjusted transformation matrix.

[0067] Specifically, the adjustment of the transformation matrix according to the pose increment satisfies the following formula: , where represents the adjusted transformation matrix, represents the transformation matrix before adjustment, represents the 3×3 identity matrix, represents the rotation vector, indicates that it is the rotation vector corresponding skew-symmetric matrix, represents the translation vector.

[0068] Furthermore, use the adjusted transformation matrix to adjust the initial registration result obtained in the previous step to obtain the local registration result.

[0069] In the embodiment, the displacement threshold of the construction equipment is set to 5 cm. If the cumulative instantaneous displacement of the equipment caused by continuous vibration within 1 second is greater than the displacement threshold, i.e., 5 cm, the registration parameters need to be corrected; otherwise, no correction is required.

[0070] S5. Construct a global and local coordinate system mapping optimization model, and use the global and local coordinate system mapping optimization model to map the local registration result to the global coordinate system to obtain the global registration result.

[0071] On the basis of ensuring the stability of local registration through dynamic error compensation, it is necessary to solve the non-linear mapping deviation problem between the global geographic coordinate system and the local construction coordinate system in large-scale highway projects to achieve geometric alignment consistency of the overall scene. By constructing a global and local coordinate system mapping optimization model and iteratively optimizing the transformation parameters, the cumulative error of segmented registration is eliminated to ensure the registration accuracy.

[0072] Specifically, the highway project has a linear distribution characteristic. Its local construction coordinate system is based on the pile number segmentation, and the coordinate systems of each segment are associated with the global geographic coordinate system (CGCS2000) through the homogeneous transformation matrix .

[0073] The constructed global and local coordinate system mapping optimization model satisfies the following formula: , where represents the minimization of the objective function, represents the set of rotation, translation, and scaling parameters to be optimized, represents the preset number of control point groups, represents the coordinate transformation function that maps the local coordinate system points to the global coordinate system, represents the three-dimensional coordinates of the i-th control point in the local construction coordinate system, represents the three-dimensional coordinates of the i-th control point in the global geographic coordinate system, represents the square of the Euclidean distance, which is used to quantify the deviation between the local point and the global point after transformation.

[0074] Furthermore, the Levenberg-Marquardt algorithm is used for iterative solution. This algorithm dynamically adjusts the damping factor to balance the convergence of the gradient descent and the Gauss-Newton method: , where is the Jacobian matrix of the residual r with respect to the parameter , is the parameter increment, and the parameters are iteratively updated until the residual converges.

[0075] If the residual decreases, then ; If the residual increases, then .

[0076] Furthermore, the local registration result is mapped to the global coordinate system by using the global and local coordinate system mapping optimization model to obtain the global registration result, which satisfies the following formula:

[0077] where represents the local coordinates, represents the inverse transformation matrix, represents all coordinates, represents the coordinates of the origin of the local coordinate system of the i-th segment in the global coordinate system. Due to construction errors and environmental factors, there are non-linear deviations in the actual coordinate transformation, and the mapping parameters need to be optimized through control points.

[0078] Even further, after completing the mapping optimization of the global and local coordinate systems, the highly accurate registered BIM model and the AR real scene need to be superimposed on the construction terminal, and interactive operations are performed to provide feedback for guiding on-site operations.

[0079] Please refer to Figure 2, in an embodiment, to efficiently execute a geometric registration method based on a BIM model and AR reality provided by the present invention, the present invention further provides a geometric registration system based on a BIM model and AR reality, including: an input device, an output device, a processor, and a memory. The input device, output device, processor, and memory are interconnected. The memory contains program instructions for the steps of the geometric registration method based on a BIM model and AR reality. The geometric registration system based on a BIM model and AR reality of the present invention has a compact structure and stable performance, and can stably execute the geometric registration method based on a BIM model and AR reality of the present invention, further improving the overall applicability and practical application ability of the present invention.

[0080] In an embodiment, the so-called processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The input device may be used to obtain data information. The output device may be used to output the result obtained from the program instructions included in the computer program stored in the memory provided by the present invention. The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A part of the memory may also include a non-volatile random access memory.

[0081] In a possible implementation, the memory may include a program storage area and a data storage area. Among them, the program storage area may store an operating system and application programs required for at least one function, etc.; the data storage area may store data created during use. In addition, the memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A part of the memory may also include NVRAM. The memory stores an operating system and operation instructions, executable modules, or data structures, or subsets thereof, or extended sets thereof. Among them, the operation instructions may include various operation instructions for implementing various operations. The operating system may include various system programs for implementing various basic tasks and processing hardware-based tasks.

[0082] The embodiment also provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above geometric registration method based on the BIM model and the AR real scene are implemented.

[0083] The storage medium may include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs.

[0084] In summary, the present invention combines multi-source sensor data and constructs a dynamic spatial reference framework, introduces an adaptive feature matching and error compensation mechanism, optimizes the mapping relationship between the local coordinate system and the global geographic information system in combination with the linear distribution characteristics of highway engineering, supports real-time dynamic calibration during construction, effectively reduces the influence of environmental interference, and realizes high-precision geometric registration between the BIM model and the AR real scene at the construction site, providing a reliable spatial data basis for quality inspection, construction guidance, and defect identification in construction scenarios such as highway engineering with a large scope, linear distribution, and variable geological conditions.

[0085] Therefore, the present invention effectively overcomes various disadvantages in the prior art and has high industrial utilization value.

[0086] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered by the scope recorded in the present invention.

Claims

1. A geometric registration method based on BIM model and AR real scene, characterized in that: The following steps are involved: Acquire multi-source data of the construction site, and generate an AR real scene of the construction site according to the multi-source data; Fusing the multi-source data to obtain accurate parameters of construction equipment; The spatial transformation matrix of the construction site is solved by using the precise parameters of the construction equipment, and the initial registration result of the BIM model and the AR real scene is obtained by combining the engineering characteristics and the spatial transformation matrix; Introducing a dynamic error compensation mechanism, and correcting the initial registration result through the dynamic error compensation mechanism to obtain a local registration result; A global and local coordinate system mapping optimization model is constructed, and the local registration result is mapped to the global coordinate system using the global and local coordinate system mapping optimization model to obtain a global registration result.

2. The geometric registration method based on BIM model and AR real scene according to claim 1 is characterized in that: The method of fusing the multi-source data to obtain accurate parameters of construction equipment includes the following steps: Obtain predicted parameters of construction equipment based on monitoring data from the inertial measurement unit; The prediction error of the prediction parameter is corrected by the Kalman gain to obtain the precise parameters of the construction equipment.

3. The geometric registration method based on BIM model and AR real scene according to claim 2 is characterized in that: The predicted parameters of the construction equipment are obtained based on the monitoring data of the inertial measurement unit, satisfying the following formula: ,in, Indicated in Time to time k The predicted value of the construction equipment system status at the moment, express The three-dimensional position of the construction equipment at any moment, express The three-dimensional speed of the construction equipment at any given moment, Indicates the time difference, Represents the rotation matrix corresponding to the quaternion, express The posture of construction equipment at all times, express k The acceleration of time, represents the acceleration due to gravity, represents quaternion multiplication, represents the mapping from angular velocity to quaternion increment, express k Angular velocity at time.

4. The geometric registration method based on BIM model and AR real scene according to claim 1 is characterized in that: The combining of the engineering characteristics and the space transformation matrix to obtain the initial registration result of the BIM model and the AR real scene includes the following steps: extracting key geometric features from the BIM model; The BIM model and the AR real scene are initially matched according to the key geometric features.

5. The geometric registration method based on BIM model and AR real scene according to claim 4 is characterized in that: The initial matching of the BIM model and the AR real scene according to the key geometric features satisfies the following formula: ,in, represents the minimization of the objective function, represents the rotation matrix, represents the translation vector, Represents the number of all matching point pairs, Indicates The weight factor of the curvature similarity of the point pair satisfies: , Indicates the BIM model The curvature value of feature points, Indicates AR real scene The curvature value of the matching points, represents the weight decay speed adjustment parameter, Represents the BIM model Feature points, Indicates AR real scene Matching points.

6. The geometric registration method based on BIM model and AR real scene according to claim 1 is characterized in that: The introducing of the dynamic error compensation mechanism, and correcting the initial registration result by the dynamic error compensation mechanism to obtain the local registration result, comprises the following steps: Based on the monitoring data of the inertial measurement unit, the instantaneous displacement of the construction equipment is obtained; Constructing a future displacement trend prediction model for the construction equipment, and obtaining a posture increment by combining the future displacement trend prediction model, the instantaneous displacement and the monitoring data; The transformation matrix is ​​adjusted according to the posture increment, and the local registration result is obtained using the adjusted transformation matrix.

7. The geometric registration method based on BIM model and AR real scene according to claim 6 is characterized in that: The transformation matrix is ​​adjusted according to the posture increment to satisfy the following formula: ,in, represents the adjusted transformation matrix, represents the transformation matrix before adjustment, represents the 3×3 identity matrix, represents the rotation vector, Represents a rotation vector The corresponding antisymmetric matrix is, Represents the translation vector.

8. The geometric registration method based on BIM model and AR real scene according to claim 1 is characterized in that: The global and local coordinate system mapping optimization model is constructed to satisfy the following formula: ,in, represents the minimization of the objective function, represents the set of rotation, translation and scaling parameters to be optimized, Indicates the number of preset control point groups, represents the coordinate transformation function, which maps the local coordinate system points to the global coordinate system. represents the three-dimensional coordinates of the i-th control point in the local construction coordinate system, represents the three-dimensional coordinates of the i-th control point in the global geographic coordinate system, Represents the square of the Euclidean distance, which is used to quantify the deviation between the local point and the global point after transformation.

9. The geometric registration method based on BIM model and AR real scene according to claim 1 is characterized in that: The local registration result is mapped to the global coordinate system by using the global and local coordinate system mapping optimization model to obtain a global registration result, which satisfies the following formula: ,in, represents the local coordinates, represents the inverse transformation matrix, Represents all coordinates, Indicates The coordinates of the origin of the segment's local coordinate system in the global coordinate system.

10. A geometric registration system based on BIM model and AR real scene, characterized in that: The geometric registration system based on BIM model and AR real scene includes: an input device, an output device, a processor, and a memory. The input device, the output device, the processor, and the memory are interconnected. The memory includes program instructions, and the program instructions are used to execute the geometric registration method based on BIM model and AR real scene according to any one of claims 1 to 9.

Images