An augmented reality BIM model drift dynamic correction method and device based on structural constraint deviation

Abstract

The application belongs to the technical field of augmented reality, and discloses an augmented reality BIM model drift dynamic correction method and equipment based on structural constraint deviation, which constructs a neighborhood covariance matrix of point clouds and performs eigenvalue decomposition, extracts line features or plane features, and then clusters to form point cloud structural features; under the constraint of a view cone, the point cloud structure is matched with the BIM structure in terms of direction consistency and distance constraint; under the condition of meeting the stable structure constraint, whether the BIM model has accumulated drift and triggers the pose optimization solution is judged by combining the K times of calculation results in the sliding time window; the two-stage BIM pose optimization solution of rotation first and then translation is adopted, the Lie algebra disturbance quantity is introduced in the rotation stage, and the optimal rotation component is iteratively solved, the optimal rotation component is constructed in the translation stage, and then the optimal pose after position calibration is solved, the pose of the BIM model in the AR world coordinate system is updated based on the optimal pose, and the BIM model drift correction is realized.

Description

Technical Field

[0001] This invention belongs to the field of augmented reality technology, and more specifically, relates to a method and device for dynamic correction of drift of augmented reality BIM model based on structural constraint deviation. Background Technology

[0002] Augmented reality (AR) technology can seamlessly integrate and overlay computer-generated virtual digital information with real-world scenes in real time, allowing users to intuitively perceive the fused information within a real environment and achieving immersive interaction between real and virtual information. In the construction industry, AR technology overlays digital information such as Building Information Models (BIM) onto the construction site, enabling visualized guidance of the construction process and detection of construction deviations. However, when navigating long distances on complex and large construction sites, cumulative pose errors can easily occur, leading to problems such as the BIM model gradually shifting away from the actual building and poor model registration stability.

[0003] Existing research on AR applications in the construction field mainly focuses on the initial registration between BIM models and on-site point cloud data, lacking a cumulative error correction mechanism for AR roaming to some extent. Regarding BIM model overlay offset correction, while some studies have introduced methods such as geometric feature matching, point cloud registration optimization, and multi-sensor fusion to alleviate model offset problems, they largely rely on continuous optimization or global update strategies, lacking the ability to identify error changes during roaming and trigger correction mechanisms, making it difficult to promptly identify when model offset occurs. Summary of the Invention

[0004] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a method and device for dynamic correction of drift in augmented reality BIM models based on structural constraint deviations. It aims to solve the problem of improving the robustness, real-time performance, and accuracy of cumulative drift correction in the registration of BIM models with real-world scenes under conditions of limited mobile AR resources and complex environments.

[0005] To achieve the above objectives, according to one aspect of the present invention, a method for correcting the drift dynamics of an augmented reality BIM model based on structural constraint deviations is provided, comprising the following steps: (1) The three-dimensional structural elements of the BIM model are mapped from the BIM local coordinate system to the AR world coordinate system through the initial registration matrix, and the three-dimensional structural elements are parametrically expressed in the AR world coordinate system. (2) Based on the camera intrinsic parameter matrix, the depth image within the current view frustum range of the mobile terminal during AR roaming is back-projected to map the pixel points to the three-dimensional space under the camera coordinate system, generating local point cloud data within the current view frustum range of the camera. After mapping the local point cloud data to the AR world coordinate system using the current device tracking pose, a lightweight point cloud is obtained. (3) Based on the obtained lightweight point cloud, a neighborhood covariance matrix is ​​constructed, and eigenvalue decomposition is performed on the neighborhood covariance matrix. Based on the obtained eigenvalues, a feature significance index is constructed, and discrete disordered points and noise points in the lightweight point cloud are removed according to the feature significance index. Then, line feature points and planar feature points are extracted based on the feature significance index and the lightweight point cloud. Finally, line feature points and planar feature points belonging to the same planar structure or line structure are merged to obtain a stable set of planar structure and line structure features, which are collectively referred to as the point cloud structure set. (4) In the AR world coordinate system, the BIM model is clipped according to the view frustum space range determined by the current mobile terminal pose to obtain the local structure set; the point cloud structure set extracted within the current view frustum space range is matched with the local structure set for directional consistency and distance constraint matching. When there is a point cloud structure in the point cloud structure set that corresponds to multiple BIM structures in the local structure set, the BIM structure with the smallest distance from the corresponding point cloud structure is selected as the matching result. Then, the point cloud structure and the matching result form a matching pair. The directional deviation and position deviation between the point cloud structure and the BIM structure in the matching pair are calculated respectively. The directional deviation and position deviation obtained are weighted and fused using the structural feature significance index as the weight coefficient to obtain the comprehensive directional deviation and comprehensive position deviation expressions used to characterize the current BIM model superposition state. (5) When the number of matching pairs successfully participating in the matching within the current camera view cone exceeds the preset minimum number threshold, and the comprehensive position deviation or comprehensive direction deviation calculated for K consecutive times within the sliding time window exceeds its allowable upper limit, it is determined that the BIM model superposition has accumulated offset. (6) The BIM model pose optimization solution is divided into a pure rotation optimization stage and a pure translation optimization stage. In the pure rotation optimization stage, Lie algebra perturbation variables are introduced to solve for the optimal pose of the BIM model after orientation correction. Then, the rotation component in the optimal pose of the BIM model is locked and pure translation optimization is performed to obtain the translation correction amount, and then the optimal pose after position calibration is determined. Based on the optimal pose after position calibration, the pose of the BIM model in the AR world coordinate system is updated to realize the BIM model drift correction.

[0006] Furthermore, in the pure rotation optimization direction stage, Lie algebra perturbation variables are introduced to construct the direction correction objective function and the direction correction normal equation. The direction correction normal equation is solved iteratively. In each iteration, the local optimal rotation increment is obtained. The effectiveness of the local optimal rotation increment obtained in this iteration is evaluated according to the direction correction objective function. If it is effective, it is applied to the current BIM model pose; otherwise, the solution is discarded and the next iteration is entered. The optimal pose of the BIM model after direction correction is determined when the iteration termination condition is met.

[0007] Furthermore, after locking the rotation component in the optimal pose of the BIM model, pure translation optimization is performed to construct the position correction objective function and the corresponding position correction normal equation. The translation correction amount is obtained by solving the position correction normal equation, and then the optimal pose after position calibration is determined. Based on the optimal pose after position calibration, the pose of the BIM model in the AR world coordinate system is updated to realize the drift correction of the BIM model.

[0008] Furthermore, in the pure rotation optimization stage, Lie algebraic rotation perturbation variables are introduced, and the perturbed rotation matrix is ​​obtained through exponential mapping. An orientation correction objective function is constructed, the rotation Jacobian matrix of the matching structure is calculated, and an orientation correction normal equation with a dynamic damping factor is constructed. The optimal rotation correction amount is obtained through iterative solution. During the iteration process, if the error decreases, the pose update is accepted and the damping factor is reduced; if the error increases, the update is rejected and the damping factor is increased before resolving, until the iteration termination condition is met. In the pure translation optimization stage, the rotation matrix after rotation correction is locked, a position correction objective function is constructed, the translation Jacobian matrix is ​​a constant matrix, and a position correction normal equation with a dynamic damping factor is constructed. The Cholesky decomposition method is used to solve the position correction normal equation, and the globally optimal translation correction amount is calculated.

[0009] Furthermore, the final pose correction matrix is ​​a homogeneous transformation matrix obtained by fusing rotation correction and translation correction. This correction matrix is ​​then used to update the world pose matrix of the BIM model in the AR engine, thereby achieving dynamic repositioning and overlay drift correction of the BIM model.

[0010] Furthermore, after orientation and position calibration, the pose correction matrix of the final BIM model is obtained. :

[0011] In the formula, The rotation matrix after orientation correction is obtained by iterative solution in the pure rotation optimization stage; In lock Then, the translation vector after position correction is obtained through pure translation optimization; [0,0,0].

[0012] Furthermore, extract the pose of the current BIM model. T Rotation matrix components in R old , to rotation matrix components R old Introducing Lie algebra pure rotational perturbation variables The perturbed BIM model pose is obtained and applied to the current AR scene, introducing a diagonal weight matrix. Construct the objective function for direction correction to be optimized:

[0013] in, .

[0014] Furthermore, the expressions for the comprehensive directional deviation and comprehensive positional deviation of the current BIM model overlay state are as follows:

[0015]

[0016] in, These are the weighting coefficients for the corresponding structures.

[0017] The present invention also provides a dynamic correction system for drift of augmented reality BIM model based on structural constraint deviation. The system includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it performs the dynamic correction method for drift of augmented reality BIM model based on structural constraint deviation as described above.

[0018] The present invention also provides a computer-readable storage medium storing machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the dynamic correction method for drift of augmented reality BIM model based on structural constraint deviation as described above.

[0019] In summary, compared with the prior art, the augmented reality BIM model drift dynamic correction method and equipment based on structural constraint deviation provided by the present invention have the following beneficial effects: 1. This invention decouples the optimization process into two independent stages: "pure rotation optimization" and "pure translation optimization." Rotation is calculated first, followed by translation, and finally, the pose is updated by merging the results. The pure rotation optimization stage introduces Lie algebraic rotation perturbation variables, while the pure translation optimization stage introduces pure translation perturbation variables. Since the rotation matrix is ​​not closed under addition, directly adding or subtracting its components will result in a non-valid rotation matrix. By introducing rotation perturbation variables into the Lie algebra space, the optimization variables become unconstrained vectors. During updates, these vectors are mapped back to the Lie group via exponential mapping, avoiding illegal solutions. Lie algebra transforms the "multiplication" update of rotation into "addition" operations, standardizing the derivation of the Jacobian matrix of the objective function. Furthermore, rotation (angle) and translation (meter) have different dimensions. In coupled optimization, the ill-conditioned number of the Jacobian matrix can lead to instability. After decoupling, the translation subproblem has a single scale, resulting in more stable optimization. By introducing translational perturbation variables to update translations on the manifold, the inconsistency in physical meaning caused by addition and subtraction in Euclidean space is avoided. This ensures that the translation update conforms to rigid body motion constraints. After fixing the rotation, the estimation of translation (including scale, velocity, and gravity direction) can be transformed into a linear or convex optimization problem, which can be solved quickly through linear global translation constraints. This method is computationally efficient and can find the global optimum. The decoupled subproblems have lower dimensions, allowing the use of specialized optimization strategies. It can run efficiently on ordinary CPUs and meet real-time requirements, thereby improving robustness, accuracy, and efficiency.

[0020] 2. Compared to traditional methods based on pure visual feature point localization and registration, this method extracts spatial line and surface geometric features from point cloud data. Since point cloud features reflect the real physical spatial topology of the building structure, they are less affected by surface texture and lighting conditions, and have strong resistance to environmental interference and robustness. At the same time, compared with global registration methods for dense point clouds such as the ICP algorithm, this method performs voxel downsampling and feature aggregation based on region growing on the local point cloud constructed within the current camera's view frustum. This effectively reduces the size of the point cloud data while extracting key geometric features with structural semantics, avoiding the overhead of computational processing on large-scale dense point clouds, improving computational efficiency, and better meeting the real-time requirements of mobile AR.

[0021] 3. During AR roaming, the superposition state of line and surface structures within the current view frustum is evaluated. The BIM pose correction process is only triggered when the overall deviation of several consecutive measurements exceeds the allowable upper limit. Compared to the passive correction strategy that relies on high-frequency continuous optimization in existing technologies, this method can effectively avoid unnecessary repeated optimization calculations, reduce the computational load and memory usage of mobile terminals, and ensure the stability of the system during long-distance AR roaming. Attached Figure Description

[0022] Figure 1This is a flowchart of an augmented reality BIM model drift dynamic correction method based on structural constraint deviation provided by an embodiment of the present invention; Figure 2 This is a schematic diagram of the BIM structure coordinate mapping and parametric construction process according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the process of local point cloud acquisition, preprocessing and structural feature extraction according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the deviation characterization process of structure matching and model superposition under the view cone constraint according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the pose drift determination and correction triggering mechanism according to an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0024] This invention provides a dynamic correction method for drifting of augmented reality BIM models based on structural constraint deviations. The correction method continuously monitors the superposition offset between the BIM model and the real scene during AR roaming at the construction site, and adaptively triggers pose correction when the offset reaches a set threshold, thereby ensuring that the virtual and real models can maintain stable alignment in the long term.

[0025] The correction method mainly includes the following steps: S1, BIM structural coordinate mapping and parametric representation: The three-dimensional structural elements of the BIM model are mapped from the BIM local coordinate system to the AR world coordinate system through the initial registration matrix, and the three-dimensional structural elements are parametrically expressed in the AR world coordinate system.

[0026] The BIM model is analyzed to extract three-dimensional structural elements such as walls, floors, and beam / column edges. These three-dimensional structural elements are then mapped from the BIM local coordinate system to the AR world coordinate system using an initial registration matrix. Finally, the three-dimensional structural elements are parametrically expressed in the AR world coordinate system.

[0027] Step S1 includes the following sub-steps: S101: Analyze the BIM model and extract three-dimensional structural elements related to the building structure based on the component semantic information of the BIM model, including structural features such as walls, floors, and beam and column edges.

[0028] S102: Let the original coordinate system of the BIM model be... O BIM That is, the BIM local coordinate system; the AR world coordinate system is... O AR Through the initial registration matrix T init From the three-dimensional structural elements O BIM Mapped to O AR .

[0029] For any structural point P bim , its in O AR Coordinates in a coordinate system are represented as follows:

[0030] Wherein, the initial registration matrix T init This includes rotation and translation components.

[0031] S103: For planar structures such as walls and floors, O AR In the coordinate system, the parameterized expression is performed using the normal vector form:

[0032] in, x =( x , y , z ) T ,express O AR The three-dimensional coordinates of a point in a coordinate system; d b The distance from the origin to the planar structure; n b = ( n x , n y , n z ) T , is the unit normal vector of the plane, and its direction is defined as diverging from the plane. O AR Origin of coordinate system (when) d b When it is not 0), d b When it is 0, take n b The first non-zero component is positive; ensure that throughout the entire O AREstablish a one-to-one correspondence between the planar structure and its characteristic parameters in the coordinate system.

[0033] S104: For the edge line structure of beams and columns, O AR In a coordinate system, the direction vector and reference point are used for parametric representation:

[0034] in, l This indicates that any point on the edge line structure is in O AR Three-dimensional coordinates in a coordinate system; q b = ( q x , q y , q z ) T , represents the reference point on the edge line structure (taken as the midpoint of the line segment); v b = ( v x , v y , v z ) T , represents the unit direction vector of the edge line structure, for v b The direction is stipulated as follows: Mandatory stipulation v b The first non-zero coordinate component is always positive; t ∈R, representing the extent of extension along the direction vector; ensuring that throughout the entire O AR Establish a one-to-one correspondence between the edge line structure and its characteristic parameters in the coordinate system.

[0035] S105: Store the parameterized planar and linear structures in a unified manner to build a lightweight BIM structure database for subsequent structural matching and error calculation.

[0036] S2, Local point cloud acquisition and preprocessing: Based on the camera intrinsic parameter matrix, the depth image within the current view frustum range of the mobile terminal during AR roaming is back-projected to map the pixels to the three-dimensional space under the camera coordinate system, generating local point cloud data within the current camera view frustum range. After mapping the local point cloud data to a unified AR world coordinate system using the current device tracking pose, the local point cloud data is sequentially subjected to voxel mesh sampling processing and statistical filtering processing to obtain the processed lightweight point cloud.

[0037] The system acquires depth image data within the current view frustum of the mobile terminal during AR roaming. Simultaneously, it uses the mobile terminal's built-in visual inertial odometry to obtain the device tracking pose at the current moment. Based on the camera intrinsic parameter matrix, it performs back projection on the acquired depth image to map the pixels to a 3D space in the camera coordinate system, generating local point cloud data within the current view frustum of the camera. After mapping the local point cloud data to a unified AR world coordinate system using the current device tracking pose, it sequentially performs voxel mesh sampling and statistical filtering on the local point cloud data to obtain the processed lightweight point cloud.

[0038] Step S2 includes the following sub-steps: S201: During AR roaming at the construction site, depth image data within the current frustum range is collected in real time through the depth sensor mounted on the mobile terminal; at the same time, the current device tracking pose is obtained through the visual inertial odometry built into the mobile terminal.

[0039] S202: Back-project the depth image based on the camera intrinsic parameter matrix to map the pixel coordinates to the three-dimensional space in the camera coordinate system, and obtain the three-dimensional point cloud data in the camera coordinate system.

[0040] The coordinates of a certain pixel are ( u , v ), corresponding to a depth value Z Camera internal parameters are The three-dimensional coordinates of this pixel in the camera coordinate system ( X , Y , Z ) T satisfy:

[0041] A local point cloud is generated by performing the above transformation on all valid pixels within the current view frustum.

[0042] S203: Based on the current tracking pose T SLAM (This represents a rigid body transformation from the camera coordinate system to the AR world coordinate system), mapping a local point cloud from the camera coordinate system to the AR world coordinate system for any local point. P local :

[0043] Obtain local point clouds in the AR unified coordinate system.

[0044] S204: Perform voxel mesh sampling processing on the local point cloud obtained above, divide the space where the local point cloud is located into several voxel units, and take the geometric center of the point set in each voxel unit as the representative point to reduce the size of the point cloud data.

[0045] S205: Perform statistical filtering on the downsampled local point cloud data. By calculating the distance distribution between each point and its neighboring points, outliers and noise points that deviate from the overall distribution are removed, resulting in lightweight point cloud data with stable structure and uniform distribution.

[0046] S3, Point Cloud Structure Feature Extraction and Structure Modeling: Based on the obtained lightweight point cloud, a neighborhood covariance matrix is ​​constructed, and eigenvalue decomposition is performed on the neighborhood covariance matrix. Based on the obtained eigenvalues, a feature significance index is constructed, and discrete disordered points and noise points in the lightweight point cloud are removed according to the feature significance index. Then, line feature points and planar feature points are extracted based on the feature significance index and the lightweight point cloud. Finally, line feature points and planar feature points belonging to the same planar structure or line structure are merged to obtain a stable set of planar structure and line structure features, collectively referred to as the point cloud structure set.

[0047] Step S3 includes the following sub-steps: S301: For any point in the lightweight point cloud in the AR world coordinate system P i Centered on that point, within a preset radius r Searching for a neighborhood set of points within a range, denoted as:

[0048] in, N i For point P i The set of neighborhood points, Indicates the number of neighboring points; P k Other points in a lightweight point cloud; The Euclidean distance between two points.

[0049] S302: For the above point set N i Calculate its geometric center:

[0050] Calculate the positional deviation of each point in the neighborhood point set relative to its geometric center, and construct a spatial distribution matrix:

[0051] The spatial distribution matrix is ​​used to describe the dispersion of neighborhood points in three-dimensional space along each direction and the distribution characteristics of the main direction: when the neighborhood point set is mainly distributed on a certain plane, the dispersion of the points in the normal direction of that plane is the smallest; when the neighborhood point geometry mainly extends along a certain direction, the dispersion of the points in the main direction is the largest; when the neighborhood point set is irregularly distributed, the dispersion in each direction is similar.

[0052] S303: Spatial distribution matrix C i Perform eigenvalue decomposition to obtain three eigenvalues ​​and their corresponding orthogonal eigenvectors:

[0053] And sort them according to the size of the eigenvalues:

[0054] in, e k These are three direction vectors that satisfy the condition of unity orthogonality; This indicates the degree of dispersion of the neighborhood point cloud in the corresponding direction. e 1 corresponds to the largest eigenvalue , is the main direction in which the distribution of the neighborhood point cloud expands the most; e 2 correspondence , indicating the secondary primary direction; e 3 correspondence , which represents the direction in which the change in the neighborhood point cloud is minimal.

[0055] S304: Using the feature values ​​obtained above, determine the lightweight point cloud and construct a significance index for planar and line structures:

[0056]

[0057] Based on the above significance indicators, the points are initially classified according to the following rules: when When this point is identified as a candidate feature point for planar features, the eigenvector corresponding to the smallest eigenvalue is selected. e 3 is used as the local normal vector of this point, with its direction defined in the same way as described in S103, and is added to the set of planar feature candidate points. P plane .

[0058] when When this point is identified as a candidate feature point for the line feature, the eigenvector corresponding to its largest eigenvalue is taken. e 1 represents the local principal direction of this point, defined in accordance with S104, and is added to the line feature candidate point set. Pline .

[0059] If none of the above conditions are met, the point is considered a non-structural point and is removed.

[0060] S305: For the set of planar feature candidate points and the set of line feature candidate points, a region growing clustering algorithm is introduced to perform continuous structure modeling to obtain mutually independent sets of planar structure points and line structure points, thereby obtaining the set of planar structure points and the set of line structure points. Details are as follows:

[0061] For the set of planar feature candidate points P plane The point with the largest planar saliency index in the set is selected as the initial seed point for region growing into the neighborhood. For the current seed point... P i With any point in its domain P j ∈ P plane If the following conditions are met:

[0062] in: n i , n j The local normal vector of the corresponding point, i.e. e 3; This is the distance threshold from the point to the local plane; This is the threshold for the angle between local normal vectors.

[0063] Then it is assumed that the two points belong to the same planar structure. P j The point is merged into the current growth region and used as a new seed point to continue the neighborhood search. When the current region can no longer grow outward, all points in the region are divided into the same independent planar point set. A new seed point is selected from the remaining points, and the growth of the next planar point begins, until all candidate points have been processed.

[0064] For each independent set of planar points P k Plane parameters are estimated using the least squares method:

[0065] Thus, the structural plane parameters are obtained by solving. n c , d c ), construct a set of planar structures.

[0066] Similarly, for the set of candidate points for line featuresP line The same growth logic is used. The point with the highest line structure saliency in the set is selected as the initial seed point for region growth. For the current seed point... P i With any point in its domain P j ∈ P line If the following conditions are met:

[0067] in: v i , v j The main direction of the neighborhood of the corresponding point, i.e. e 1; The first condition indicates the degree to which the line connecting the two points deviates from the main direction of the neighborhood of the first point; and These are the deviation threshold and the angle threshold between the main directions of the two points' neighborhoods, respectively.

[0068] If the two points are considered to belong to the same line structure, the subsequent steps are the same as for the planar structure.

[0069] For each set of line points L k With the geometric center of the point set q c As a reference point, the line direction vector is then solved based on this:

[0070] Thus, the structural line parameters are obtained ( q c , v c ), thereby constructing a set of line structures.

[0071] S4, Establishment of structural matching mechanism and characterization of model superposition deviation under view frustum constraint: In the AR world coordinate system, the BIM model is clipped according to the view frustum space range determined by the current mobile terminal pose to obtain a local structure set; the point cloud structure set extracted within the current view frustum space range is matched with the local structure set for directional consistency and distance constraints. When one point cloud structure in the point cloud structure set corresponds to multiple BIM structures in the local structure set, the BIM structure with the smallest distance from the corresponding point cloud structure is selected as the matching result. Then, the point cloud structure and the matching result form a matching pair. The directional deviation and positional deviation between the point cloud structure and the BIM structure in the matching pair are calculated respectively. The obtained directional deviation and positional deviation are weighted and fused using the structural feature significance index as the weight coefficient to obtain the comprehensive directional deviation and comprehensive positional deviation expressions used to characterize the current BIM model superposition state.

[0072] Step S4 includes the following sub-steps: S401: In order to reduce the amount of structural matching calculation and improve matching efficiency, the spatial range corresponding to the current camera frustum is determined based on the current device tracking pose of the mobile terminal in the AR world coordinate system. Based on this spatial range, BIM structural elements located outside the spatial range of the frustum are filtered out to obtain the local structural set.

[0073] S402: For the set of planar and line structure features extracted and constructed in S3 within the current view frustum space, determine the orientation consistency with the structure in the BIM local structure set in the AR world coordinate system.

[0074] For a planar structure, let the plane normal vector of the planar structure be... n c The BIM plane normal vector is n b If the following conditions are met:

[0075] Then the directions are considered to be consistent; For a line structure, let the direction vector of the line structure be... v c The BIM line direction vector is v b If the following conditions are met:

[0076] Then it is assumed that the directions are consistent.

[0077] above The directional consistency threshold is used. Point cloud structures that satisfy the above directional consistency are combined with BIM structures to form candidate matching pairs.

[0078] S403: Among candidate matching pairs that satisfy directional consistency, filtering is performed based on spatial distance constraints and structural parameter consistency, and a unique matching relationship is established. For planar structures, based on the point cloud structure planar parameters obtained in S3 ( n c , d c ), and the BIM structural planar parameters obtained in S1 ( n b , d b When the following conditions are met:

[0079] It is assumed that the two planes have consistent spatial positions in the AR coordinate system; For line structures, based on the point cloud structure line parameters obtained in S3 ( q c , q c ), and the BIM structural line parameters obtained in S1 ( q b , q b When the following conditions are met:

[0080] It is assumed that the two straight lines have consistent spatial positions in the AR world coordinate system.

[0081] above and These are the distance constraint thresholds for planar structures and line structures, respectively. When the same point cloud structure corresponds to multiple candidate BIM structures, the BIM structure that satisfies the above constraints and has the smallest distance from the point cloud structure is selected as the final matching result.

[0082] Finally, the one-to-one matching relationship between the point cloud structure under the current view frustum and the lines and surfaces of the BIM local structure was determined.

[0083] S404: Based on the matched point cloud structure and BIM structure and their related structural parameters, construct the BIM structure superposition deviation expression in the AR world coordinate system.

[0084] For planar structural deviations, we construct expressions for the directional and positional deviations between planes:

[0085]

[0086] For line structure deviations, construct expressions for the directional and positional deviations between lines:

[0087] In the formula, This refers to the positional deviation of the planar structure; This refers to the directional deviation of the planar structure. This refers to the positional deviation of the linear structure; This refers to the directional deviation of the linear structure.

[0088] S405: Based on the deviation of the above matching structure, and combined with the significance index of the structural features in step S3, the directional deviation and positional deviation under the current camera view frustum are weighted and fused to construct the comprehensive directional deviation and comprehensive positional deviation expressions of the BIM model overlay state at the current moment:

[0089]

[0090] in, The weighting coefficients for the corresponding structures are determined by the structural feature significance index in S3.

[0091] S5, establish a pose drift judgment and triggering mechanism: count the number of matching pairs that have successfully participated in the matching within the current camera's view frustum. When the number of matching pairs exceeds the preset minimum number threshold; and when the comprehensive position deviation or comprehensive direction deviation calculated K times consecutively within the sliding time window exceeds its allowable upper limit, it is determined that the BIM model superposition has accumulated offset, and the subsequent BIM pose optimization solution process is triggered.

[0092] Step S5 includes the following sub-steps: S501: Count the number of matching pairs formed within the current camera's view frustum space. If the number is greater than the preset minimum threshold, proceed to step S502; otherwise, continue to move the camera's viewpoint to complete the matching of the point cloud structure and the BIM structure under the current view frustum.

[0093] S502: Under the premise of satisfying S501, perform comprehensive directional deviation and comprehensive position deviation calculation according to step S4, and record the corresponding deviation value of each calculation within the sliding time window.

[0094] S503: When the combined orientation deviation and combined position deviation obtained from K consecutive calculations within the sliding time window both satisfy the following conditions: or

[0095] It is assumed that the current tracking pose of the mobile terminal has accumulated offset, triggering a subsequent pose reregistration process. This is the upper limit threshold for allowable positional deviation. This is the upper limit threshold for directional deviation.

[0096] S6, BIM Model Pose Optimization Solution: The optimization solution process is divided into a pure rotation optimization direction stage and a pure translation optimization position stage. In the pure rotation optimization direction stage, Lie algebra perturbation variables are introduced to construct the orientation correction objective function and the orientation correction normal equation. The orientation correction normal equation is solved iteratively. In each iteration, a local optimal rotation increment is obtained, and the effectiveness of the local optimal rotation increment obtained in this iteration is evaluated according to the orientation correction objective function. If it is effective, it is applied to the current BIM model pose; otherwise, the solution is discarded and the next iteration begins. This process continues until the iteration termination condition is met, at which point the optimal pose of the orientation-corrected BIM model is determined. Subsequently, the rotation component in the optimal pose of the BIM model is locked, and pure translation optimization is performed to construct the position correction objective function and the corresponding position correction normal equation. The position correction normal equation is solved to obtain the translation correction amount, thereby determining the optimal pose after position calibration. Based on the optimal pose after position calibration, the pose of the BIM model in the AR world coordinate system is updated to achieve BIM model drift correction.

[0097] In the pure rotation optimization stage, Lie algebraic rotation perturbation variables are introduced, and the perturbed rotation matrix is ​​obtained through exponential mapping. The orientation correction objective function is constructed, the rotation Jacobian matrix of the matching structure is calculated, and the orientation correction normal equation with dynamic damping factor is constructed. The optimal rotation correction amount is obtained through iterative solution. During the iteration process, if the error decreases, the pose update is accepted and the damping factor is reduced. If the error increases, the update is rejected and the damping factor is increased and the solution is recalculated until the iteration termination condition is met.

[0098] In the pure translation optimization stage, the rotation matrix after rotation correction is locked, the position correction objective function is constructed, the translation Jacobian matrix is ​​a constant matrix, the position correction normal equation with dynamic damping factor is constructed, the Cholesky decomposition method is used to solve the position correction normal equation, and the globally optimal translation correction amount is calculated.

[0099] The final pose correction matrix is ​​a homogeneous transformation matrix obtained by fusing rotation correction and translation correction. This correction matrix is ​​then used to update the world pose matrix of the BIM model in the AR engine, enabling dynamic repositioning and overlay drift correction of the BIM model.

[0100] Step S6 includes the following steps: S601: Let the pose of the BIM model in the AR world coordinate system at the current moment when pose reregistration is triggered be... The pose increment optimization process is decomposed into two steps: the first stage is defined as the pure rotation optimization stage, and Lie algebra rotation perturbation variables are introduced. The second stage is defined as the pure translation optimization stage, and pure translation perturbation variables are introduced. .

[0101] S602: Extract the pose of the current BIM model T Rotation matrix components in R old , to rotation matrix components R old Introducing Lie algebra pure rotational perturbation variables The perturbed BIM model pose is obtained and applied to the current AR scene. A diagonal weight matrix is ​​introduced. Construct the objective function for direction correction to be optimized:

[0102] in,

[0103] S603: At the Starting Point of Optimization At point =0, calculate the Jacobian matrix of the k-th matching pair. This leads to the global rotation Jacobian matrix of n matching pairs within the view cone. :

[0104] in, Represents an antisymmetric matrix form; and These are the unit direction vectors for the BIM line structure and the point cloud line structure, respectively. and These are the unit normal vectors for the BIM planar structure and the point cloud planar structure, respectively. Indicates introduction Rotation matrices at previous and current times.

[0105]

[0106] S604: Construct the normal equation for direction correction and introduce a dynamic damping factor. With the identity matrix I To ensure that when local features degenerate The absolute invertibility of the coefficient matrix, the direction correction normal equation is:

[0107] Because the 3D rotation space is highly nonlinear, the orientation correction normal equation is constructed based on a local linear approximation of the orientation correction objective function at the current pose. A single solution cannot reach the global minimum. To obtain the optimal orientation correction pose, iterative calculations are required: In each iteration, the structural deviation and corresponding Jacobian matrix are calculated based on the pose in the current state, and substituted into the orientation correction normal equation to solve for the current local optimal rotation increment, consistent with the previous steps. Subsequently, the local optimal rotation increment is updated to the current pose using the Lie group exponential mapping, and the orientation correction objective function is re-evaluated. If the error decreases, the pose update is accepted and the damping factor is reduced. The updated pose is used as the starting point for the next iteration; if the error increases, the update is rejected and the damping factor is increased. Resolve the problem. Iterate until the pose increment is less than the minimum increment threshold, and finally output the optimal pose with corrected orientation.

[0108] S605: After obtaining the optimal pose for orientation correction, record the current pose and use it as the starting pose for position correction, as shown below:

[0109] During the position correction process, R new It remains unchanged.

[0110] S606: Entering the position correction stage, constructing the position correction objective function to be optimized:

[0111] S607: Since the derivative of the positional deviation with respect to the translation vector degenerates into a purely linear relationship, the translation Jacobian matrix is ​​a constant matrix.

[0112]

[0113]

[0114] S608: Reuse the diagonal weight matrix of S602 W Similarly, a dynamic damping factor is introduced. make sure Given that the coefficient matrix is ​​absolutely positive definite and invertible, we construct the position correction normal equation for position correction:

[0115] The above equations are mathematically a standard three-dimensional linear system of equations. The Cholesky decomposition method is introduced to solve this linear system of equations. Since the positional deviation has a strictly linear relationship with the translation variable, the objective function for position correction is mathematically a standard quadratic surface. This optimization process is essentially a linear least squares solution, which typically allows for the direct calculation of the globally optimal translational increment and convergence within a single step, ultimately yielding the optimal pose for position calibration.

[0116] S609: After orientation and position calibration, the final BIM model pose correction matrix is ​​obtained. :

[0117] Update the world pose matrix of the BIM model in the AR engine to achieve dynamic repositioning of the BIM model, and finally complete the correction of BIM model overlay drift.

[0118] The present invention also provides a dynamic correction system for drift of augmented reality BIM model based on structural constraint deviation. The system includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it performs the dynamic correction method for drift of augmented reality BIM model based on structural constraint deviation as described above.

[0119] The present invention also provides a computer-readable storage medium storing machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the dynamic correction method for drift of augmented reality BIM model based on structural constraint deviation as described above.

[0120] The present invention will be further described in detail below with reference to specific embodiments.

[0121] This invention discloses a dynamic correction method for augmented reality BIM model drift based on structural constraint deviation detection. Applicable to AR roaming scenarios at construction sites, this method can correct the cumulative pose drift of the BIM model during long-distance roaming in real time, ensuring stable alignment between the virtual and real models. Please see Figure 1 , Figure 2 , Figure 3 , Figure 4 and Figure 5 The method mainly includes the following steps: S1 BIM structural coordinate mapping and parametric representation: S101: Parse the Revit format BIM model, extract the component semantic information of the BIM model through the Revit API, filter out structural components of type "wall", "floor slab", "beam" and "column", and extract their three-dimensional geometric boundary information respectively. Among them, the walls and floors are extracted as planar structures, and the edges of beams and columns are extracted as line structures.

[0122] S102: Establish the original coordinate system of the BIM model O BIM With AR world coordinate system O AR The initial registration relationship is established. Three non-collinear feature points (such as wall corners and beam-column intersections) are selected at the construction site, and their coordinates are marked in both the BIM model and the actual scene. The initial registration matrix T is calculated using the three-point method. init All BIM structural elements from O BIM Mapped to O AR Coordinate system.

[0123] For any structural point P bim , its in O AR Coordinates in a coordinate system are represented as follows:

[0124] Wherein, the initial registration matrix T init This includes rotation and translation components.

[0125] S103: Parametrically represent the mapped structural elements: For planar structures such as walls and floors, parameterization is performed using the normal method:

[0126] in, x =( x , y , z ) T ,express O AR The three-dimensional coordinates of a point in a coordinate system; d b The distance from the origin to the planar structure; n b = ( n x , n y , n z ) T , is the unit normal vector of the plane, and its direction is defined as diverging from the plane. O AR Origin of coordinate system (when) d b When it is not 0), d b When it is 0, take n b The first non-zero component is positive; ensure that throughout the entireO AR Establish a one-to-one correspondence between the planar structure and its characteristic parameters in the coordinate system.

[0127] For the edge line structure of beams and columns, parameterization is performed using direction vectors and reference points:

[0128] in, l This indicates that any point on the edge line structure is in O AR Three-dimensional coordinates in a coordinate system; q b = ( q x , q y , q z ) T , represents the reference point on the edge line structure (taken as the midpoint of the line segment); v b = ( v x , v y , v z ) T , represents the unit direction vector of the edge line structure, for v b The direction is stipulated as follows: Mandatory stipulation v b The first non-zero coordinate component is always positive; t ∈R, representing the extent of extension along the direction vector; ensuring that throughout the entire O AR Establish a one-to-one correspondence between the edge line structure and its characteristic parameters in the coordinate system.

[0129] S104: Store all parameterized planar and linear structures as a lightweight database in JSON format to build a BIM structure database. Each record in the database contains information such as structure type, parameter values, and component ID.

[0130] S2 Local Point Cloud Acquisition and Preprocessing: S201: During AR roaming at the construction site, the 3D ToF depth sensor of the mobile terminal acquires depth images within the current view frustum in real time. The depth image resolution is 320×240, and the frame rate is 30fps. Simultaneously, the device tracking pose at the current moment is obtained through ARCore's visual inertial odometry (VIO). T SLAM .

[0131] S202: Back-project the depth image based on the camera intrinsic parameter matrix to generate a local point cloud in the camera coordinate system. The camera intrinsic parameters are obtained in real-time through ARCore's CameraIntrinsics class; the coordinates of a certain pixel are ( u , v ), corresponding to a depth value Z Camera internal parameters are The three-dimensional coordinates of this pixel in the camera coordinate system ( X , Y , Z ) T satisfy:

[0132] The process iterates through all pixels in the depth image, filtering out pixels with depth values ​​of 0 (no data), 65535 (saturated), and invalid pixels that are outside the effective measurement range of the mobile terminal's ToF sensor. The remaining valid pixels are then subjected to the transformation described above to generate an initial local point cloud. The generated point cloud is temporarily stored in the device's RAM as a dynamic array.

[0133] S203: Pose tracking via device T SLAM This maps local point clouds from the camera coordinate system to a unified AR world coordinate system.

[0134] Device tracking pose T SLAM The ARCore visual inertial odometry (VIO) outputs data in real time, mapping the local point cloud from the camera coordinate system to the AR world coordinate system for any local point. P local :

[0135] Obtain local point clouds in the AR unified coordinate system.

[0136] S204: Perform voxel mesh downsampling on the local point cloud, set the voxel size to 0.05m, and take the geometric center of all points within each voxel as the representative point.

[0137] S205: Perform statistical filtering to denoise the downsampled point cloud. Set the number of neighboring points to 50 and the standard deviation factor to 1.0. Remove outliers and noise points that deviate from the overall distribution to obtain a structurally stable lightweight point cloud.

[0138] S3 Point Cloud Structure Feature Extraction and Structure Modeling: S301: For any point in the lightweight point cloud P i Using this point as the center, search for a neighborhood set of points within a radius of r = 0.1m. Ni .

[0139] S302: For the above point set N i Calculate its geometric center:

[0140] Where N is the number of points in the neighborhood point set.

[0141] Calculate the positional deviation of each point in the neighborhood point set relative to the geometric center, and construct the spatial distribution matrix:

[0142] S303: Spatial distribution matrix C i Perform eigenvalue decomposition to obtain three eigenvalues ​​sorted in descending order. and the corresponding unit orthogonal eigenvectors e 1. e 2. e 3.

[0143] S304: Constructing significance evaluation indices for planar and line structures based on eigenvalues:

[0144]

[0145] Set a plane saliency threshold =0.8, linear significance threshold =0.8; when When this point is identified as a candidate feature point for planar features, the eigenvector corresponding to the smallest eigenvalue is selected. e 3 is used as the local normal vector of this point, with its direction defined in the same way as described in S103, and is added to the set of planar feature candidate points. P plane ;when When this point is identified as a candidate feature point for the line feature, the eigenvector corresponding to its largest eigenvalue is taken. e 1 represents the local principal direction of this point, defined in accordance with S104, and is added to the line feature candidate point set. P line .

[0146] If none of the above conditions are met, the corresponding point is considered a non-structure point and is removed.

[0147] S305: The region growing clustering algorithm is used to model the structure of feature points. Planar structure modeling: The point with the highest planar saliency index in the set is selected as the initial seed point for region growing into the neighborhood. For the current seed point and any planar feature candidate point within its neighborhood... ， Two points are considered to belong to the same planar structure if both of the following conditions are met: 1. The angle between the local normal vectors of the two points is less than 5°; 2. The distance from the neighboring point to the local plane defined by the seed point is less than 0.02m. When two points simultaneously meet the above conditions, they will be merged into the current growth region and used as the new seed point to continue searching the neighborhood. When the current region can no longer grow outward, the least squares method is used to perform global plane fitting on the set of all points in the region to obtain the unified parameters of the planar structure. n c , d c Then, a seed point is selected from the remaining points to start the growth of the next plane, until all candidate points have been processed.

[0148] Linear structure modeling: Select the point with the largest line significance index from the set as the initial seed point and grow it into the neighborhood. For the current seed point... P i With any line feature candidate point in its neighborhood P j Two points are considered to belong to the same line structure if both of the following conditions are met: 1. The angle between the principal directions of the neighborhood of the two points is less than 5°. 2. Neighboring points... P j To the seed point P i The distance between the defined local straight lines is less than 0.02m. When two points simultaneously satisfy the above conditions, [the following will occur]. P j The point is merged into the current growth region and used as a new seed point to continue searching the neighborhood. When the current region can no longer grow outward, principal component analysis is used to perform global line fitting on the set of all points in the region to obtain the unified parameters of the line structure. q c , v c Then, a new seed point is selected from the remaining points, and the growth of the next straight line begins until all candidate points have been processed.

[0149] Structural matching and deviation characterization under S4 view cone constraint: S401: Based on the current device pose and camera intrinsic parameters, calculate the spatial range of the camera's view frustum to trim the BIM structure database, retaining only the structural elements within the view frustum, thus obtaining a BIM local structure set.

[0150] S402: For the actual set of point cloud structures extracted and constructed in S3 within the current view frustum space, perform orientation consistency matching and determination with the structures in the BIM local structure set in the AR world coordinate system.

[0151] For a planar structure, let the plane normal vector of the point cloud be... n c The BIM plane normal vector is n b If the following conditions are met:

[0152] Then the directions are considered to be consistent; For a line structure, let the point cloud line direction vector be... v c The BIM line direction vector is v b If the following conditions are met:

[0153] Then it is assumed that the directions are consistent.

[0154] above =5° is the directional consistency threshold. Point cloud structures that meet the above directional consistency requirements are combined with BIM structures to form candidate matching pairs.

[0155] S403: Among the candidate matching pairs that satisfy the consistency of direction, further screening is performed based on the consistency of structural parameters and a unique matching relationship is established.

[0156] For planar structures, based on the point cloud structure planar parameters obtained in S3 ( n c , d c ), and the BIM structural plan parameters obtained in S1 ( n b , d b When the following conditions are met:

[0157] It is assumed that the two planes have consistent spatial positions in the AR coordinate system.

[0158] For line structures, based on the point cloud structure line parameters obtained in S3 ( q c , q c ), and the BIM structural line parameters obtained in S1 ( q b , q b When the following conditions are met:

[0159] It is assumed that the two straight lines have consistent spatial positions in the AR coordinate system.

[0160] above and The values ​​are all set to 0.05m, representing the distance constraint thresholds for planar and line structures, respectively. When the same point cloud structure corresponds to multiple candidate BIM structures, the one that satisfies the above constraints and has the smallest distance from the point cloud structure is selected as the final matching result. This ultimately determines the one-to-one matching relationship between the point cloud structure under the current view frustum and the lines and surfaces of the BIM local structure.

[0161] S404: Based on the matched point cloud structure and BIM structure and their related structural parameters, construct the BIM structure superposition deviation expression in the AR world coordinate system.

[0162] For planar structural deviations, we construct expressions for the directional and positional deviations between planes:

[0163]

[0164] For line structure deviations, construct expressions for the directional and positional deviations between lines:

[0165]

[0166] S405: Based on the deviation of the above matching structure, and combined with the saliency of the structural features in step S3, the deviation under the current camera frustum is weighted and fused to construct an expression for the model superposition deviation at the current moment:

[0167]

[0168] in, The weighting coefficients for the corresponding structures are determined by the structural feature significance index in S3.

[0169] S5 Pose Drift Detection and Triggering Mechanism: S501: Count the number of valid structures participating in matching within the current view frustum, and set a minimum threshold N. min =5. If the number of effective structures is greater than N. min If the deviation is not calculated, proceed with the deviation calculation; otherwise, continue collecting data.

[0170] S502: Use a sliding time window to record the comprehensive deviation value for K consecutive times, with a window step size of 1.

[0171] S503: If the overall position deviation calculated for 5 consecutive times within the sliding window exceeds 0.05m (the upper limit of the position deviation allowable), or the overall direction deviation calculated for 5 consecutive times exceeds 3° (the upper limit of the direction deviation allowable), then the BIM model is determined to have accumulated drift, triggering pose optimization solution.

[0172] S6 Two-Stage BIM Pose Optimization Solution: S601: Set the current BIM model pose to T old The optimization process is decomposed into two stages: pure rotation optimization and pure translation optimization.

[0173] S602: Pure rotation optimization stage: Introducing Lie algebra pure rotational perturbation variables The perturbation rotation matrix is ,in( )^ represents an antisymmetric matrix.

[0174] Introducing a diagonal weight matrix Construct the objective function for optimizing the directional deviation:

[0175] in,

[0176] At the starting point of seeking excellence At point = 0, calculate the Jacobian matrix of the k-th matching structure. This leads to the global rotation Jacobian matrix of the n matching structures within the view frustum. : Jacobian row vectors corresponding to the line structure: = The Jacobian row vectors corresponding to the planar structure: = ;in, Represents an antisymmetric matrix form; and These are the unit direction vectors for the BIM line structure and the point cloud line structure, respectively. and These are the unit normal vectors for the BIM planar structure and the point cloud planar structure, respectively. Indicates introduction Rotation matrices at previous and current times.

[0177]

[0178] Constructing with dynamic damping factor With the identity matrix I The direction of the normal equation is corrected to ensure the absolute invertibility of the coefficient matrix in the event of local feature degradation:

[0179] Set the initial damping factor =0.001.

[0180] The orientation correction normal equation is solved using the LM iterative method: In each iteration, the structural deviation and the corresponding Jacobian matrix are calculated based on the current reference pose, and then substituted into the orientation correction normal equation to solve for the current local optimal rotation increment. The increment is then used to update the current pose using the Lie group exponent mapping, and the orientation correction objective function is re-evaluated. If the error decreases, the pose update is accepted and the damping factor is adjusted. Divide by 10; if the error increases, reject the update and adjust the damping factor. Multiply by 10 and solve again. The iteration termination condition is... Alternatively, after 10 iterations, the optimal rotation matrix for direction correction can be output. .

[0181] S603: Pure translation optimization stage: Optimal rotation matrix after locking direction correction Introducing pure translational perturbation variables Construct the objective function for position deviation optimization:

[0182] Since the derivative of the positional deviation with respect to the translation vector degenerates into a purely linear relationship, the translation Jacobian matrix is ​​a constant matrix: The translational Jacobian row vectors corresponding to the planar structure: The translated Jacobian row vector corresponding to the line structure: Thus constructing :

[0183] Reusing the S602 diagonal weight matrix W Similarly, a dynamic damping factor is introduced. make sure The coefficient matrix is ​​absolutely positive definite and invertible, and a normal equation for position correction is constructed:

[0184] The aforementioned normal equations are mathematically a standard three-dimensional linear system of equations. The Cholesky decomposition method is introduced to solve this system. Since the translation objective function is a standard quadratic surface, the optimization process is essentially a linear least squares solution. A single-step calculation can yield the globally optimal translation vector, ultimately leading to the optimal pose for position calibration. .

[0185] S604: Fusion yields homogeneous pose correction matrix :

[0186] Update the world pose matrix of the BIM model in the AR engine to achieve dynamic repositioning of the BIM model, and finally complete the correction of BIM model overlay drift.

[0187] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for correcting the drift dynamics of an augmented reality BIM model based on structural constraint deviations, characterized in that, The steps are as follows: (1) The three-dimensional structural elements of the BIM model are mapped from the BIM local coordinate system to the AR world coordinate system through the initial registration matrix, and the three-dimensional structural elements are parametrically expressed in the AR world coordinate system. (2) Based on the camera intrinsic parameter matrix, the depth image within the current view frustum range of the mobile terminal during AR roaming is back-projected to map the pixel points to the three-dimensional space under the camera coordinate system, generating local point cloud data within the current view frustum range of the camera. After mapping the local point cloud data to the AR world coordinate system using the current device tracking pose, a lightweight point cloud is obtained. (3) Based on the obtained lightweight point cloud, a neighborhood covariance matrix is ​​constructed, and eigenvalue decomposition is performed on the neighborhood covariance matrix. Based on the obtained eigenvalues, a feature significance index is constructed, and discrete disordered points and noise points in the lightweight point cloud are removed according to the feature significance index. Then, line feature points and planar feature points are extracted based on the feature significance index and the lightweight point cloud. Finally, line feature points and planar feature points belonging to the same planar structure or line structure are merged to obtain a stable set of planar structure and line structure features, which are collectively referred to as the point cloud structure set. (4) In the AR world coordinate system, the BIM model is clipped according to the view frustum space range determined by the current mobile terminal pose to obtain the local structure set; the point cloud structure set extracted within the current view frustum space range is matched with the local structure set for directional consistency and distance constraint matching. When there is a point cloud structure in the point cloud structure set that corresponds to multiple BIM structures in the local structure set, the BIM structure with the smallest distance from the corresponding point cloud structure is selected as the matching result. Then, the point cloud structure and the matching result form a matching pair. The directional deviation and position deviation between the point cloud structure and the BIM structure in the matching pair are calculated respectively. The directional deviation and position deviation obtained are weighted and fused using the structural feature significance index as the weight coefficient to obtain the comprehensive directional deviation and comprehensive position deviation expressions used to characterize the current BIM model superposition state. (5) When the number of matching pairs successfully participating in the matching within the current camera view cone exceeds the preset minimum number threshold, and the comprehensive position deviation or comprehensive direction deviation calculated for K consecutive times within the sliding time window exceeds its allowable upper limit, it is determined that the BIM model superposition has accumulated offset. (6) The BIM model pose optimization solution is divided into a pure rotation optimization stage and a pure translation optimization stage; In the pure rotation optimization stage, Lie algebra perturbation variables are introduced to solve for the optimal pose of the BIM model after orientation correction. Then, after locking the rotation component in the optimal pose of the BIM model, pure translation optimization is performed to obtain the translation correction amount, thereby determining the optimal pose after position calibration. Based on the optimal pose after position calibration, the pose of the BIM model in the AR world coordinate system is updated to realize the drift correction of the BIM model.

2. The method for correcting drift dynamics of augmented reality BIM models based on structural constraint deviations as described in claim 1, characterized in that: In the pure rotation optimization direction stage, Lie algebra perturbation variables are introduced to construct the direction correction objective function and the direction correction normal equation. The direction correction normal equation is solved iteratively. In each iteration, the local optimal rotation increment is obtained. The effectiveness of the local optimal rotation increment obtained in this iteration is evaluated according to the direction correction objective function. If it is effective, it is applied to the current BIM model pose; otherwise, the solution is discarded and the next iteration is entered. The optimal pose of the BIM model after direction correction is determined when the iteration termination condition is met.

3. The method for correcting drift dynamics of augmented reality BIM models based on structural constraint deviations as described in claim 2, characterized in that: After locking the rotation component in the optimal pose of the BIM model, pure translation optimization is performed to construct the position correction objective function and the corresponding position correction normal equation. The translation correction amount is obtained by solving the position correction normal equation, and then the optimal pose after position calibration is determined. Based on the optimal pose after position calibration, the pose of the BIM model in the AR world coordinate system is updated to realize the drift correction of the BIM model.

4. The method for correcting drift dynamics of augmented reality BIM models based on structural constraint deviations as described in claim 1, characterized in that: In the pure rotation optimization stage, Lie algebraic rotation perturbation variables are introduced, and the perturbed rotation matrix is ​​obtained through exponential mapping. An orientation correction objective function is constructed, the rotation Jacobian matrix of the matching structure is calculated, and an orientation correction normal equation with a dynamic damping factor is constructed. The optimal rotation correction amount is obtained through iterative solution. During the iteration process, if the error decreases, the pose update is accepted and the damping factor is reduced. If the error increases, the update is rejected and the damping factor is increased and the solution is recalculated until the iteration termination condition is met. In the pure translation optimization stage, the rotation matrix after rotation correction is locked, the position correction objective function is constructed, the translation Jacobian matrix is ​​a constant matrix, the position correction normal equation with dynamic damping factor is constructed, the Cholesky decomposition method is used to solve the position correction normal equation, and the globally optimal translation correction amount is calculated.

5. The method for dynamic correction of drift in augmented reality BIM models based on structural constraint deviation as described in claim 4, characterized in that: The final pose correction matrix is ​​a homogeneous transformation matrix obtained by fusing rotation correction and translation correction. This correction matrix is ​​then used to update the world pose matrix of the BIM model in the AR engine, enabling dynamic repositioning and overlay drift correction of the BIM model.

6. The method for dynamic correction of drift in augmented reality BIM models based on structural constraint deviation as described in claim 5, characterized in that: After orientation and position calibration, the pose correction matrix of the final BIM model is obtained. : In the formula, The rotation matrix after orientation correction is obtained by iterative solution in the pure rotation optimization stage; In lock Then, the translation vector after position correction is obtained through pure translation optimization; [0,0,0].

7. The method for dynamic correction of drift in augmented reality BIM models based on structural constraint deviations as described in claim 5, characterized in that: Extract the pose of the current BIM model T Rotation matrix components in R old , to rotation matrix components R old Introducing Lie algebra pure rotational perturbation variables The perturbed BIM model pose is obtained and applied to the current AR scene, introducing a diagonal weight matrix. Construct the objective function for direction correction to be optimized: in, .

8. The method for dynamic correction of drift in augmented reality BIM models based on structural constraint deviations as described in any one of claims 1-7, characterized in that: The expressions for the overall directional deviation and overall positional deviation of the current BIM model overlay state are as follows: in, These are the weighting coefficients for the corresponding structures.

9. A dynamic correction system for drift of augmented reality BIM model based on structural constraint deviation, characterized in that: The system includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it performs the dynamic correction method for drift of augmented reality BIM model based on structural constraint deviation as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the augmented reality BIM model drift dynamic correction method based on structural constraint deviation as described in any one of claims 1-8.

Images