Immersive review method and system for building design achievements based on BIM (Building Information Modeling) and AR (Augmented Reality)

Abstract

The invention discloses a building design achievement immersive review method and system based on BIM and AR. The method comprises the following steps of 1, obtaining a BIM model and performing semantic analysis; 2, constructing a semantic review grid unit set; 3, completing space registration between the model and a real scene, and presenting the grid units in the view of a user in a layer superposition manner; 4, tracking the spatial position and visual angle parameters of the user through AR equipment to generate a view cone, and outputting a user interaction record set; 5, inputting an image frame corresponding to the current visual angle of the user into the improved MambaVision visual recognition model to generate a shielding probability graph; 6, calculating an examination risk score value; and step 7, screening high-risk semantic review grid units and carrying out visual angle guide prompt to assist the user in supplementary review. According to the invention, the BIM model and the visual identification model are fused, and intelligent guidance of architectural design review is realized.

Description

Technical Field

[0001] This invention relates to the field of architectural design review technology, and in particular to an immersive review method and system for architectural design deliverables based on BIM and AR. Background Technology

[0002] With the continuous development of Building Information Modeling (BIM) and Augmented Reality (AR) technologies, design review methods based on spatial information visualization have received increasing attention in the field of architectural engineering. Existing architectural design review processes mainly rely on methods such as two-dimensional drawing verification, static three-dimensional model browsing, and on-site expert inspections. However, the following problems commonly exist in actual engineering practice: The lack of real-time registration and immersive integration between the design model and the real-world scene makes it difficult for reviewers to accurately determine the spatial relationships and installation accessibility of components. The semantic information of components in the model is not fully expressed in a structured manner, resulting in low efficiency and high risk of omissions in component attribute tracing during multi-disciplinary collaborative reviews. Traditional 3D model browsing tools lack a fine tracking mechanism for user perspective behavior, making it impossible to identify which areas have been fully reviewed and which areas have been ignored due to occlusion or lack of attention. In addition, the current lack of a dynamic risk analysis mechanism that combines visual recognition results with component semantic grids makes it difficult to identify high-risk component areas in a timely manner and guide users to supplement the review, thus reducing the quality and efficiency of the overall design review.

[0003] Therefore, how to provide an immersive review method and system for architectural design deliverables based on BIM and AR is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] One objective of this invention is to propose an immersive review method and system for architectural design deliverables based on BIM and AR. This invention integrates BIM semantic modeling, AR spatial registration, user perspective tracking, and image semantic recognition to construct an immersive intelligent review method for architectural design deliverables. Through component semantic-driven non-uniform spatial division, combined with an improved MambaVision visual recognition model to extract image semantic embedding features, the visibility of the review area is quantified and interactive behavior is recorded. Furthermore, perspective guidance prompts are driven by review risk scoring to assist users in completing supplementary reviews of high-risk areas.

[0005] An immersive review method for architectural design deliverables based on BIM and AR according to an embodiment of the present invention includes the following steps: Step 1: Obtain the BIM model and perform semantic parsing on the components in the BIM model to form a component semantic set; Step 2: Based on the semantic set of the components, the BIM model space is non-uniformly divided to construct a set of semantic review grid units. Each semantic review grid unit is bound to a unique identifier and review attribute. Step 3: Load the BIM model and the semantic review grid set into the AR device to complete the spatial registration between the BIM model and the real scene, and present the semantic review grid set in the user's field of vision of the AR device as an overlay layer; Step 4: Using the AR device, the user's spatial position and viewing angle parameters are tracked in real time to generate a view frustum. The interaction status between each semantic review grid unit and the view frustum is determined, and a user interaction record set is output. Step 5: Collect the image frame corresponding to the user's current viewpoint, input it into the improved MambaVision visual recognition model to generate an image semantic embedding feature map, and combine the semantic review grid cell set with the component space mapping relationship to generate an occlusion probability map; Step 6: For each semantic review grid cell, calculate the review risk score based on the review attribute fields, the interaction status in the user interaction record set, and the occlusion probability value in the occlusion probability map. Step 7: Select high-risk semantic review grid units whose review risk score exceeds the preset risk threshold, and provide perspective guidance prompts for the high-risk semantic review grid units in the AR device to assist users in conducting supplementary reviews.

[0006] Optionally, step one specifically includes: Extract the component's category attributes, system attributes, spatial coordinate parameters, geometric dimension parameters, and unique identifier. The system attributes include the component's professional field, equipment category, operating function, and installation process requirements. The category attribute is mapped to a component type label, the system attribute is mapped to a component system label, the spatial coordinate parameter and geometric dimension parameter are mapped to a component location label, and the unique identifier is mapped to a component number label. The component type label, component system label, component location label, and component number label are combined one by one according to the component to form a component semantic label; All component semantic tags are aggregated to form a component semantic set.

[0007] Optionally, step two specifically involves: Based on the component system label and component location label of each component in the component semantic set, the BIM model space is non-uniformly divided; When the component system label belongs to the water supply and drainage system, fire protection system, electrical system, or HVAC system, the size is set to a semantic review grid cell with a side length of 0.4 meters; when the component system label belongs to the structural system or interior decoration system, the size is set to a semantic review grid cell with a side length of 0.6 meters; when the component system label does not belong to the water supply and drainage system, fire protection system, electrical system, HVAC system, structural system, or interior decoration system, the size is set to a semantic review grid cell with a side length of 0.3 meters. When the component location label belongs to an evacuation corridor, pipe shaft, equipment room, refuge floor, or fire elevator lobby, the grid division size of the corresponding area is reduced to 80% of the original size based on the component system label division. Each semantic review grid cell is bound to a unique identifier of its corresponding component, and the component type label and component system label are written into the review attribute field of the semantic review grid cell. All semantic review grid units are merged to form a semantic review grid unit set.

[0008] Optionally, step three specifically includes: The BIM model and semantic review grid cell set are exported in a unified three-dimensional coordinate format and loaded into the AR device through the glTF extended format; In the physical scene, at least three component surfaces are selected as spatial anchor points. Based on the camera of the AR device, image frames of the spatial anchor points are acquired and the corresponding two-dimensional image coordinates are extracted. The corresponding three-dimensional anchor point coordinates are retrieved in the BIM model to construct a point pair set of two-dimensional image coordinates and three-dimensional anchor point coordinates. Based on the point pair set, the PnP algorithm is used to calculate the initial pose of the AR device relative to the BIM model coordinate system. In each iteration, the camera rotation and translation parameters are corrected by comparing the pixel error between the virtual projection point and the actual image point until the pixel error is less than the preset accuracy threshold or the preset maximum number of iterations is reached, and the perspective projection matrix is ​​output. The perspective projection matrix is ​​applied to the spatial coordinates of all components in the BIM model to achieve spatial registration between the BIM model and the real scene. The semantic review grid unit set is loaded into the AR device as a semi-transparent layer with an opacity of 0.3 and displayed on the surface of the spatially registered BIM model components. It presents an overlay effect in the user's field of vision and is rendered and updated in real time as the user's spatial position and viewing angle change.

[0009] Optionally, step four specifically involves: The AR device acquires the user's spatial position and viewing angle parameters, and, in conjunction with the AR device's built-in field of view, calculates and generates a visual cone with the user's eye center point as the vertex and the head orientation as the main axis. The view frustum is projected onto the three-dimensional coordinate space where the semantic review grid unit set is located. Using a ray-by-ray projection method, multiple directional rays are emitted outward from the vertex of the view frustum at a set resolution. Each ray is then sequentially checked to see if it intersects with the boundary of each semantic review grid unit. If the ray intersects with the boundary of the semantic review grid unit, and the intersection point is within a set threshold range around the center point of the semantic review grid unit, then the interaction state between the semantic review grid unit and the view frustum is recorded as a direct hit; if the ray intersects with the boundary of the semantic review grid unit, and the intersection point is outside a set threshold range around the center point of the semantic review grid unit, then the interaction state between the semantic review grid unit and the view frustum is recorded as edge visible; if the ray does not intersect with the boundary of the semantic review grid unit, then the interaction state between the semantic review grid unit and the view frustum is recorded as no interaction. Simultaneously, the spatial location and timestamp of the user within each view frustum update cycle are recorded. The interaction status and duration of all semantic review grid units that are judged to be directly hit or edge visible are summarized within the update cycle, and a user interaction record set is output. The user interaction record set includes the unique identifier of the semantic review grid unit, the interaction status, the start time, and the end time.

[0010] Optionally, the improved MambaVision visual recognition model adopts a state evolution-driven sequence feature extraction structure to initialize a state vector for each pixel position of the input image frame; The state evolution-driven sequence feature extraction structure includes a component region sequence channel and a boundary mutation sequence channel. The two sequence channels model the temporal dependency relationship between the main component and the background transition region, respectively, and the state output is weighted and integrated through a gated fusion module after each layer calculation. During the state update process, a modulation function based on pixel spatial coordinates is introduced to adjust the spatial sensitivity of the update weights in the state transition. The modulation function takes the two-dimensional coordinates of the pixel in the image coordinate system as input, normalizes them to the [0,1] interval, and multiplies them element by element with each dimension component of the previous layer's state vector to generate a modulation coefficient vector. The modulation coefficient vector is used to scale the element weights of the current layer's state transition matrix item by item to achieve differentiated updates of the state at different spatial locations, so that the state change rate near the component edge and geometric change region is higher than that in the homogeneous region inside the component. Pixel-wise mapping is performed on all modulated and updated state vectors to generate an image semantic embedding feature map with the same size as the input image frame.

[0011] Optionally, the step of generating an occlusion probability map by combining the semantic review grid cell set with the component spatial mapping relationship specifically involves: The semantic embedding feature map of the image corresponding to the user's current viewpoint is jointly processed with the viewpoint parameters and spatial registration information of the AR device to calculate the corresponding coordinate position of each pixel in the image frame in the three-dimensional space of the BIM model. Based on the component space mapping relationship, the projection area of ​​each semantic review grid unit in the image frame is determined, and the semantic embedding features of all pixels in the projection area are extracted. After sorting the extracted semantic embedding features according to pixel coordinates, the principal component direction vectors distributed in the semantic embedding space are statistically analyzed, and the Mahalanobis distance between the principal component direction vectors and the mean vector of the embedding features corresponding to the component system labels is calculated. When the Mahalanobis distance exceeds a set threshold, the pixel is marked as an unrecognizable pixel; The ratio between the number of unidentifiable pixels in the image region corresponding to each semantic review grid unit and the total number of pixels is statistically analyzed to calculate the occlusion probability value. The occlusion probability values ​​of all semantic review grid units are then used to generate an occlusion probability map based on their spatial location mapping relationship.

[0012] Optionally, step six specifically includes: Read the component system label in the review attribute field of each semantic review grid unit, call the interaction status and interaction duration in the user interaction record set, and match the occlusion probability value at the corresponding position in the occlusion probability map; The review level weight coefficient is set according to the component system label in the review attribute field. The interaction state and occlusion probability value are jointly calculated. The interaction factor of the semantic review grid unit with the interaction state of direct hit is set to 1, the interaction factor of the semantic review grid unit with the interaction state of edge visible is set to 0.6, and the interaction factor of the semantic review grid unit with the interaction state of no interaction is set to 0. The interaction score is obtained by multiplying the interaction factor of the semantic review grid unit by the interaction duration. The interaction score and the occlusion probability value are weighted and fused according to a preset ratio to obtain the visibility risk value of the semantic review grid unit, and then multiplied by the review level weight coefficient to obtain the review risk score of the semantic review grid unit.

[0013] Optionally, step seven specifically includes: The review risk scores of all semantic review grid units are screened, and semantic review grid units whose review risk scores exceed the preset risk threshold are extracted and marked as high-risk semantic review grid units; The spatial coordinates of the high-risk semantic review grid unit are compared and analyzed with the user's current view parameters. When the high-risk semantic review grid unit is not in the current view cone, or the area displayed in the view is smaller than the set area threshold, the view guidance prompt mechanism is activated. The perspective guidance and prompting mechanism includes generating a dynamic arrow at the edge of the user's current perspective in the AR device interface to guide the user to adjust the perspective direction to face the actual spatial orientation of the high-risk semantic review grid unit. When the high-risk semantic review grid unit is fully within the user's field of view and the observation time exceeds the set duration threshold, the supplementary review of the high-risk semantic review grid unit is marked as complete.

[0014] An immersive review system for architectural design deliverables based on BIM and AR according to an embodiment of the present invention includes the following modules: The component semantic parsing module is used to perform semantic parsing on components in the BIM model and generate a component semantic set. The spatial partitioning module is used to non-uniformly partition the BIM model space based on the semantic set of components, construct a set of semantic review grid units, and bind a unique identifier and review attribute. The AR loading and registration module is used to load the BIM model and semantic review grid unit set into the AR device, complete the spatial registration between the BIM model and the real scene, and present the semantic review grid unit in the user's field of vision of the AR device in the form of an overlay layer. The viewpoint tracking and interaction acquisition module is used to track the user's viewpoint and spatial position, generate a view frustum, and output a set of user interaction records. The visual recognition and occlusion analysis module is used to call the improved MambaVision visual recognition model to generate image semantic embedding feature maps, and combine the semantic review grid cell set and component space mapping relationship to generate occlusion probability maps; The review risk assessment module is used to calculate the review risk score of the semantic review grid unit; The perspective guidance prompt module is used to identify high-risk semantic review grid cells and provide perspective guidance prompts in AR devices to assist users in supplementing the review.

[0015] The beneficial effects of this invention are: This invention addresses the problems of low component visualization accuracy, coarse tracking of review behavior, and inability to distinguish occlusion omissions in existing architectural design reviews by constructing an immersive review process based on component semantic parsing, non-uniform spatial partitioning, and AR spatial registration. It proposes an immersive review method for architectural design deliverables based on BIM and AR. The improved MambaVision visual recognition model introduces a state evolution-driven sequence feature extraction structure, and enhances the sensitivity of image semantic embedding feature maps to component edges and occlusion areas through a dual-channel sequence modeling structure of component region and boundary mutations combined with a spatial coordinate modulation mechanism. In the occlusion analysis stage... An algorithm for identifying unidentifiable pixels based on the Mahalanobis distance between the principal component direction and the embedding mean of the component system is constructed to improve the accuracy and stability of occlusion area determination. In interactive behavior modeling, a review risk score is calculated by combining user interaction record sets and occlusion probability maps and setting review level weights through component system labels, thereby achieving a fine assessment of the visibility risk of semantic review grid units. Finally, a high-risk semantic review grid unit screening and perspective guidance prompt mechanism is proposed to guide users to supplement the review of occluded or omitted areas, ensuring the completeness of design review coverage and the depth of key component verification, thereby significantly improving the intelligence level and quality assurance capability of architectural design outcome review. Attached Figure Description

[0016] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is an overall flowchart of an immersive review method for architectural design results based on BIM and AR proposed in this invention. Figure 2 This is a schematic diagram of the structure of an immersive review system for architectural design results based on BIM and AR proposed in this invention. Detailed Implementation

[0017] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0018] refer to Figure 1 An immersive review method for architectural design deliverables based on BIM and AR includes the following steps: Step 1: Obtain the BIM model and perform semantic parsing on the components in the BIM model to form a component semantic set; Step 2: Based on the semantic set of the components, the BIM model space is non-uniformly divided to construct a set of semantic review grid units. Each semantic review grid unit is bound to a unique identifier and review attribute. Step 3: Load the BIM model and the semantic review grid set into the AR device to complete the spatial registration between the BIM model and the real scene, and present the semantic review grid set in the user's field of vision of the AR device as an overlay layer; Step 4: Using the AR device, the user's spatial position and viewing angle parameters are tracked in real time to generate a view frustum. The interaction status between each semantic review grid unit and the view frustum is determined, and a user interaction record set is output. Step 5: Collect the image frame corresponding to the user's current viewpoint, input it into the improved MambaVision visual recognition model to generate an image semantic embedding feature map, and combine the semantic review grid cell set with the component space mapping relationship to generate an occlusion probability map; Step 6: For each semantic review grid cell, calculate the review risk score based on the review attribute fields, the interaction status in the user interaction record set, and the occlusion probability value in the occlusion probability map. Step 7: Select high-risk semantic review grid units whose review risk score exceeds the preset risk threshold, and provide perspective guidance prompts for the high-risk semantic review grid units in the AR device to assist users in conducting supplementary reviews.

[0019] In this embodiment, step one specifically includes: Extract the component's category attributes, system attributes, spatial coordinate parameters, geometric dimension parameters, and unique identifier. The system attributes include the component's professional field, equipment category, operating function, and installation process requirements. The category attribute is mapped to a component type label, the system attribute is mapped to a component system label, the spatial coordinate parameter and geometric dimension parameter are mapped to a component location label, and the unique identifier is mapped to a component number label. The component type label, component system label, component location label, and component number label are combined one by one according to the component to form a component semantic label; All component semantic tags are aggregated to form a component semantic set.

[0020] In this embodiment, step two specifically includes: Based on the component system label and component location label of each component in the component semantic set, the BIM model space is non-uniformly divided; When the component system label belongs to the water supply and drainage system, fire protection system, electrical system, or HVAC system, the size is set to a semantic review grid cell with a side length of 0.4 meters; when the component system label belongs to the structural system or interior decoration system, the size is set to a semantic review grid cell with a side length of 0.6 meters; when the component system label does not belong to the water supply and drainage system, fire protection system, electrical system, HVAC system, structural system, or interior decoration system, the size is set to a semantic review grid cell with a side length of 0.3 meters. When the component location label belongs to an evacuation corridor, pipe shaft, equipment room, refuge floor, or fire elevator lobby, the grid division size of the corresponding area is reduced to 80% of the original size based on the component system label division. Each semantic review grid cell is bound to a unique identifier of its corresponding component, and the component type label and component system label are written into the review attribute field of the semantic review grid cell. All semantic review grid units are merged to form a semantic review grid unit set; In the immersive review process of architectural design deliverables based on BIM and AR, the core purpose of spatial division is to provide refined spatial unit support for subsequent perspective interaction analysis, occlusion identification, and risk assessment. Adopting a non-uniform division strategy, tailored to the system attributes and spatial location characteristics of different components, helps improve review accuracy while ensuring computational efficiency. This invention controls the basic division granularity through component system tags. For electromechanical systems such as water supply and drainage, fire protection, electrical, and HVAC, which have complex structures and high maintenance requirements, a side length of 0.4 meters is used; for structural and decoration systems with stable layouts, a side length of 0.6 meters is used; and for other systems, a default minimum size of 0.3 meters is set to improve the identification accuracy of atypical areas. Simultaneously, by combining component location tags to identify key functional spaces, the basic dimensions are further reduced by 80% to achieve localized high-density division. Each semantic review grid unit is a cubic structure, bound to a unique component identifier and its semantic attributes, achieving a consistent mapping between spatial data and component semantics. This facilitates risk location, path guidance, and closed-loop review management in the AR environment.

[0021] In this embodiment, step three specifically includes: The BIM model and semantic review grid cell set are exported in a unified three-dimensional coordinate format and loaded into the AR device through the glTF extended format; In the physical scene, at least three component surfaces are selected as spatial anchor points. Based on the camera of the AR device, image frames of the spatial anchor points are acquired and the corresponding two-dimensional image coordinates are extracted. The corresponding three-dimensional anchor point coordinates are retrieved in the BIM model to construct a point pair set of two-dimensional image coordinates and three-dimensional anchor point coordinates. Based on the point pair set, the PnP algorithm is used to calculate the initial pose of the AR device relative to the BIM model coordinate system. In each iteration, the camera rotation and translation parameters are corrected by comparing the pixel error between the virtual projection point and the actual image point until the pixel error is less than the preset accuracy threshold or the preset maximum number of iterations is reached, and the perspective projection matrix is ​​output. The perspective projection matrix is ​​applied to the spatial coordinates of all components in the BIM model to achieve spatial registration between the BIM model and the real scene. The semantic review grid unit set is loaded into the AR device as a semi-transparent layer with an opacity of 0.3 and displayed on the surface of the spatially registered BIM model components. It presents an overlay effect in the user's field of vision and is rendered and updated in real time as the user's spatial position and viewing angle change. This invention initializes the camera's pose estimation by calling the PnP algorithm, which solves for the camera's rotation matrix and translation vector, so that the 3D points, after projection, fit their 2D positions in the image as closely as possible. It also combines the RANSAC algorithm to eliminate incorrect matches and improve the robustness of the initial solution. In the iterative optimization stage, the rotation and translation parameters are continuously adjusted by minimizing the reprojection error until the pixel error is less than a preset accuracy threshold or the preset maximum number of iterations is reached, resulting in a perspective projection matrix, which is used to achieve visual alignment between the real scene and the virtual model.

[0022] In this embodiment, step four specifically includes: The AR device acquires the user's spatial position and viewing angle parameters, and, in conjunction with the AR device's built-in field of view, calculates and generates a visual cone with the user's eye center point as the vertex and the head orientation as the main axis. The view frustum is projected onto the three-dimensional coordinate space where the semantic review grid unit set is located. Using a ray-by-ray projection method, multiple directional rays are emitted outward from the vertex of the view frustum at a set resolution. Each ray is then sequentially checked to see if it intersects with the boundary of each semantic review grid unit. If the ray intersects with the boundary of the semantic review grid unit, and the intersection point is within a set threshold range around the center point of the semantic review grid unit, then the interaction state between the semantic review grid unit and the view frustum is recorded as a direct hit; if the ray intersects with the boundary of the semantic review grid unit, and the intersection point is outside a set threshold range around the center point of the semantic review grid unit, then the interaction state between the semantic review grid unit and the view frustum is recorded as edge visible; if the ray does not intersect with the boundary of the semantic review grid unit, then the interaction state between the semantic review grid unit and the view frustum is recorded as no interaction. Simultaneously, the spatial location and timestamp of the user within each view frustum update cycle are recorded. The interaction status and duration of all semantic review grid units that are judged to be directly hit or edge visible are summarized within the update cycle, and a user interaction record set is output. The user interaction record set includes the unique identifier of the semantic review grid unit, the interaction status, the start time, and the end time.

[0023] In this embodiment, the improved MambaVision visual recognition model adopts a state evolution-driven sequence feature extraction structure to initialize a state vector for each pixel position of the input image frame. The state evolution-driven sequence feature extraction structure includes a component region sequence channel and a boundary mutation sequence channel. The two sequence channels model the temporal dependency relationship between the main component and the background transition region, respectively, and the state output is weighted and integrated through a gated fusion module after each layer calculation. During the state update process, a modulation function based on pixel spatial coordinates is introduced to adjust the spatial sensitivity of the update weights in the state transition. The modulation function takes the two-dimensional coordinates of the pixel in the image coordinate system as input, normalizes them to the [0,1] interval, and multiplies them element by element with each dimension component of the previous layer's state vector to generate a modulation coefficient vector. The modulation coefficient vector is used to scale the element weights of the current layer's state transition matrix item by item to achieve differentiated updates of the state at different spatial locations, so that the state change rate near the component edge and geometric change region is higher than that in the homogeneous region inside the component. Pixel-wise mapping is performed on all modulated and updated state vectors to generate an image semantic embedding feature map with the same size as the input image frame. In the state update process of the improved MambaVision visual recognition model, the state vector represents the feature representation of each pixel position in the image frame at the current sequence modeling layer, and evolves continuously as the model layers advance. The state transition matrix is ​​the core structure in state space modeling, and is a set of trainable parameter matrices used to model the state evolution relationship over time or between layers. The weight of each element in the state transition matrix determines the update ratio and mutual influence of each component of the current state vector. In the standard model, the state transition matrix is ​​uniformly shared in the spatial dimension and does not have the ability to express differences based on pixel positions.

[0024] To introduce spatial sensitivity, this invention designs a modulation function based on pixel spatial coordinates to dynamically adjust the scaling ratio of the weight of each element in the state transition matrix. The modulation function takes the two-dimensional image coordinates of each pixel in the image frame as input, first normalizes them to the [0,1] interval as a position-aware factor, and then multiplies the normalized position coordinates element-wise with the state vector of the previous layer of the pixel to generate a modulation coefficient vector with the same dimension as the state. The modulation coefficient vector does not directly affect the state update, but is used to scale the element weights of the current layer's state transition matrix item by item, so that each spatial location has an independent set of state update weights.

[0025] Through the aforementioned modulation mechanism, the MambaVision visual recognition model can perform differentiated modeling of state vectors at different spatial locations. Especially in areas of geometric or semantic abrupt change, such as component boundaries, structural seams, and material transitions, the modulation coefficients amplify the state transition weights and enhance the local state update capability because the changes in pixel position and state response are synchronized and significant. In contrast, the modulation coefficients tend to be flatter inside components or in the background, limiting drastic changes in state updates, thereby improving the model's stability and feature abstraction capability.

[0026] In this embodiment, the step of generating an occlusion probability map by combining the semantic review grid cell set with the component space mapping relationship specifically involves: The semantic embedding feature map of the image corresponding to the user's current viewpoint is jointly processed with the viewpoint parameters and spatial registration information of the AR device to calculate the corresponding coordinate position of each pixel in the image frame in the three-dimensional space of the BIM model. Based on the component space mapping relationship, the projection area of ​​each semantic review grid unit in the image frame is determined, and the semantic embedding features of all pixels in the projection area are extracted. After sorting the extracted semantic embedding features according to pixel coordinates, the principal component direction vectors distributed in the semantic embedding space are statistically analyzed, and the Mahalanobis distance between the principal component direction vectors and the mean vector of the embedding features corresponding to the component system labels is calculated. When the Mahalanobis distance exceeds a set threshold, the pixel is marked as an unrecognizable pixel; The ratio between the number of unidentifiable pixels in the image region corresponding to each semantic review grid unit and the total number of pixels is statistically analyzed to calculate the occlusion probability value. The occlusion probability values ​​of all semantic review grid units are then used to generate an occlusion probability map based on their spatial location mapping relationship. In this invention, each pixel in the image semantic embedding feature map corresponds to a multi-dimensional semantic vector, representing the image semantic information extracted from that pixel after passing through the MambaVision visual recognition model. To determine whether a pixel in a certain region is occluded, it is necessary to evaluate whether its semantic expression is consistent with the component features that the region should present. To this end, firstly, the projection region of a semantic review grid unit in the image frame is extracted based on the viewpoint projection relationship, and the semantic embedding features of all pixels in that region are collected.

[0027] For the semantic embedding features of all the aforementioned pixels, principal component analysis (PCA) is employed to extract the principal component directions in the semantic embedding space. These principal component directions reflect the main trends in the semantic distribution of pixels in that region. Based on the component system labels bound to the semantic review grid units, the mean vector of the standard semantic embedding features of the corresponding category components is statistically obtained during the training phase, and this mean is used as the central reference for the component semantic space.

[0028] Calculate the Mahalanobis distance between the principal component direction vector and the mean vector of the embedded features corresponding to the component system label. Mahalanobis distance comprehensively considers the covariance relationship between various dimensions and reflects the true distribution differences in high-dimensional semantic space better than Euclidean distance. If the Mahalanobis distance exceeds a set judgment threshold, it indicates that the semantic expression of the current pixel deviates from the standard semantic range of the component, and it can be inferred that the pixel is occluded or semantically confused, and is marked as an unrecognizable pixel.

[0029] This technique improves the accuracy and adaptability of occlusion detection by dynamically modeling occlusion scenarios based on actual semantic distribution without relying on fixed feature templates.

[0030] In this embodiment, step six specifically includes: Read the component system label in the review attribute field of each semantic review grid unit, call the interaction status and interaction duration in the user interaction record set, and match the occlusion probability value at the corresponding position in the occlusion probability map; The review level weight coefficient is set based on the component system label in the review attribute field. Different review level weight coefficients are assigned to various systems according to their varying importance in architectural design review, reflecting their impact on the overall review risk. For example, critical systems such as water supply and drainage and fire protection are assigned higher review level weight coefficients due to their safety functions, while interior decoration systems are assigned relatively lower review level weight coefficients due to their smaller impact, thus reflecting the review priority in the scoring. The interaction state and occlusion probability value are jointly calculated. The interaction factor for semantic review grid units with a directly hit interaction state is set to 1, the interaction factor for semantic review grid units with an edge-visible interaction state is set to 0.6, and the interaction factor for semantic review grid units with a non-interactive interaction state is set to 0. The interaction score is obtained by multiplying the interaction factor of the semantic review grid unit by the interaction duration. The interaction score and the occlusion probability value are weighted and fused according to a preset ratio to obtain the visibility risk value of the semantic review grid unit, and then multiplied by the review level weight coefficient to obtain the review risk score of the semantic review grid unit.

[0031] In this embodiment, step seven specifically includes: The review risk scores of all semantic review grid units are screened, and semantic review grid units whose review risk scores exceed the preset risk threshold are extracted and marked as high-risk semantic review grid units; The spatial coordinates of the high-risk semantic review grid unit are compared and analyzed with the user's current view parameters. When the high-risk semantic review grid unit is not in the current view cone, or the area displayed in the view is smaller than the set area threshold, the view guidance prompt mechanism is activated. The perspective guidance and prompting mechanism includes generating a dynamic arrow at the edge of the user's current perspective in the AR device interface to guide the user to adjust the perspective direction to face the actual spatial orientation of the high-risk semantic review grid unit. When the high-risk semantic review grid unit is fully within the user's field of view and the observation time exceeds the set duration threshold, the supplementary review of the high-risk semantic review grid unit is marked as complete.

[0032] refer to Figure 2 An immersive review system for architectural design deliverables based on BIM and AR includes the following modules: The component semantic parsing module is used to perform semantic parsing on components in the BIM model and generate a component semantic set. The spatial partitioning module is used to non-uniformly partition the BIM model space based on the semantic set of components, construct a set of semantic review grid units, and bind a unique identifier and review attribute. The AR loading and registration module is used to load the BIM model and semantic review grid unit set into the AR device, complete the spatial registration between the BIM model and the real scene, and present the semantic review grid unit in the user's field of vision of the AR device in the form of an overlay layer. The viewpoint tracking and interaction acquisition module is used to track the user's viewpoint and spatial position, generate a view frustum, and output a set of user interaction records. The visual recognition and occlusion analysis module is used to call the improved MambaVision visual recognition model to generate image semantic embedding feature maps, and combine the semantic review grid cell set and component space mapping relationship to generate occlusion probability maps; The review risk assessment module is used to calculate the review risk score of the semantic review grid unit; The perspective guidance prompt module is used to identify high-risk semantic review grid cells and provide perspective guidance prompts in AR devices to assist users in supplementing the review.

[0033] Example 1

[0034] To verify the feasibility of this invention in practice, it was applied to the review of a BIM model linked with on-site AR technology for a high-rise complex project. The project has a building area of ​​approximately 82,000 square meters, including office buildings, commercial podiums, and underground parking spaces, encompassing six major professional systems: structure, architecture, electrical, water supply and drainage, HVAC, and fire protection. The review objective was to evaluate the spatial matching accuracy, occlusion recognition accuracy, and review efficiency of this invention in complex system scenarios.

[0035] In actual testing, after processing by the component semantic parsing module, the BIM model yielded 21,847 component semantic tags and 12,600 semantic review grid cells. The system used the PnP algorithm to calculate the spatial pose of the AR device relative to the BIM model, with an average registration error of 2.3 cm. Reviewers used AR headsets to inspect each floor of the building; the system tracked the viewing angle in real time and generated a view frustum, determining the interaction status through light projection. At a resolution of 1280×720, the improved MambaVision visual recognition model achieved an average frame processing time of 58 milliseconds, achieving a stable real-time analysis rate of 30 frames per second. The occlusion probability map recognition accuracy was 94.6%, with a false negative rate of 3.7%, accurately reflecting the occlusion between equipment pipelines and structural components.

[0036] To further quantify system performance, the main performance indicators of three modes were compared: traditional manual review of 2D drawings, desktop review of ordinary BIM models, and the BIM+AR immersive review system proposed in this invention.

[0037] Table 1 Performance Comparison of Immersive Review Systems for Architectural Design Outcomes Indicator Name Traditional 2D drawing review Standard BIM Desktop Review This invention provides an immersive BIM+AR review experience. Average review time per unit model (min) 14.6 8.2 4.4 Number of components that can be reviewed per person per day (pieces) 135 226 398 Model space registration error (cm) — 6.7 2.3 Occlusion recognition accuracy (%) — 82.5 94.6 Risk identification rate (%) 68.4 79.2 93.7 False positive rate (%) 7.9 4.3 2.8 Average number of design conflicts found (items / times) 41 63 95 Completion rate of supplementary review in high-risk areas (%) — 76.5 98.2 Average operation response time (ms) — 112 58 User subjective experience satisfaction rating (out of 10) 6.1 7.4 9.3 As can be seen from the data in Table 1 above, the system proposed in this invention performs excellently in many core indicators. Especially in the spatial occlusion identification and review risk identification stages, thanks to the improved semantic embedding and spatial modulation mechanism of the MambaVision visual recognition model, the system can generate high-resolution occlusion probability maps in real time and achieve dynamic review risk scoring by combining interaction records. Compared with ordinary BIM desktop review, the number of design conflicts discovered increased by 51%, and the false positive rate decreased to 2.8%. Furthermore, through the dynamic arrow guidance mechanism in the AR interface, users can quickly locate high-risk semantic review grid cells, significantly reducing the average supplementary review time. User subjective experience surveys show that participants generally believe that the system of this invention is natural to operate, has a strong sense of space, and provides intuitive problem location, significantly improving the collaborative review efficiency of complex building systems.

[0038] This embodiment achieves high-precision spatial registration of BIM components in real-world scenarios by deeply integrating the BIM model with AR devices, and identifies user interaction states in real time based on view tracking and ray casting algorithms. Simultaneously, by combining an improved MambaVision visual recognition model to extract image semantic embedding features and generate an occlusion probability map, the system significantly improves the accuracy of occlusion area identification. Furthermore, by reviewing attribute fields, interaction records, and occlusion probability values, a quantitative review risk score is output, achieving simultaneous assessment of visibility and risk. Compared to traditional manual review methods, this invention effectively reduces review blind spots, improves review efficiency and risk identification accuracy, and provides an efficient, intuitive, and intelligent solution for collaborative design review of complex building systems.

[0039] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An immersive review method for architectural design deliverables based on BIM and AR, characterized in that, Includes the following steps: Step 1: Obtain the BIM model and perform semantic parsing on the components in the BIM model to form a component semantic set; Step 2: Based on the semantic set of the components, the BIM model space is non-uniformly divided to construct a set of semantic review grid units. Each semantic review grid unit is bound to a unique identifier and review attribute. Step 3: Load the BIM model and the semantic review grid set into the AR device to complete the spatial registration between the BIM model and the real scene, and present the semantic review grid set in the user's field of vision of the AR device as an overlay layer; Step 4: Using the AR device, the user's spatial position and viewing angle parameters are tracked in real time to generate a view frustum. The interaction status between each semantic review grid unit and the view frustum is determined, and a user interaction record set is output. Step 5: Collect the image frame corresponding to the user's current viewpoint, input it into the improved MambaVision visual recognition model to generate an image semantic embedding feature map, and combine the semantic review grid cell set with the component space mapping relationship to generate an occlusion probability map; Step 6: For each semantic review grid cell, calculate the review risk score based on the review attribute fields, the interaction status in the user interaction record set, and the occlusion probability value in the occlusion probability map. Step 7: Select high-risk semantic review grid units whose review risk score exceeds the preset risk threshold, and provide perspective guidance prompts for the high-risk semantic review grid units in the AR device to assist users in conducting supplementary reviews.

2. The immersive review method for architectural design deliverables based on BIM and AR according to claim 1, characterized in that, Step one specifically involves: Extract the component's category attributes, system attributes, spatial coordinate parameters, geometric dimension parameters, and unique identifier. The system attributes include the component's professional field, equipment category, operating function, and installation process requirements. The category attribute is mapped to a component type label, the system attribute is mapped to a component system label, the spatial coordinate parameter and geometric dimension parameter are mapped to a component location label, and the unique identifier is mapped to a component number label. The component type label, component system label, component location label, and component number label are combined one by one according to the component to form a component semantic label; All component semantic tags are aggregated to form a component semantic set.

3. The immersive review method for architectural design deliverables based on BIM and AR according to claim 1, characterized in that, Step two specifically involves: Based on the component system label and component location label of each component in the component semantic set, the BIM model space is non-uniformly divided; When the component system label belongs to the water supply and drainage system, fire protection system, electrical system, or HVAC system, the size is set to a semantic review grid cell with a side length of 0.4 meters; when the component system label belongs to the structural system or interior decoration system, the size is set to a semantic review grid cell with a side length of 0.6 meters; when the component system label does not belong to the water supply and drainage system, fire protection system, electrical system, HVAC system, structural system, or interior decoration system, the size is set to a semantic review grid cell with a side length of 0.3 meters. When the component location label belongs to an evacuation corridor, pipe shaft, equipment room, refuge floor, or fire elevator lobby, the grid division size of the corresponding area is reduced to 80% of the original size based on the component system label division. Each semantic review grid cell is bound to a unique identifier of its corresponding component, and the component type label and component system label are written into the review attribute field of the semantic review grid cell. All semantic review grid units are merged to form a semantic review grid unit set.

4. The immersive review method for architectural design deliverables based on BIM and AR according to claim 1, characterized in that, Step three specifically involves: The BIM model and semantic review grid cell set are exported in a unified three-dimensional coordinate format and loaded into the AR device through the glTF extended format; In the physical scene, at least three component surfaces are selected as spatial anchor points. Based on the camera of the AR device, image frames of the spatial anchor points are acquired and the corresponding two-dimensional image coordinates are extracted. The corresponding three-dimensional anchor point coordinates are retrieved in the BIM model to construct a point pair set of two-dimensional image coordinates and three-dimensional anchor point coordinates. Based on the point pair set, the PnP algorithm is used to calculate the initial pose of the AR device relative to the BIM model coordinate system; In each iteration, the camera rotation and translation parameters are corrected by comparing the pixel error between the virtual projection point and the actual image point until the pixel error is less than the preset accuracy threshold or the preset maximum number of iterations is reached, and then the perspective projection matrix is ​​output. The perspective projection matrix is ​​applied to the spatial coordinates of all components in the BIM model to achieve spatial registration between the BIM model and the real scene. The semantic review grid unit set is loaded into the AR device as a semi-transparent layer with an opacity of 0.3 and displayed on the surface of the spatially registered BIM model components. It presents an overlay effect in the user's field of vision and is rendered and updated in real time as the user's spatial position and viewing angle change.

5. The immersive review method for architectural design deliverables based on BIM and AR according to claim 1, characterized in that, Step four specifically involves: The AR device acquires the user's spatial position and viewing angle parameters, and, in conjunction with the AR device's built-in field of view, calculates and generates a visual cone with the user's eye center point as the vertex and the head orientation as the main axis. The view frustum is projected onto the three-dimensional coordinate space where the semantic review grid unit set is located. Using a ray-by-ray projection method, multiple directional rays are emitted outward from the vertex of the view frustum at a set resolution. Each ray is then sequentially checked to see if it intersects with the boundary of each semantic review grid unit. If the ray intersects with the boundary of the semantic review grid unit, and the intersection point is within a set threshold range around the center point of the semantic review grid unit, then the interaction state between the semantic review grid unit and the view frustum is recorded as a direct hit; if the ray intersects with the boundary of the semantic review grid unit, and the intersection point is outside a set threshold range around the center point of the semantic review grid unit, then the interaction state between the semantic review grid unit and the view frustum is recorded as edge visible; if the ray does not intersect with the boundary of the semantic review grid unit, then the interaction state between the semantic review grid unit and the view frustum is recorded as no interaction. Simultaneously, the spatial location and timestamp of the user within each view frustum update cycle are recorded. The interaction status and duration of all semantic review grid units that are judged to be directly hit or edge visible are summarized within the update cycle, and a user interaction record set is output. The user interaction record set includes the unique identifier of the semantic review grid unit, the interaction status, the start time, and the end time.

6. The immersive review method for architectural design deliverables based on BIM and AR according to claim 1, characterized in that, The improved MambaVision visual recognition model adopts a state evolution-driven sequence feature extraction structure to initialize a state vector for each pixel position of the input image frame. The state evolution-driven sequence feature extraction structure includes a component region sequence channel and a boundary mutation sequence channel. The two sequence channels model the temporal dependency relationship between the main component and the background transition region, respectively, and the state output is weighted and integrated through a gated fusion module after each layer calculation. During the state update process, a modulation function based on pixel spatial coordinates is introduced to adjust the spatial sensitivity of the update weights in the state transition. The modulation function takes the two-dimensional coordinates of the pixel in the image coordinate system as input, normalizes them to the [0,1] interval, and multiplies them element by element with each dimension component of the previous state vector to generate a modulation coefficient vector. The modulation coefficient vector is used to scale the element weights of the current layer state transition matrix item by item, so as to realize the differentiated update of the state at different spatial locations, and make the state change rate near the edge of the component and the geometric change region higher than that in the homogeneous region inside the component. Pixel-wise mapping is performed on all modulated and updated state vectors to generate an image semantic embedding feature map with the same size as the input image frame.

7. The immersive review method for architectural design deliverables based on BIM and AR according to claim 1, characterized in that, The process of generating an occlusion probability map by combining the semantic review grid cell set with the component space mapping relationship is as follows: The semantic embedding feature map of the image corresponding to the user's current viewpoint is jointly processed with the viewpoint parameters and spatial registration information of the AR device to calculate the corresponding coordinate position of each pixel in the image frame in the three-dimensional space of the BIM model. Based on the component space mapping relationship, the projection area of ​​each semantic review grid unit in the image frame is determined, and the semantic embedding features of all pixels in the projection area are extracted. After sorting the extracted semantic embedding features according to pixel coordinates, the principal component direction vectors distributed in the semantic embedding space are statistically analyzed, and the Mahalanobis distance between the principal component direction vectors and the mean vector of the embedding features corresponding to the component system labels is calculated. When the Mahalanobis distance exceeds a set threshold, the pixel is marked as an unrecognizable pixel; The ratio between the number of unidentifiable pixels in the image region corresponding to each semantic review grid unit and the total number of pixels is statistically analyzed to calculate the occlusion probability value. The occlusion probability values ​​of all semantic review grid units are then used to generate an occlusion probability map based on their spatial location mapping relationship.

8. The immersive review method for architectural design deliverables based on BIM and AR according to claim 1, characterized in that, Step six specifically involves: Read the component system label in the review attribute field of each semantic review grid unit, call the interaction status and interaction duration in the user interaction record set, and match the occlusion probability value at the corresponding position in the occlusion probability map; The review level weight coefficient is set according to the component system label in the review attribute field. The interaction state and occlusion probability value are jointly calculated. The interaction factor of the semantic review grid unit with the interaction state of direct hit is set to 1, the interaction factor of the semantic review grid unit with the interaction state of edge visible is set to 0.6, and the interaction factor of the semantic review grid unit with the interaction state of no interaction is set to 0. The interaction score is obtained by multiplying the interaction factor of the semantic review grid unit by the interaction duration. The interaction score and the occlusion probability value are weighted and fused according to a preset ratio to obtain the visibility risk value of the semantic review grid unit, and then multiplied by the review level weight coefficient to obtain the review risk score of the semantic review grid unit.

9. The immersive review method for architectural design deliverables based on BIM and AR according to claim 1, characterized in that, Step seven specifically involves: The review risk scores of all semantic review grid units are screened, and semantic review grid units whose review risk scores exceed the preset risk threshold are extracted and marked as high-risk semantic review grid units; The spatial coordinates of the high-risk semantic review grid unit are compared and analyzed with the user's current view parameters. When the high-risk semantic review grid unit is not in the current view cone, or the area displayed in the view is smaller than the set area threshold, the view guidance prompt mechanism is activated. The perspective guidance and prompting mechanism includes generating a dynamic arrow at the edge of the user's current perspective in the AR device interface to guide the user to adjust the perspective direction to face the actual spatial orientation of the high-risk semantic review grid unit. When the high-risk semantic review grid unit is fully within the user's field of view and the observation time exceeds the set duration threshold, the supplementary review of the high-risk semantic review grid unit is marked as complete.

10. An immersive review system for architectural design deliverables based on BIM and AR, comprising the immersive review method for architectural design deliverables based on BIM and AR as described in any one of claims 1 to 9, characterized in that, Includes the following modules: The component semantic parsing module is used to perform semantic parsing on components in the BIM model and generate a component semantic set. The spatial partitioning module is used to non-uniformly partition the BIM model space based on the semantic set of components, construct a set of semantic review grid units, and bind a unique identifier and review attribute. The AR loading and registration module is used to load the BIM model and semantic review grid unit set into the AR device, complete the spatial registration between the BIM model and the real scene, and present the semantic review grid unit in the user's field of vision of the AR device in the form of an overlay layer. The viewpoint tracking and interaction acquisition module is used to track the user's viewpoint and spatial position, generate a view frustum, and output a set of user interaction records. The visual recognition and occlusion analysis module is used to call the improved MambaVision visual recognition model to generate image semantic embedding feature maps, and combine the semantic review grid cell set and component space mapping relationship to generate occlusion probability maps; The review risk assessment module is used to calculate the review risk score of the semantic review grid unit; The perspective guidance prompt module is used to identify high-risk semantic review grid cells and provide perspective guidance prompts in AR devices to assist users in supplementing the review.

Images