A BIM model mobile terminal and mixed reality collaboration method, system and medium

Abstract

The application relates to a BIM model mobile terminal and mixed reality collaboration method, system and medium, wherein the method comprises: according to a pre-created index structure, performing spatial segmentation and spatial grid division processing on a BIM model, obtaining index information comprising a plurality of space models and a plurality of space grids, and constructing a unique identification code of the space model based on the index information. The space model corresponding to the identification code is loaded through mobile terminal identification of the identification code, the space coordinates in the space model are obtained according to a gesture operation, the space coordinates are converted into space grid index information through a preset conversion rule, and a loading instruction of a mixed reality device is generated. The mixed reality device loads the space model corresponding to the space grid index information according to the loading instruction, and superimposes the space model onto a real building scene through a positioning strategy. The problem of low BIM model mobile terminal and mixed reality collaboration efficiency in the related art is solved.

Description

Technical Field

[0001] This application relates to the field of building information modeling technology, and in particular to a collaborative method, system and medium for BIM model mobile terminals and mixed reality. Background Technology

[0002] The collaborative application of Building Information Modeling (BIM) and Mixed Reality (MR) technologies has demonstrated enormous potential in scenarios such as building construction and operation and maintenance management. BIM provides rich digital information about buildings, while MR technology can intuitively overlay this information onto the real environment to assist on-site operations.

[0003] However, existing BIM model and MR collaboration methods consume too many resources when loading BIM models into MR devices, resulting in low efficiency in overlaying spatial models onto real building scenes.

[0004] Therefore, there is an urgent need to provide an efficient method for collaborating BIM models with mixed reality to solve the above problems. Summary of the Invention

[0005] This application provides an efficient method for collaborating BIM models with mixed reality, which at least solves the problem of low efficiency in collaborating BIM models with mobile devices and mixed reality in related technologies.

[0006] In a first aspect, embodiments of this application provide a collaborative method between a mobile BIM model and mixed reality, the method comprising:

[0007] Based on the pre-created index structure, the BIM model is spatially segmented and divided into spatial grids to obtain index information including multiple spatial models and multiple spatial grids. Based on the index information, a unique identifier code for the spatial model is constructed.

[0008] The mobile device identifies the identification code, loads the spatial model corresponding to the identification code, obtains the spatial coordinates in the spatial model based on gesture operations, converts the spatial coordinates into spatial grid index information through a preset conversion rule, and generates a loading instruction for the mixed reality device.

[0009] According to the loading instruction, the mixed reality device loads the spatial model corresponding to the spatial grid index information, and superimposes the spatial model onto the real building scene through a positioning strategy.

[0010] In one embodiment, the BIM model is spatially segmented and gridded according to a pre-created index structure to obtain index information including multiple spatial models and multiple spatial grids. Based on the index information, a unique identifier for each spatial model is constructed, including:

[0011] The BIM model is divided according to the spatial index in the index structure to generate multiple spatial models, and the index information of the spatial models is obtained.

[0012] The spatial grid is divided based on the actual size of the spatial model and the preset grid density. Based on the divided spatial grid, the spatial grid index information and the spatial grid bounding box are determined.

[0013] Based on the index information of the spatial model and the index information of the spatial grid, a unique identifier for the spatial model is generated using a hash algorithm.

[0014] In one embodiment, the BIM model is segmented according to the space in the index structure to generate multiple spatial models, and the index information of the spatial models is obtained, including:

[0015] Obtain the geometric vertex coordinate data of the model with a spatial index structure, and determine the three-dimensional bounding box of the spatial model based on the geometric vertex coordinate data by calculating the coordinate extrema.

[0016] Based on the three-dimensional bounding box, a geometric filtering algorithm is used to retain building components and cross-space connection points located inside the spatial model, generating multiple initial spatial models. Among the multiple spatial models, cross-space components are anchored by connection points to determine the topological integrity of the cross-space components.

[0017] The initial spatial model is converted into a GLB format spatial model using a 3D graphics compression and conversion algorithm.

[0018] Based on the index structure, the index information of the spatial model is generated through multi-level digital encoding rules.

[0019] In one embodiment, the step of dividing the spatial grid based on the actual size and preset grid density of the spatial model, and determining the spatial grid index information and spatial grid bounding box based on the divided spatial grid, includes:

[0020] Set the size parameters of the standard grid according to the preset grid division density;

[0021] Based on the geometric boundary data of the spatial model, the actual physical dimensions of the spatial model are obtained through a three-dimensional dimension measurement algorithm;

[0022] Based on the ratio between the actual physical size and the standard mesh size, the number of spatial meshes in the spatial model is calculated using a mesh coverage algorithm. Based on the number of spatial meshes, the model is divided into multiple spatial meshes.

[0023] Through multi-level encoding rules, spatial grid index information is generated for each grid, and the spatial grid index information includes spatial location information.

[0024] Based on the spatial location information, the grid bounding box of each grid is determined by a coordinate allocation algorithm.

[0025] In one embodiment, converting the spatial coordinates into spatial grid index information using a preset transformation rule includes:

[0026] Based on the spatial coordinates obtained from the mobile device, the spatial coordinates of the mobile device are converted into architectural spatial coordinates in the architectural coordinate system using a coordinate system transformation matrix;

[0027] A grid boundary list is constructed based on the bounding box information of the spatial grid. In the grid boundary list, the initial grid boundary corresponding to the building spatial coordinates is determined by a spatial hash acceleration algorithm based on the three-level index of row, column and layer.

[0028] Determine whether the initial mesh boundary satisfies the set of mesh boundary inequalities. If it does, take the initial mesh boundary as the target mesh boundary and determine that the target mesh boundary is located within a predetermined three-dimensional space.

[0029] Based on the target grid boundary, spatial grid index information is determined according to the index structure, wherein the spatial grid index information includes grid identifier and grid boundary box.

[0030] In one embodiment, the grid boundary corresponding to the building spatial coordinates is determined in the grid boundary list using a three-level index of rows, columns, and layers, including:

[0031] Based on the x-axis coordinate value of the building space, the row index number is calculated and obtained by the ratio of the minimum coordinate in the x-direction of the building space to the size of the grid in the x-direction.

[0032] Based on the y-axis coordinate value of the building space, the column index number is calculated and obtained by the ratio of the minimum coordinate in the y-direction of the building space to the size of the grid in the y-direction.

[0033] Based on the z-axis coordinate value of the building space, the layer index number is calculated and obtained by the ratio of the minimum coordinate in the z-direction of the building space to the size of the grid in the z-direction.

[0034] Based on the row index number, column index number, and layer index number, the grid boundary corresponding to the building spatial coordinates is determined.

[0035] In one embodiment, the mixed reality device includes a visual sensor, and the step of overlaying the spatial model onto a real architectural scene using a positioning strategy includes:

[0036] A set of feature points for a real building scene and the mapping relationship between the feature points and the world coordinates are pre-constructed.

[0037] The visual sensor of the mixed reality device scans the feature points of the QR code to extract the image coordinates of the spatial model; based on the mapping relationship, a correspondence between the image coordinates and the world coordinates is established.

[0038] Based on the correspondence, the rotation matrix and translation vector of the mixed reality device are obtained through an optimization algorithm. The optimization is performed by minimizing the sum of the reprojection errors of each feature point. The reprojection error represents the sum of squares of the differences between the image coordinates and the actual projected coordinates.

[0039] Verify that the reprojection error conforms to the preset distance. If it does, determine the preliminary positioning of the mixed reality device model and the real building scene.

[0040] Based on the rotation matrix and translation vector of the initial positioning, environmental point clouds in the real building scene are collected by the depth sensor of the mixed reality device, and model point clouds associated with the spatial model are loaded.

[0041] Through an iterative optimization process, the sum of the transformed Euclidean distances between the environmental point cloud points and the model point cloud points is minimized to obtain the optimal rotation matrix and translation vector. The transformation is to apply the current rotation matrix and translation vector to map the model point cloud points to the environmental point cloud space.

[0042] The iteration is stopped when the error change between two adjacent iterations is less than a preset condition, thus completing the superposition of the spatial model and the real building scene.

[0043] In one embodiment, before performing spatial segmentation and spatial gridding processing on the BIM model according to a pre-created index structure, the method further includes:

[0044] The BIM model is analyzed by three-dimensional boundary analysis, and the geometric boundary information of the spatial model is extracted from the BIM model. The geometric boundary information includes vertex coordinates and boundary length.

[0045] Based on the positional relationships between components, the spatial correlation between components is analyzed to generate topological relationship data, wherein the positional relationships include inclusion and adjacency relationships;

[0046] Based on the geometric boundary information and topological relationships, an index structure for the BIM model is generated through the hierarchical relationships of the BIM model. The index structure includes indexes for floors, spaces, and grids. The floor index includes floor identifiers, the space index includes the floor identifiers and space identifiers, and the grid index includes the floor identifiers, space identifiers, and grid identifiers.

[0047] In one embodiment, after superimposing the spatial model onto a real architectural scene using a positioning strategy, the method further includes:

[0048] Based on the priority message queue mechanism, the real-time transmission and processing of mixed reality device operation data is carried out through the stream processing engine. The data processing priority is determined by a dynamic weight allocation algorithm according to the type and urgency of the operation data.

[0049] Based on the grid spatial index, the association and binding between mixed reality device operation records and BIM components are established through a three-level mapping relationship. Based on the spatial inclusion relationship determination criteria, the ownership relationship between components and grids is determined.

[0050] Based on the version control mechanism, incremental data push is triggered by the difference detection algorithm, and data conflicts between multiple terminals are resolved by the weighted fusion algorithm according to the timestamp arbitration rules.

[0051] Secondly, embodiments of this application provide a collaborative system between a mobile BIM model and mixed reality, the system comprising: an identification code module, a loading instruction module, and an overlay module, wherein:

[0052] The identification code module is used to perform spatial segmentation and spatial grid division processing on the BIM model according to the pre-created index structure, obtain index information including multiple spatial models and multiple spatial grids, and construct a unique identification code for the spatial model based on the index information.

[0053] The loading instruction module is used to identify the identification code through the mobile terminal, load the spatial model corresponding to the identification code, obtain the spatial coordinates in the spatial model according to the gesture operation, convert the spatial coordinates into spatial grid index information through a preset conversion rule, and generate a loading instruction for the mixed reality device.

[0054] The overlay module is used by the mixed reality device to load the spatial model corresponding to the spatial grid index information according to the loading instruction, and to overlay the spatial model onto the real building scene through a positioning strategy.

[0055] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a collaborative method between a BIM model mobile terminal and mixed reality as described in the first aspect above.

[0056] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a collaborative method between a BIM model mobile terminal and mixed reality as described in the first aspect above.

[0057] The present application provides a collaborative method, system, and medium for BIM model mobile terminal and mixed reality, which has at least the following technical effects.

[0058] By spatially segmenting and meshing the BIM model through a pre-created index structure, a structured organization of building space data is achieved, improving the retrieval and processing efficiency of large-scale BIM models. A rapid identification and loading mechanism based on unique identifiers enables mobile devices to accurately locate and instantly retrieve the target spatial model, effectively reducing data transmission volume. Gesture operation and spatial coordinate transformation technology enable natural and intuitive human-computer interaction, significantly improving operational convenience. The use of mesh index information to generate mixed reality loading instructions ensures the accuracy and real-time performance of model loading. Finally, a high-precision virtual-real spatial positioning strategy achieves seamless overlay of the BIM model with the real building scene, solving the problem of low collaboration efficiency in traditional methods. This constructs an efficient, accurate, and easy-to-use method for BIM model mobile device and mixed reality collaboration.

[0059] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0060] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0061] Figure 1 This is a flowchart of a collaborative method between BIM model mobile devices and mixed reality;

[0062] Figure 2 This is a flowchart illustrating step S101 according to an exemplary embodiment;

[0063] Figure 3 This is a flowchart illustrating step S102 according to an exemplary embodiment;

[0064] Figure 4 This is a system structure block diagram illustrating a collaborative system between a BIM model mobile terminal and mixed reality, according to an exemplary embodiment.

[0065] Figure 5 This is a structural block diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0067] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0068] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0069] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0070] In this document, it should be understood that the terms used may be technical means used to implement part of the present invention or other summary technical terms. For example, the terms may include:

[0071] Spatial model: refers to the independent three-dimensional model unit that is divided from the BIM model according to spatial hierarchy (such as room, area), containing geometric data, attribute information and spatial topological relationships, and is used for local loading and interaction in mixed reality.

[0072] Spatial grid: A regular three-dimensional grid cell formed by dividing a spatial model according to a preset size. Each grid has a unique code and boundary definition, which is used for rapid positioning of spatial data.

[0073] Spatial coordinates: Describes the position data of the target point in three-dimensional space.

[0074] Unique Identifier: A string identifier generated based on a hash algorithm or hierarchical encoding, with the following structure:

[0075] Project ID-Floor ID-Space ID-Grid ID, used to uniquely identify spatial models or grid cells.

[0076] Mesh boundary list: A data structure that stores the boundary parameters of all mesh cells, recording the minimum or maximum X / Y / Z coordinates of each mesh.

[0077] 3D bounding box (AABB): An axis-aligned cube bounding box defined by minimum / maximum coordinate values.

[0078] World coordinates: A global coordinate system defined by the BIM model, which is mapped to the real scene through QR codes or feature point calibration.

[0079] Firstly, embodiments of this application provide a collaborative method between a mobile BIM model and mixed reality. Figure 1 This is a flowchart of a collaborative method between BIM model mobile devices and mixed reality, such as... Figure 1 As shown, the method includes:

[0080] Step S101: Based on the pre-created index structure, perform spatial segmentation and spatial grid division on the BIM model to obtain index information including multiple spatial models and multiple spatial grids. Based on the index information, construct a unique identifier for the spatial model.

[0081] Step S102: Identify the identification code through the mobile terminal, load the spatial model corresponding to the identification code, obtain the spatial coordinates in the spatial model according to the gesture operation, convert the spatial coordinates into spatial grid index information through the preset conversion rules, and generate the loading instruction for the mixed reality device.

[0082] Step S103: The mixed reality device loads the spatial model corresponding to the spatial grid index information according to the loading instruction, and superimposes the spatial model onto the real building scene through the positioning strategy.

[0083] In summary, this application provides a collaborative method for BIM model mobile devices and mixed reality. By using a pre-created index structure to spatially segment and mesh the BIM model, it achieves structured organization of building space data, improving the retrieval and processing efficiency of large-scale BIM models. A rapid identification and loading mechanism based on unique identifiers enables the mobile device to accurately locate and instantly retrieve the target spatial model, effectively reducing data transmission volume. Gesture operation and spatial coordinate transformation technology enable natural and intuitive human-computer interaction, significantly improving operational convenience. The use of grid index information to generate mixed reality loading instructions ensures the accuracy and real-time performance of model loading. Finally, a high-precision virtual-real spatial positioning strategy achieves seamless overlay of the BIM model and the real building scene, solving the problem of low collaborative efficiency in traditional methods. This constructs an efficient, accurate, and easy-to-use collaborative method for BIM model mobile devices and mixed reality.

[0084] Figure 2 This is a flowchart illustrating step S101 according to an exemplary embodiment, as follows: Figure 2As shown, step S101 involves spatial segmentation and spatial grid division of the BIM model based on a pre-created index structure, obtaining index information including multiple spatial models and multiple spatial grids, and constructing a unique identifier for the spatial model based on the index information. Specifically, this includes the following steps:

[0085] Step S1011: Divide the BIM model according to the spatial index in the index structure to generate multiple spatial models, and obtain the index information of the spatial models. Specifically, this includes the following steps:

[0086] Step 1: Obtain the geometric vertex coordinate data of the model with a spatial index structure. Based on the geometric vertex coordinate data, determine the 3D bounding box of the spatial model through coordinate extremum calculation.

[0087] Step 2: Based on the 3D bounding box, the building components and cross-space connection points located inside the spatial model are retained through the geometric filtering algorithm to generate multiple initial spatial models. Among these multiple spatial models, the cross-space components are anchored by the connection points to determine the topological integrity of the cross-space components.

[0088] Step 3: Convert the initial spatial model into a GLB format spatial model using a 3D graphics compression and conversion algorithm.

[0089] Step 4: Based on the index structure, generate the index information of the spatial model through multi-level digital encoding rules.

[0090] Optionally, based on the room's bounding box, the overall BIM model is trimmed, retaining the components within each room and the connections across spaces. For cross-space components such as pipes penetrating walls, a specific algorithm is used to maintain their topological integrity, ensuring that the model still reflects the actual physical connections after segmentation. The trimmed model is converted to GLB format, ensuring the model's lightweight nature and compatibility. Specifically:

[0091] Bounding box definition: The room bounding box is defined by its minimum or maximum coordinates, as shown in the following formula:

[0092]

[0093] In the formula, x min This represents the minimum position on the x-axis of the room's bounding box. x max This represents the maximum position of the room's bounding box on the x-axis; y min This represents the minimum position within the room's bounding box along the y-axis. y max This represents the maximum position of the room's bounding box along the y-axis; z minThis represents the minimum position on the z-axis of the room's bounding box. z max This represents the maximum position on the z-axis of the room's bounding box.

[0094] For cross-space components such as pipes that penetrate walls, the connection point anchoring method is used to identify the intersection points where the pipes cross the room boundary. C 1 , C 2 , ..., C i The geometric parameters (length, diameter, slope) of the pipeline segments between intersections are preserved, and the pipeline models of adjacent rooms are associated through the coordinates of the intersections. The formula is as follows:

[0095]

[0096] Where Room A is room A, and Room B is room B. i This refers to the number of specific intersection points. Among them, the connection point anchoring method is an algorithm used to maintain topological integrity when handling cross-space components (such as through-wall pipes, beams, etc.) during BIM model segmentation. This method identifies the connection points where components cross spatial boundaries, preserving their geometric and attribute information to ensure that the model segmentation still accurately reflects the actual physical connections.

[0097] Step S1012: Divide the spatial grid based on the actual size of the spatial model and the preset grid density. Based on the divided spatial grid, determine the spatial grid index information and the spatial grid bounding box. Specifically, this includes the following steps:

[0098] Step 1: Set the size parameters of the standard grid according to the preset grid division density;

[0099] Step 2: Based on the geometric boundary data of the spatial model, obtain the actual physical dimensions of the spatial model using a 3D dimension measurement algorithm;

[0100] Step 3: Based on the ratio between the actual physical size and the standard mesh size, calculate the number of spatial meshes in the spatial model using a mesh coverage algorithm, and then perform mesh division based on the number of spatial meshes to obtain multiple spatial meshes;

[0101] Step 4: Generate spatial grid index information for each grid through multi-level coding rules. The spatial grid index information includes spatial location information.

[0102] Step 5: Based on spatial location information, determine the grid bounding box of each grid using a coordinate allocation algorithm.

[0103] Optionally, the room can be subdivided into multiple spatial grids, each with its own spatial grid index and bounding box. A standard grid size is defined using configurable parameters (e.g., 5m × 5m) through a predefined `generate_space_grid()` function, and the number of grids is dynamically adjusted based on the actual size of the room. Wherein:

[0104] Dynamic calculation of grid number: Assume the actual room size is... L x (length), L y (Width), the default grid size is S x * S y (e.g., 5m x 5m), then the number of grid cells is:

[0105]

[0106] (Round up function to ensure complete coverage of room boundaries) Grid identifier encoding rules: A three-level encoding structure is adopted:

[0107]

[0108] Where i [1, n_x] is the row index, j [1, n_y] is the column index, GridID is the spatial grid index information, FloorID is the floor identifier, and RoomID is the space identifier. Grid identifier encoding rules:

[0109] The boundary coordinates of the (i,j)th grid:

[0110]

[0111] Furthermore, the generate_space_grid() function is used to automatically divide a room into a regular spatial grid, and its processing flow includes the following steps:

[0112] 1. Space Dimension Calculation: Based on the room's geometric bounding box, extract its actual physical dimensions (length, width, and height).

[0113] 2. Grid size adaptation: Based on the preset standard grid size (e.g., 5m×5m) and combined with the actual size of the room, the grid division strategy is dynamically adjusted;

[0114] 3. Grid Quantity Calculation: Based on the ratio between room size and standard grid size, the number of grid divisions for each axis is determined using an up-rounding algorithm;

[0115] 4. Mesh boundary and identifier generation: Based on the row and column layer rules, assign a unique identifier to each mesh cell and calculate its three-dimensional boundary coordinates;

[0116] 5. Index Mapping Construction: The generated grid identifiers and boundary information are mapped to the spatial index structure, supporting fast querying and positioning.

[0117] Step S1013: Based on the index information and spatial grid index information of the spatial model, generate a unique identifier for the spatial model using a hash algorithm.

[0118] Optionally, a unique QR code is generated for each room, including floor identifier, space identifier, grid identifier, and project code. The data associated with the QR code includes the room-level GLB model URL address and spatial grid index, facilitating quick access to relevant information by mobile devices. It supports direct access to the spatial model of the target room by scanning the QR code, simplifying on-site work processes. Specifically;

[0119] The structured data fields contained in the QR code:

[0120] Data={ProjectCode,FloorNo,RoomID,GLB_URL,GridIndexMap}

[0121] Among them, ProjectCode is the project identifier, FloorNo is the floor identifier, RoomID is the grid identifier, GLB model, URL address, and GridIndexMap is a mapping table between grid ID (representation) and boundary coordinates.

[0122] The unique identifier for the QR code is generated using a hash algorithm, as shown in the formula below:

[0123] QR_UUID = Hash(ProjectCode + FloorN} + RoomID)

[0124] The formula uses the SHA-256 hash function to ensure that the QR code for the same room is unique.

[0125] Encoding Conversion: The integrated data is converted into an ISO / IEC 18004 standard QR code matrix, and the recognition error tolerance is improved by using error correction code algorithms (such as Reed-Solomon codes).

[0126] Steps S1011-S1013 utilize a hierarchical index structure to spatially segment the BIM model, achieving logical organization of building data and significantly improving the management efficiency of large-scale models. A grid division method based on actual dimensions and preset density ensures standardized processing of spatial units and optimizes subsequent spatial computation performance. A hash algorithm is used to generate unique identifiers, establishing a globally unique identity for each spatial model, greatly improving the accuracy of data retrieval. By associating model data, URL addresses, and grid information through these identifiers, a complete data access channel is constructed, enabling rapid acquisition and integration of multi-source information.

[0127] In one embodiment, before performing spatial segmentation and spatial meshing processing on the BIM model according to a pre-created index structure in step S101, the method includes:

[0128] The BIM model is analyzed by three-dimensional boundary analysis, and the geometric boundary information of the spatial model is extracted from the BIM model. The geometric boundary information includes vertex coordinates and boundary length.

[0129] Based on the positional relationships between components, analyze the spatial correlation between components and generate topological relationship data, where positional relationships include containment and adjacency relationships;

[0130] Based on geometric boundary information and topological relationships, an index structure for the BIM model is generated through the hierarchical relationship of the BIM model. The index structure includes indexes for floors, spaces, and grids. The floor index includes floor identifiers, the space index includes floor identifiers and space identifiers, and the grid index includes floor identifiers, space identifiers, and grid identifiers.

[0131] Optionally, Revit spatial attribute extraction: Utilize the Autodesk Revit API (such as Autodesk.Revit.DB) to read and parse the geometric boundary parameters (e.g., vertex coordinates, boundary length, etc.) and spatial topological relationships (e.g., adjacency, containment, etc.) of Room and Space elements in the Revit model. Construct a tree-like index structure based on the hierarchical structure of "Floor ID → Room ID → Grid ID". Specifically:

[0132] The three-dimensional boundary vector analytical method resolves spatial topological relationships through three-dimensional vector operations.

[0133] Boundary vector generation:

[0134] Geometric boundary vertex coordinates of Room / Space elements P 1( x 1, y 1,z1), P 2( x 2, y2,z2)…… P n ( x n , y n ,z n ), calculate the vector difference between adjacent vertices:

[0135]

[0136] Topological relationship determination:

[0137] Adjacency: If the boundary vectors of two spaces have collinear and opposite components (their dot product is negative and their magnitudes are similar), they are considered adjacent. The formula is as follows:

[0138]

[0139] Containment relation: If all vertices of space A are within the minimum bounding box of space B, then A is considered to be contained in B, as expressed by the following formula:

[0140]

[0141] Hierarchical indexes are constructed using a tree-like encoding rule, and the index value is calculated using the following formula:

[0142]

[0143] Among them, the coefficients 10^6 and 10^3 are hierarchical separation constants, which can be adjusted according to the project scale.

[0144] By extracting geometric boundary information from the BIM model using 3D boundary analysis technology, accurate digital representation of the geometric features of building components was achieved, laying a data foundation for subsequent spatial analysis. A topology analysis algorithm based on component positional relationships automatically identifies spatial relationships such as inclusion and adjacency, significantly improving the efficiency of resolving building spatial relationships. By integrating geometric boundary and topological relationship data to construct a hierarchical index structure, a multi-level spatial index system including floors, rooms, and grids was formed.

[0145] Figure 3 This is a flowchart illustrating step S102 according to an exemplary embodiment, as follows: Figure 3 As shown, step S102 involves identifying the identifier code via the mobile terminal, loading the spatial model corresponding to the identifier code, obtaining the spatial coordinates in the spatial model based on gesture operations, converting the spatial coordinates into spatial grid index information using preset conversion rules, and generating a loading instruction for the mixed reality device. Specifically, this includes the following steps:

[0146] Step S1021: Identify the identification code through the mobile terminal and load the spatial model corresponding to the identification code.

[0147] Optionally, open-source QR code recognition libraries such as ZXing or commercial SDKs such as Zebra can be integrated to support offline scanning. After a user scans the space QR code, the system automatically parses the project code, floor number, room number, and other information, and loads the corresponding room-level BIM model and grid index data from the cloud. The data associated with the QR code includes the room-level GLB model URL and the grid spatial index map, facilitating quick access to relevant information on mobile devices. It supports direct access to the spatial model of the target room by scanning the QR code, simplifying on-site work processes, greatly improving work efficiency, and reducing the time consumed in traditional positioning processes.

[0148] Step S1022: Obtain spatial coordinates in the spatial model based on gesture operation.

[0149] Optionally, a WebGL or UE-based graphics engine can be integrated with gesture recognition capabilities to support common 3D operations such as rotation, scaling, component highlighting and selection. Gesture events are captured and processed through front-end JavaScript or UE plugins to ensure smooth and natural interaction and provide users with a seamless operating experience.

[0150] Specifically, users scan the QR codes deployed on the building site using their mobile devices. The system quickly parses the information contained within, such as the project code, floor number, and room number. Based on the parsing results, the mobile device quickly loads the corresponding room-level BIM model from the cloud and accurately renders and displays it in a 3D view. It supports rich interactive operations on the model via gesture recognition, including but not limited to: two-finger swipes to rotate the model around its center, allowing users to view the model from different angles; pinching or spreading two fingers to zoom in and out, flexibly adjusting the model's display size; drawing cross-sections with gestures or clicking buttons to switch cross-sectional views, clearly presenting the model's internal structure; and clicking on components to trigger highlighting, pop-up information display, or coordinate picking, facilitating users to obtain detailed component information and spatial location.

[0151] Users can obtain a unique identifier (component ID) by clicking on a specific component (such as an offset duct), or record the target spatial coordinates (x, y, z) by selecting an area. The system has a built-in mapping algorithm from spatial coordinates to grid IDs to determine the spatial grid cell to which the currently selected location belongs. This algorithm achieves rapid positioning from spatial coordinates to the logical grid by traversing and judging the room grid bounding box, providing accurate spatial basis for subsequent task distribution by MR equipment. In practical applications, when construction personnel discover a problem in a certain area, they can obtain the coordinates by selecting the problem area, and the system uses this algorithm to quickly determine the corresponding grid ID, laying the foundation for accurate positioning of the problem area by MR equipment.

[0152] Step S1023: Convert spatial coordinates into spatial grid index information using preset transformation rules. This specifically includes the following steps:

[0153] Step 1: Based on the spatial coordinates obtained from the mobile device, convert the spatial coordinates of the mobile device into architectural spatial coordinates in the architectural coordinate system using a coordinate system transformation matrix;

[0154] Step 2: Construct a grid boundary list based on the bounding box information of the spatial grid. In the grid boundary list, determine the initial grid boundary corresponding to the building spatial coordinates using a spatial hash acceleration algorithm based on the three-level index of row, column and layer.

[0155] Step 3: Determine whether the initial mesh boundary satisfies the set of mesh boundary inequalities. If it does, take the initial mesh boundary as the target mesh boundary and determine that the target mesh boundary is located within the predetermined three-dimensional space.

[0156] Step 4: Based on the target mesh boundary and according to the index structure, determine the spatial mesh index information, which includes the mesh identifier and the mesh bounding box.

[0157] Specifically, for coordinate preprocessing, the spatial coordinates (x, y, z) obtained by the mobile device need to be converted to the building coordinate system consistent with the grid encoding (to avoid device coordinate system deviation). The formula is as follows:

[0158] (x', y', z') = T(x, y, z)

[0159] Where T(·) is the coordinate system transformation matrix (containing translation and rotation parameters, obtained through QR code positioning calibration), ensuring that the coordinates and grid boundary parameters are under the same reference.

[0160] Based on the known boundary parameters of all grids within the target room (from the spatial grid coding module), for each grid... G k Its boundary is:

[0161]

[0162] The specific matching steps are as follows:

[0163] a. Traverse the boundary data of all grids within the room and construct a list of grid boundaries. G 1 , G 2 ,..., G m};

[0164] b. For the target coordinates (x', y', z'), verify one by one whether they meet the grid requirements. Gk A system of boundary inequalities;

[0165] c. When a certain grid G k If all inequalities are satisfied, then the coordinate belongs to G k .

[0166] To avoid traversing the entire grid, a three-level index of "row-column-layer" is used for fast location:

[0167] The formula for calculating the row index is as follows:

[0168]

[0169] Where x_{\min,\text{room}} is the minimum coordinate of the room in the x-direction. S x The grid dimension is in the x-direction.

[0170] The formula for calculating column indexes is as follows:

[0171]

[0172] The formula for calculating the layer index (if vertical grid division exists) is as follows:

[0173]

[0174] By directly locating the candidate grid from (i, j, l) and then verifying whether the boundary matches, the time complexity can be reduced from O(m) to O(1).

[0175] After a successful match, an ID is generated according to the grid's three-level encoding rules:

[0176] Where n y n represents the total number of grid cells in the y-direction. z The total number of grid cells in the z direction (default 1, can be omitted if there is no vertical division).

[0177] For example:

[0178] If the coordinates (x', y', z') match the grid cell in the 2nd row (i=2), 3rd column (j=3) of room 5 (RoomID=5) on the 3rd floor (FloorID=3), and n_y=5, then:

[0179] GridID = "3-5-" + (2 * 5 + 3) = "3-5-13".

[0180] The spatial hashing acceleration algorithm in step two achieves index mapping with O(1) time complexity. After the spatial hashing acceleration algorithm, secondary hashing can be used to avoid hash collisions between different floor grids, thus improving mapping accuracy. Specifically:

[0181] Spatial hashing accelerated mapping: This refers to pre-converting coordinates into corresponding row, column, and level three indices, and then mapping the three-level indices to unique integers through a hash function to avoid hash collisions and achieve O(1) time complexity for querying.

[0182] Double hashing: To resolve potential hash collisions across different floor grids (e.g., grids on different floors may have the same i, j, l indices), a floor ID (or other unique spatial identifier) ​​is appended to the hash value to form a string of "hash value_floor ID" (e.g., 123456_3 represents a grid on the 3rd floor), ensuring that the hash value of grids across different floors is globally unique.

[0183] Spatial hashing and secondary hashing are progressive. The tertiary index (i, j, l) is the foundation, used to identify the position of the grid in space. Spatial hashing is an efficiency optimization based on the tertiary index. It converts the tertiary index into a single-value hash using a hash function, avoiding the O(n) complexity of traditional traversal queries. Secondary hashing is a supplement to hashing, solving the hash collision problem across spaces (such as different floors). Step S1024: Based on the spatial grid index information, generate loading instructions for the mixed reality device.

[0184] Optionally, the mobile device determines the spatial grid index information based on user operations, generates an MR loading command, and forwards it to the target MR device via the cloud. Upon receiving the command, the MR device automatically loads the corresponding grid model and focuses on the specified coordinate point, enabling rapid response and visual confirmation of on-site issues. For example, during construction, if workers discover a deviation in the installation of components in a certain grid area on their mobile device, they can generate an MR loading command through the above operations and send it to the MR device. After the MR device operator arrives on-site wearing the device, the device automatically loads the corresponding grid model and focuses on the problem coordinate point, allowing for intuitive viewing of the problem and subsequent processing, greatly improving the efficiency and accuracy of problem solving.

[0185] Steps S1021-S1024 utilize mobile devices to quickly identify unique identifiers and accurately load corresponding spatial models, significantly improving the efficiency of BIM model retrieval and loading, and avoiding the resource waste of loading the entire model. Spatial coordinates are obtained through natural gesture interaction, achieving an intuitive and efficient human-computer interaction experience and lowering the user threshold. Coordinates are converted into grid index information through preset transformation rules, ensuring the accuracy and consistency of spatial positioning. Finally, dynamic loading instructions for mixed reality devices are generated based on the grid index, achieving on-demand loading and real-time rendering, significantly reducing data transmission volume and improving response speed, solving the problem of low collaborative efficiency in traditional methods. This constructs an efficient, accurate, and user-friendly interactive system.

[0186] Step S103: The mixed reality device loads the spatial model corresponding to the spatial grid index information according to the loading instruction, and overlays the spatial model onto the real building scene using a positioning strategy. This specifically includes the following steps:

[0187] Step 1: Pre-construct a set of feature points for a real building scene and the mapping relationship between the feature point set and the corresponding world coordinates to reduce the amount of computation for real-time feature matching.

[0188] Step 2: Scan the QR code feature points using the visual sensor of the mixed reality device to extract the image coordinates of the spatial model; establish the correspondence between the image coordinates and the world coordinates based on the mapping relationship.

[0189] Step 3: Based on the correspondence, the rotation matrix and translation vector of the mixed reality device are obtained through the EPnP optimization algorithm. The optimization is achieved by minimizing the sum of the reprojection errors of each feature point. The reprojection error represents the sum of squares of the differences between the image coordinates and the actual projected coordinates.

[0190] Step 4: Verify that the reprojection error conforms to the preset distance. If it does, determine the initial positioning of the mixed reality device model and the real building scene.

[0191] Step 5: Based on the rotation matrix and translation vector of the initial positioning, collect the environmental point cloud in the real building scene through the depth sensor of the mixed reality device, and load the model point cloud associated with the spatial model.

[0192] Step 6: Through iterative optimization, minimize the sum of the transformed Euclidean distances between the environmental point cloud points and the model point cloud points to obtain the optimal rotation matrix and translation vector. The transformation is to apply the current rotation matrix and translation vector to map the model point cloud points to the environmental point cloud space.

[0193] Step 7: Stop iterating when the error change between two adjacent iterations is less than a preset condition, thus completing the overlay of the spatial model and the real building scene.

[0194] Optionally, after receiving the MR loading command from the mobile device, the MR device parses the target grid ID and focus coordinates contained therein. Based on the grid ID, it downloads only the BIM model file (glTF or Unity AssetBundle format) of the corresponding grid in the room from the cloud, rather than the entire floor model. The model loading strategy is based on a lightweight transmission protocol, supporting breakpoint resume and cache preloading mechanisms, significantly reducing data transmission volume and improving loading speed.

[0195] In the initial alignment stage: users scan the QR codes deployed in the building space using MR devices. The system uses the PnP (Perspective-n-Point) algorithm to solve the pose by combining the QR code image features with real-world coordinates, thereby achieving the initial spatial registration of the MR model with a positioning accuracy of ±3cm.

[0196] In the fine alignment stage: Based on the coarse positioning, the system further calls the ICP (Iterative Closest Point) point cloud registration algorithm according to the focus_coord focus coordinates sent by the mobile terminal. It uses the depth sensor built into the MR device to acquire environmental point cloud data and compares and corrects it with the target mesh model. Finally, the position of the MR model is adjusted to the millimeter level accuracy, and the positioning error is controlled within ±3mm.

[0197] Voice-driven archiving mechanism: The MR device integrates a voice recognition interface, supporting keyword wake-up (such as "Record View", "Take Photo", "Start Annotation", etc.). When the user issues the "Record View" command, the system automatically captures the current AR overlay state and generates structured archived data. The archived data is uploaded to the cloud database and bound to the corresponding BIM components, problem descriptions, and operator information to form a complete construction process traceability record.

[0198] Specifically, the core principle of the coarse localization stage (PnP algorithm) is to solve the device pose (rotation matrix R and translation vector t) by using the 3D world coordinates of the QR code and the 2D image coordinates captured by the MR device camera.

[0199] Perspective projection model: Let P be the world coordinates of a feature point on the QR code. w = (X, Y, Z, 1)^T, whose pixel coordinates on the MR camera image are p = (u, v, 1)^T, satisfying the following formula:

[0200]

[0201] Where: s is the scale factor, K is the camera intrinsic parameter matrix (known, obtained from equipment calibration), and t] is the 3×4 pose matrix (R is the 3×3 rotation matrix, and t is the 3×1 translation vector). Pose solution: The EfficientPerspective-n-Point (EPnP) optimization algorithm is used to solve for R and t by minimizing the reprojection error, as shown in the formula:

[0202]

[0203] Where π(.) is the perspective projection function, and n is the number of QR code feature points (usually 4 corner points).

[0204] The implementation steps are as follows:

[0205] a. The MR device scans the QR code and extracts the image coordinates p1 ~ p4 of the four corner points;

[0206] b. Read the world coordinates P bound to the QR code w 1 ~ P w 4;

[0207] c. Substitute the above formula to solve for the pose matrix [R | t], and complete the initial alignment of the MR model with the real space;

[0208] d. Positioning accuracy verification: Calculate the reprojection error and ensure that the error is ≤3cm.

[0209] The core principle of the precision alignment stage (ICP algorithm) is to minimize the spatial distance between the environmental point cloud collected by the MR device and the target mesh model point cloud through iterative optimization, thereby achieving millimeter-level calibration.

[0210] Let the environmental point cloud be Q = {q1, q2,..., qm}, and the model point cloud be P = {p1, p2,..., pm}. Find the optimal transformation (R, t) using the following formula:

[0211]

[0212] The iterative convergence condition is: the iteration stops when the error change between two consecutive iterations is less than a threshold (e.g., 0.1 mm), as expressed by the following formula:

[0213]

[0214] Implementation steps:

[0215] Based on the coarse localization results, the MR device acquires the environmental point cloud Q of the target grid area using a depth sensor;

[0216] Load the model point cloud P of the target mesh from the cloud (associated with the mobile device's focus coordinates).

[0217] Iterative process:

[0218] Step 1: Establish the point correspondence between Q and P through nearest neighbor search;

[0219] Step 2: Solve the above objective function to obtain the current optimal transformation (R, t);

[0220] Step 3: Apply the transformation to update the point cloud P and calculate the error E;

[0221] Step 4: If convergence is not achieved, return to Step 1 and repeat the iteration;

[0222] The final positioning error was controlled within ±3mm, achieving accurate alignment between the MR model and the real environment.

[0223] Step S103 significantly reduces the computational load of real-time feature matching for mixed reality devices and improves positioning efficiency by pre-constructing a set of scene feature points and their world coordinate mapping relationships. Based on QR code feature points and the EPnP optimization algorithm, high-precision initial pose estimation is achieved, ensuring rapid alignment between the virtual model and the real scene. A reprojection error verification mechanism effectively filters reliable positioning results and avoids erroneous overlay. Combining environmental point clouds collected by depth sensors with Iterative Closest Point (ICP) optimization further improves model alignment accuracy and ensures the stability of virtual-real fusion. Finally, dynamic convergence criteria optimize computational resource consumption while maintaining positioning accuracy. This solves the key problems of slow loading, inaccurate alignment, and unstable virtual-real fusion of BIM models in mixed reality, achieving efficient, accurate, and stable spatial model overlay.

[0224] In one embodiment, after step S103, where the mixed reality device loads the spatial model corresponding to the spatial grid index information according to the loading instruction, and overlays the spatial model onto the real building scene using a positioning strategy, the method further includes:

[0225] Step 1: Based on the priority message queue mechanism, the real-time transmission and processing of mixed reality device operation data is carried out through the stream processing engine. The data processing priority is determined by a dynamic weight allocation algorithm according to the type and urgency of the operation data.

[0226] Optionally, the core logic of the event-driven data flow algorithm (MR end response link) is to ensure that MR operation data is efficiently synchronized to all terminals through priority queues and stream processing rules.

[0227] Specifically, message queue priority sorting (Kafka phase) requires MR-side operation records (such as quality inspection results and annotation data) to be synchronized first. A dynamic weight allocation algorithm is used to assign message priorities, expressed by the following formula:

[0228] \text{Priority} = w1 \cdot \text{urgency} + w2 \cdot \text{data_size}

[0229] Where: \text{urgency} represents the urgency level (level 1-5, with level 5 being the default for records triggered by voice commands), and \text{data_size} represents the normalized value of the data size (0-1).

[0230] w1=0.7, w2=0.3 are weighting coefficients (prioritizing emergency operations).

[0231] Kafka sorts messages by their priority value, and high-priority messages (>3.5) skip the regular queue and go directly into Spark's fast processing pipeline.

[0232] Data cleaning for stream processing (Spark stage) involves real-time cleaning of the raw data (such as pose parameters and speech annotations) uploaded from the MapReduce client. Data integrity verification is expressed by the following formula:

[0233]

[0234] Coordinate standardization transformation (unifying to the architectural coordinate system) is expressed by the following formula:

[0235]

[0236] In the formula, coord MR : Mixed reality coordinates, offset Room Room coordinate offset, coord Standard Standard coordinates.

[0237] Step 2: Based on the spatial grid index, establish the association and binding between mixed reality device operation records and BIM components through a three-level mapping relationship, and determine the ownership relationship between components and grids according to the spatial inclusion relationship judgment criteria.

[0238] Optionally, the core principle of the data association mapping algorithm (cross-table association based on grid ID) is to achieve precise binding between MR operation records and BIM components through a three-level index mapping relationship.

[0239] The association formula for binding MR records to the grid index is as follows:

[0240] (One-to-one mapping, achieved through primary key and foreign key constraints)

[0241] Among them, MR_Record: Mixed Reality Operation Record Table, Grid: Spatial Grid Basic Information Table, grid_id: Grid Table Primary Key, Unique Identifier.

[0242] The association between the grid index and BIM components is as follows:

[0243] Grid table.element_ids = {e1,e2...ek}

[0244] Where ek represents the element_id of all BIM components within this grid, and element_ids is a field in the grid table used to store the IDs of all components contained within the current grid. The spatial containment relationship is used to calculate the affiliation between components and grids.

[0245]

[0246] Where, ei: the i-th component of Element, Grudk: the k-th grid of Grid, AABB(ei): AxisAlignedBoundingBox up, an axis-aligned bounding box that encloses ei with an axis-aligned cuboid, the spatial extent of which is determined by the extreme coordinates of the component's geometric vertices, and Grid_k.bounding_box: the bounding box of grid K. :equivalence, : subset, ∈: belong to.

[0247] Step 3: Based on the version control mechanism, incremental data push is triggered through the difference detection algorithm, and data conflicts between multiple terminals are resolved through the weighted fusion algorithm according to the timestamp arbitration rules.

[0248] Optionally, the core logic of the real-time synchronization triggering mechanism (WebSocket push) is to trigger a mobile UI update based on the difference detection between the data version number and the timestamp.

[0249] Version control formula: After data is uploaded to the MR client, a version identifier is automatically generated. The formula is as follows:

[0250] Version = BaseVersion + UpdateCount

[0251] BaseVersion is the initial version (1.0), and UpdateCount is the cumulative number of updates.

[0252] Push notification trigger conditions:

[0253] When the cloud detects that the MR client's data version is higher than the mobile client's cached version, a push notification is triggered, as shown in the formula below:

[0254]

[0255] Wherein, Version_cloud: cloud storage version number, Version_mobile: mobile cache version number, >: version number comparison operator, WenSocket: network communication protocol, Push: data push, Delte_data: incremental data.

[0256] The incremental data extraction rule only pushes changed fields to reduce data transmission volume. The formula is as follows:

[0257] ΔData=diff(MR_Recordnew,MR_Recordold)

[0258] Wherein, ΔData: Transmitted data, Diff: Difference calculation, MR_Recordnew: New version mixed reality operation record, MR_Recordold: Old version mixed reality operation record.

[0259] To ensure data consistency, the algorithm employs distributed locks and timestamp arbitration to avoid synchronization conflicts across multiple terminals. Lock contention is resolved by determining priority based on timestamps when both the MR (Mobile Module) and mobile devices simultaneously access the same grid data. The formula is as follows:

[0260] Winner=argmax(TimestampMR,Timestampmobile)

[0261] Among them, Winner: operation permission owner, Argmax: maximum value, TimestampMR: MR operation timestamp, Timestampmobile: mobile operation timestamp.

[0262] Conflict resolution: Conflicting data are fused using a weighted fusion method (MR data has a higher weight because it includes field measurement information), as shown in the following formula:

[0263] MergedData=0.7Data_MR+0.3Data_mobile

[0264] Wherein, MergedData: merged data, weight coefficient 0.7 / 0.3, Data_MR: MR data, Data_mobile: mobile data.

[0265] Data association rules: Operation records are associated with the BIM component library through grid ID: MR_Record table.grid_id → Grid index table → Revit component table.element_id.

[0266] Among them, MR_Record is the MR operation record table, grid index table is the grid index table, Revit component table is the Revit component library table, Grid_id is the unique identifier of the grid, and Element_id is the unique identifier of the component.

[0267] By employing a priority message queue and dynamic weight allocation mechanism, intelligent hierarchical processing of mixed reality device operation data is achieved, ensuring real-time response to high-urgency operations (such as quality inspection annotations) while optimizing system resource allocation. Based on a three-level mapping relationship using grid spatial indexing and boundary box detection, a precise association between operation records and BIM components is established, supporting cross-terminal data consistency maintenance and rapid retrieval. Version control and incremental data push ensure that only changed data fields are synchronized, significantly reducing network transmission load. Combining timestamp arbitration and weighted fusion algorithms effectively resolves conflicts arising from concurrent operations across multiple terminals, prioritizing the retention of MR terminal operation results containing on-site measured data. This constructs an efficient, reliable, and low-latency multi-terminal data collaboration system, solving the problems of data asynchrony, frequent conflicts, and slow response in traditional BIM mixed reality applications.

[0268] In summary, the BIM model mobile terminal and mixed reality collaboration method provided in this application embodiment achieves structured organization of building space data by spatially segmenting and meshing the BIM model through a pre-created index structure, thereby improving the retrieval and processing efficiency of large-scale BIM models. The rapid identification and loading mechanism based on unique identifiers enables the mobile terminal to accurately locate and instantly retrieve the target spatial model, effectively reducing data transmission volume. Gesture operation and spatial coordinate transformation technology enable natural and intuitive human-computer interaction, significantly improving operational convenience. The use of mesh index information to generate mixed reality loading instructions ensures the accuracy and real-time performance of model loading. Finally, a high-precision virtual-real spatial positioning strategy achieves seamless overlay of the BIM model and the real building scene, solving the problem of low collaboration efficiency in traditional methods. This constructs an efficient, accurate, and easy-to-use BIM model mobile terminal and mixed reality collaboration method.

[0269] Secondly, embodiments of this application provide a collaborative system between a mobile BIM model terminal and mixed reality. Figure 4 This is a structural block diagram of a collaborative system between a BIM model, a mobile terminal, and mixed reality. For example... Figure 4 As shown, the device includes: an identification code module 410, a loading instruction module 420, and an overlay module 430, wherein:

[0270] The identification code module 410 is used to perform spatial segmentation and spatial grid division processing on the BIM model according to the pre-created index structure, obtain index information including multiple spatial models and multiple spatial grids, and construct a unique identification code for the spatial model based on the index information.

[0271] The loading instruction module 420 is used to identify the identification code through the mobile terminal, load the spatial model corresponding to the identification code, obtain the spatial coordinates in the spatial model according to the gesture operation, convert the spatial coordinates into spatial grid index information through the preset conversion rules, and generate the loading instruction for the mixed reality device.

[0272] The overlay module 430 is used by the mixed reality device to load the spatial model corresponding to the spatial grid index information according to the loading instruction, and to overlay the spatial model onto the real building scene through the positioning strategy.

[0273] In summary, this application provides a collaborative system between a BIM model mobile terminal and mixed reality. By spatially segmenting and meshing the BIM model through a pre-created index structure, it achieves structured organization of building space data, improving the retrieval and processing efficiency of large-scale BIM models. A rapid identification and loading mechanism based on unique identifiers enables the mobile terminal to accurately locate and instantly retrieve the target spatial model, effectively reducing data transmission volume. Gesture operation and spatial coordinate transformation technology enable natural and intuitive human-computer interaction, significantly improving operational convenience. The use of mesh index information to generate mixed reality loading instructions ensures the accuracy and real-time performance of model loading. Finally, a high-precision virtual-real spatial positioning strategy achieves seamless overlay of the BIM model and the real building scene, solving the problem of low collaborative efficiency in traditional methods. This constructs an efficient, accurate, and easy-to-use collaborative method between a BIM model mobile terminal and mixed reality.

[0274] It should be noted that the BIM model mobile terminal and mixed reality collaboration provided in this embodiment are used to implement the above-described implementation methods, and details already described will not be repeated. As used above, the terms "module," "unit," "subunit," etc., can refer to combinations of software and / or hardware that perform predetermined functions. Although the apparatus described in the above embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0275] Thirdly, embodiments of this application provide an electronic device, Figure 5 This is a block diagram illustrating an electronic device according to an exemplary embodiment. (e.g.) Figure 5 As shown, the electronic device may include a processor 51 and a memory 52 storing computer program instructions.

[0276] Specifically, the processor 51 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0277] The memory 52 may include a large-capacity memory for data or instructions. For example, and not limitingly, the memory 52 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 52 may include removable or non-removable (or fixed) media. Where appropriate, the memory 52 may be internal or external to a data processing device. In a particular embodiment, the memory 52 is non-volatile memory. In a particular embodiment, the memory 52 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), Extended Data Out Dynamic Random-Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0278] The memory 52 can be used to store or cache various data files that need to be processed and / or used for communication, as well as possible computer program instructions executed by the processor 51.

[0279] The processor 51 reads and executes computer program instructions stored in the memory 52 to implement any of the collaborative methods between the BIM model mobile terminal and mixed reality in the above embodiments.

[0280] In one embodiment, a collaborative device for BIM model mobile terminal and mixed reality may further include a communication interface 53 and a bus 50. Wherein, as Figure 5 As shown, the processor 51, memory 52, and communication interface 53 are connected through bus 50 and complete communication with each other.

[0281] The communication interface 53 is used to enable communication between the various modules, devices, units, and / or equipment in the embodiments of this application. The communication port 53 can also enable data communication with other components such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations.

[0282] Bus 50 includes hardware, software, or both, that couples components of a BIM model mobile device to each other with components of a mixed reality collaborative device. Bus 50 includes, but is not limited to, at least one of the following: Data Bus, Address Bus, Control Bus, Expansion Bus, and Local Bus. For example, and not as a limitation, bus 50 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 50 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0283] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements a collaborative method between a BIM model mobile terminal and mixed reality as provided in the first aspect.

[0284] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0285] In a possible implementation, the invention can also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform steps implementing the collaborative method of BIM model mobile terminal and mixed reality provided in the first aspect.

[0286] The program code for executing the present invention can be written in any combination of one or more programming languages. The program code can be executed entirely on the user device, partially on the user device, as a standalone software package, partially on the user device and partially on a remote device, or entirely on a remote device.

[0287] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0288] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for BIM model mobile terminal and mixed reality collaboration, characterized in that, The method comprises: According to the pre-created index structure, the BIM model is subjected to spatial segmentation and spatial grid division processing, index information including a plurality of spatial models and a plurality of spatial grids is obtained, and a spatial model unique identification code is constructed based on the index information; The identification code is identified through a mobile terminal, the spatial model corresponding to the identification code is loaded, the spatial coordinates in the spatial model are obtained according to a gesture operation, the spatial coordinates are converted into spatial grid index information through a preset conversion rule, and a loading instruction of a mixed reality device is generated; The mixed reality device loads the spatial model corresponding to the spatial grid index information according to the loading instruction, and superimposes the spatial model onto a real building scene through a positioning strategy; Wherein, converting the spatial coordinates into spatial grid index information through a preset conversion rule comprises: According to the spatial coordinates obtained by the mobile terminal, the spatial coordinates of the mobile terminal are converted into building spatial coordinates in a building coordinate system through a coordinate system conversion matrix; a grid boundary list is constructed according to the boundary box information of the spatial grid, and the initial grid boundary corresponding to the building spatial coordinates is determined through a spatial hash acceleration algorithm according to the three-level index of row, column and layer in the grid boundary list; it is judged whether the initial grid boundary satisfies the grid boundary inequality group, and in the case of satisfaction, the initial grid boundary is taken as the target grid boundary, and it is determined that the target grid boundary is located in the predetermined three-dimensional space range; based on the target grid boundary, the spatial grid index information is determined according to the index structure, wherein the spatial grid index information includes grid identification and grid boundary box. 2.The BIM model mobile terminal and mixed reality collaboration method of claim 1, wherein, According to the pre-created index structure, the BIM model is subjected to spatial segmentation and spatial grid division processing, index information including a plurality of spatial models and a plurality of spatial grids is obtained, and a spatial model unique identification code is constructed based on the index information, comprising: The BIM model is segmented according to the spatial index in the index structure to generate a plurality of spatial models, and the index information of the spatial model is obtained; Based on the actual size of the spatial model and the preset grid density, the spatial grid is divided, and the spatial grid index information and the spatial grid boundary box are determined based on the divided spatial grid; Based on the index information of the spatial model and the spatial grid index information, a spatial model unique identification code is generated through a hash algorithm. 3.The BIM model mobile terminal and mixed reality collaboration method of claim 2, wherein, The BIM model is segmented according to the space in the index structure to generate a plurality of spatial models, and the index information of the spatial model is obtained, comprising: The model geometry vertex coordinate data of the index structure is obtained as the space index, and based on the geometry vertex coordinate data, the three-dimensional bounding box of the spatial model is determined through coordinate extreme value calculation; Based on the three-dimensional bounding box, the building components and the cross-space connection points located inside the spatial model are retained through a geometry screening algorithm to generate a plurality of initial spatial models, wherein in a plurality of the spatial models, the cross-space components are anchored through the connection points to determine the topological integrity of the cross-space components; The initial space model is converted into a space model in GLB format through a three-dimensional graphics compression conversion algorithm; According to the index structure, index information of the space model is generated through a multi-level digital coding rule.

4. The BIM model mobile terminal and mixed reality collaboration method of claim 2, wherein, The actual size of the space model and the preset grid density are used to divide a space grid, and based on the divided space grid, space grid index information and a space grid bounding box are determined, including: According to a preset grid division density, a size parameter of a standard grid is set; According to geometric boundary data of the space model, an actual physical size of the space model is obtained through a three-dimensional size measurement algorithm; Based on a proportional relationship between the actual physical size and the size of the standard grid, the number of space grid divisions in the space model is calculated through a grid covering algorithm, and grid division is performed based on the number of space grid divisions, to obtain a plurality of space grids; Through a multi-level coding rule, each space grid index information is generated, and the space grid index information includes space position information; Based on the space position information, a grid bounding box of each grid is determined through a coordinate allocation algorithm.

5. The BIM model mobile terminal and mixed reality collaboration method of claim 1, wherein, In the grid boundary list, the grid boundary corresponding to the building space coordinate is determined through a three-level index of rows, columns and layers, including: According to the x-axis coordinate value of the building space coordinate, the row index number is calculated and obtained through the ratio relationship between the minimum coordinate of the building space x direction and the size of the grid x direction; According to the y-axis coordinate value of the building space coordinate, the column index number is calculated and obtained through the ratio relationship between the minimum coordinate of the building space y direction and the size of the grid y direction; According to the z-axis coordinate value of the building space coordinate, the layer index number is calculated and obtained through the ratio relationship between the minimum coordinate of the building space z direction and the size of the grid z direction; Based on the row index number, the column index number and the layer index number, the grid boundary corresponding to the building space coordinate is determined.

6. The BIM model mobile terminal and mixed reality collaboration method of claim 1, wherein, The mixed reality device includes a visual sensor, and the space model is superimposed on the real building scene through a positioning strategy, including: A set of feature points of the real building scene and a mapping relationship between the feature points and world coordinates are constructed in advance; The image coordinates of the space model are extracted by scanning the two-dimensional code feature points through the visual sensor of the mixed reality device, and the corresponding relationship between the image coordinates and the world coordinates is established based on the mapping relationship; Based on the corresponding relationship, the rotation matrix and the translation vector of the mixed reality device are obtained through an optimization algorithm, wherein the optimization is performed by minimizing the sum of the re-projection errors of each feature point, and the re-projection error represents the sum of squares of differences between the image coordinates and the actual projection coordinates; It is verified that the re-projection error meets the preset distance, and in the case of meeting, the preliminary positioning of the mixed reality device model and the real building scene is determined; Based on the rotation matrix and the translation vector of the preliminary positioning, the environment point cloud in the real building scene is collected through the depth sensor of the mixed reality device, and the model point cloud associated with the space model is loaded. The optimal rotation matrix and translation vector are obtained by minimizing the sum of the Euclidean distances between the transformed environment point cloud points and the model point cloud points through an iterative optimization process, wherein the transformation is mapping the model point cloud points to the environment point cloud space by applying the current rotation matrix and translation vector; The superposition of the spatial model and the real building scene is completed by stopping iteration when the error change amount of adjacent two iterations is less than a preset condition.

7. The BIM model mobile terminal and mixed reality collaboration method of claim 1, wherein, Before performing spatial segmentation and spatial grid division processing on the BIM model according to the pre-created index structure, the method further comprises: The BIM model is parsed through three-dimensional boundaries, and geometric boundary information of the spatial model is extracted from the BIM model, the geometric boundary information including vertex coordinates and boundary lengths; According to the positional relationship between the components, the spatial correlation between the components is analyzed, and topological relationship data is generated, wherein the positional relationship includes inclusion and adjacent relationship; Based on the geometric boundary information and the topological relationship, an index structure of the BIM model is generated through the hierarchical relationship of the BIM model, the index structure including floor index, space index and grid index, wherein the floor index includes floor identification, the space index includes the floor identification and space identification, and the grid index includes the floor identification, space identification and grid identification.

8. The BIM model mobile terminal and mixed reality collaboration method of claim 1, wherein, After superimposing the spatial model on the real building scene through the positioning strategy, the method further comprises: According to the priority message queue mechanism, real-time transmission and processing of mixed reality device operation data are performed through a stream processing engine, and data processing priority is determined through a dynamic weight distribution algorithm according to operation data types and urgency; According to the spatial grid index, an association binding between the mixed reality device operation record and the BIM component is established through a three-level mapping relationship, and the ownership relationship between the component and the grid is determined according to the spatial inclusion relationship determination criterion; Based on the version control mechanism, incremental data pushing is triggered through a difference detection algorithm, and multi-terminal data conflicts are solved through a weighted fusion algorithm according to a timestamp arbitration rule. 9.A BIM model mobile terminal and mixed reality collaborative system, characterized in that, The system comprises an identification code module, a loading instruction module and a superposition module, wherein: The identification code module is configured to perform spatial segmentation and spatial grid division processing on the BIM model according to the pre-created index structure, obtain index information including a plurality of spatial models and a plurality of spatial grids, and construct a unique spatial model identification code based on the index information. The loading instruction module is configured to identify the identification code through the mobile terminal, load the space model corresponding to the identification code, acquire space coordinates in the space model according to a gesture operation, convert the space coordinates into space grid index information through a preset conversion rule, and generate a loading instruction of a mixed reality device; wherein converting the space coordinates into space grid index information through the preset conversion rule comprises: converting the space coordinates acquired by the mobile terminal into building space coordinates in a building coordinate system through a coordinate system conversion matrix; constructing a grid boundary list according to the boundary box information of the space grid, determining an initial grid boundary corresponding to the building space coordinates through a space hash acceleration algorithm in the grid boundary list according to a three-level index of rows, columns and layers; judging whether the initial grid boundary satisfies a grid boundary inequality set, and in the case of satisfaction, taking the initial grid boundary as a target grid boundary and determining that the target grid boundary is located within a predetermined three-dimensional space range; and determining space grid index information according to the index structure based on the target grid boundary, wherein the space grid index information comprises a grid identifier and a grid boundary box. The superimposition module is configured to load, according to the loading instruction, a space model corresponding to the space grid index information by the mixed reality device, and superimpose the space model onto a real building scene through a positioning strategy.

10. An electronic device, comprising: The computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the BIM model mobile terminal and mixed reality collaborative method of any one of claims 1 to 8.

11. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the BIM model mobile terminal and mixed reality collaborative method of any one of claims 1 to 8.

Images