BIM model mobile terminal and mixed reality cooperation method and system, and medium

Abstract

The invention relates to a BIM model mobile terminal and mixed reality cooperation method and system and a medium, and the method comprises the steps: carrying out the space segmentation and space grid division processing of a BIM model according to a pre-created index structure, obtaining the index information of a plurality of space models and a plurality of space grids, and obtaining a plurality of space models and a plurality of space grids based on the index information; and constructing a unique identification code of the spatial model. Identifying the identification code through the mobile terminal, loading the spatial model corresponding to the identification code, obtaining spatial coordinates in the spatial model according to the gesture operation, converting the spatial coordinates into spatial grid index information through a preset conversion rule, and generating a loading instruction of the mixed reality equipment. And the mixed reality equipment loads a space model corresponding to the space grid index information according to the loading instruction, and superposes the space model to a real building scene through a positioning strategy. The problem of low cooperation efficiency of a BIM model mobile terminal and mixed reality in related technologies is solved.

Description

Technical Field

[0001] The present application relates to the technical field of building information modeling, and in particular to a method, system and medium for collaboration between a mobile BIM model terminal and mixed reality. Background Art

[0002] The collaborative application of Building Information Modeling (BIM) and Mixed Reality (MR) technology demonstrates tremendous potential in building construction, operations, and management. BIM provides a wealth of digital building information, while MR technology intuitively overlays this information onto the real environment, assisting on-site operations.

[0003] However, the existing BIM model and MR collaboration method consumes too much resources when loading the BIM model on the MR device, resulting in low efficiency in superimposing the spatial model onto the real building scene.

[0004] Therefore, there is an urgent need to provide an efficient BIM model and mixed reality collaboration method to solve the above problems. Summary of the Invention

[0005] The embodiments of the present application provide an efficient method for collaboration between BIM models and mixed reality, so as to at least solve the problem of low efficiency of collaboration between BIM model mobile terminals and mixed reality in related technologies.

[0006] In a first aspect, an embodiment of the present application provides a method for collaboration between a BIM model mobile terminal and mixed reality, the method comprising: Performing spatial segmentation and spatial grid division processing on the BIM model according to a pre-created index structure to obtain index information including multiple spatial models and multiple spatial grids, and constructing a unique identification code for the spatial model based on the index information; Identifying the identification code through a mobile terminal, loading the spatial model corresponding to the identification code, obtaining spatial coordinates in the spatial model according to a gesture operation, converting the spatial coordinates into spatial grid index information according to a preset conversion rule, and generating a loading instruction for a mixed reality device; The mixed reality device loads the spatial model corresponding to the spatial grid index information according to the loading instruction, and superimposes the spatial model on the real building scene through a positioning strategy.

[0007] In one embodiment, the BIM model is spatially segmented and spatially meshed according to a pre-created index structure to obtain index information including multiple spatial models and multiple spatial meshes. Based on the index information, a unique identification code for the spatial model is constructed, including: Segmenting the BIM model according to the spatial index in the index structure to generate multiple spatial models, and obtaining index information of the spatial models; Dividing a spatial grid based on an actual size of the spatial model and a preset grid density, and determining spatial grid index information and a spatial grid bounding box based on the divided spatial grid; Based on the index information of the spatial model and the spatial grid index information, a spatial model unique identification code is generated by a hash algorithm.

[0008] In one embodiment, segmenting the BIM model according to the spaces in the index structure to generate multiple space models, and obtaining index information of the space models includes: Acquire model geometric vertex coordinate data whose index structure is a spatial index, and determine a three-dimensional bounding box of the spatial model by calculating coordinate extreme values ​​based on the geometric vertex coordinate data; Based on the three-dimensional bounding box, a geometric screening algorithm is used to retain the building components and cross-space connection points located inside the spatial model to generate multiple initial spatial models, wherein, in the multiple spatial models, cross-space components are anchored by connection points to determine the topological integrity of the cross-space components; Converting the initial spatial model into a spatial model in GLB format using a three-dimensional graphics compression conversion algorithm; According to the index structure, the index information of the spatial model is generated through multi-level digital coding rules.

[0009] In one embodiment, dividing the spatial grid based on the actual size of the spatial model and the preset grid density, and determining the spatial grid index information and the spatial grid bounding box based on the divided spatial grid, includes: Set the size parameters of the standard grid according to the preset grid division density; According to the geometric boundary data of the spatial model, the actual physical size of the spatial model is obtained by a three-dimensional size measurement algorithm; Based on the proportional relationship between the actual physical size and the standard grid size, the number of spatial grid divisions in the spatial model is calculated using a grid coverage algorithm, and grid division is performed based on the number of spatial grid divisions to obtain a plurality of spatial grids; Generate spatial grid index information of each item through multi-level coding rules, wherein the spatial grid index information includes spatial position information; Based on the spatial position information, a grid bounding box of each grid is determined by a coordinate assignment algorithm.

[0010] In one embodiment, converting the spatial coordinates into spatial grid index information using a preset conversion rule includes: According to the spatial coordinates obtained by the mobile terminal, the spatial coordinates of the mobile terminal are converted into building spatial coordinates in a building coordinate system through a coordinate system conversion matrix; Constructing a grid boundary list based on the bounding box information of the spatial grid, and determining the initial grid boundary corresponding to the building space coordinates using a spatial hash acceleration algorithm based on the three-level index of row, column, and layer in the grid boundary list; determining whether the initial grid boundary satisfies a grid boundary inequality group, and if so, using the initial grid boundary as a target grid boundary, and determining that the target grid boundary is within a predetermined three-dimensional space range; Based on the target grid boundary and according to the index structure, spatial grid index information is determined, wherein the spatial grid index information includes a grid identifier and a grid bounding box.

[0011] In one embodiment, in the grid boundary list, determining the grid boundary corresponding to the building space coordinates by a three-level index of row, column, and layer includes: According to the x-axis coordinate value of the building space coordinate, the row index number is calculated by the ratio of the minimum coordinate in the x-direction of the building space to the x-direction size of the grid; According to the y-axis coordinate value of the building space coordinate, the column index number is calculated by the ratio of the minimum coordinate in the y-direction of the building space to the y-direction size of the grid; According to the z-axis coordinate value of the building space coordinate, the layer index number is calculated by the ratio of the minimum coordinate in the z direction of the building space to the z-direction size of the grid; A grid boundary corresponding to the building space coordinate is determined based on the row index number, the column index number and the layer index number.

[0012] In one embodiment, the mixed reality device includes a visual sensor, and superimposing the spatial model onto the real building scene using a positioning strategy includes: Pre-constructing a real building scene feature point set and a mapping relationship between the feature point set and the world coordinates; Scanning the feature points of the QR code with the visual sensor of the mixed reality device to extract the image coordinates of the spatial model; and establishing a corresponding relationship between the image coordinates and the world coordinates based on the mapping relationship; Based on the correspondence, obtaining a rotation matrix and a translation vector of the mixed reality device through an optimization algorithm, wherein the optimization is performed by minimizing a sum of reprojection errors of each feature point, wherein the reprojection error represents the sum of squares of differences between image coordinates and actual projected coordinates; Verifying that the reprojection error meets a preset distance, and if so, determining a preliminary positioning of the mixed reality device model and the real building scene; Based on the rotation matrix and the translation vector obtained by the preliminary positioning, collecting an environment point cloud in a real building scene through a depth sensor of a mixed reality device, and loading a model point cloud associated with a spatial model; Through an iterative optimization process, the sum of the transformed Euclidean distances between the environment point cloud points and the model point cloud points is minimized to obtain the optimal rotation matrix and translation vector, where the transformation is to map the model point cloud points to the environment point cloud space using the current rotation matrix and translation vector; The superposition of the spatial model and the real building scene is completed by stopping the iteration when the error variation between two adjacent iterations is less than a preset condition.

[0013] In one embodiment, before performing space segmentation and space meshing processing on the BIM model according to the pre-created index structure, the method further includes: parsing the BIM model through the three-dimensional boundary, and extracting geometric boundary information of the spatial model from the BIM model, the geometric boundary information including vertex coordinates and boundary length; Analyzing the spatial association between components according to the positional relationship between the components to generate topological relationship data, wherein the positional relationship includes containment and adjacency relationships; Based on the geometric boundary information and topological relationship, an index structure of the BIM model is generated through the hierarchical relationship of the BIM model, wherein the index structure includes indexes of floors, spaces, and grids, wherein the floor index includes a floor identifier, the space index includes the floor identifier and the space identifier, and the grid index includes the floor identifier, the space identifier, and the grid identifier.

[0014] In one embodiment, after superimposing the spatial model onto the real building scene using a positioning strategy, the method further includes: Based on the priority message queue mechanism, the stream processing engine performs real-time transmission and processing of mixed reality device operation data. The dynamic weight allocation algorithm determines the data processing priority based on the operation data type and urgency. Based on the grid space index, the association and binding between the mixed reality equipment operation record and the BIM component is established through the three-level mapping relationship. According to the spatial inclusion relationship judgment criteria, the attribution relationship between the component and the grid is determined; Based on the version control mechanism, incremental data push is triggered by the difference detection algorithm, and multi-terminal data conflicts are resolved through the weighted fusion algorithm according to the timestamp arbitration rules.

[0015] In a second aspect, an embodiment of the present application provides a collaboration between a BIM model mobile terminal and mixed reality, wherein the system includes: an identification code module, a loading instruction module, and an overlay module, wherein: The identification code module is used to perform spatial segmentation and spatial grid division processing on the BIM model according to a pre-created index structure, obtain index information including multiple spatial models and multiple spatial grids, and construct a unique identification code for the spatial model based on the index information; The loading instruction module is configured to identify the identification code through a mobile terminal, load the spatial model corresponding to the identification code, obtain spatial coordinates in the spatial model according to a gesture operation, convert the spatial coordinates into spatial grid index information according to a preset conversion rule, and generate a loading instruction for the mixed reality device; The overlay module is used for the mixed reality device to load the spatial model corresponding to the spatial grid index information according to the loading instruction, and overlay the spatial model onto the real building scene through a positioning strategy.

[0016] In a third aspect, an embodiment of the present application provides a computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, a method for collaborating between a mobile BIM model terminal and mixed reality as described in the first aspect above is implemented.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for collaboration between a BIM model mobile terminal and mixed reality as described in the first aspect above.

[0018] The embodiments of the present application provide a method, system, and medium for collaboration between a mobile BIM model terminal and mixed reality, which have at least the following technical effects.

[0019] By using a pre-created index structure to spatially segment and grid the BIM model, the structured organization of building space data is achieved, improving the efficiency of retrieval and processing of large-scale BIM models. The rapid recognition and loading mechanism based on the unique identification code enables the mobile terminal to accurately locate and instantly retrieve the target space model, effectively reducing the amount of data transmission. Through gesture operation and spatial coordinate conversion technology, natural and intuitive human-computer interaction is achieved, greatly improving the convenience of operation; the use of grid index information to generate mixed reality loading instructions ensures the accuracy and real-time performance of model loading; finally, through a high-precision virtual and real space positioning strategy, the seamless superposition of BIM models and real building scenes is achieved, solving the problem of low collaboration efficiency in traditional methods. An efficient, accurate and easy-to-use BIM model mobile terminal and mixed reality collaboration method is constructed.

[0020] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 It is a flow chart of a collaborative method between a mobile BIM model and mixed reality; Figure 2 is a flowchart of step S101 according to an exemplary embodiment; Figure 3 is a flowchart of step S102 according to an exemplary embodiment; Figure 4 This is a system structure diagram of a collaborative system between a BIM model mobile terminal and mixed reality according to an exemplary embodiment; Figure 5 It is a structural block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is described and illustrated below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely used to explain this application and are not intended to limit this application. Based on the embodiments provided in this application, all other embodiments obtained by those of ordinary skill in the art without making any creative efforts are within the scope of protection of this application.

[0023] Obviously, the drawings described below are merely examples or embodiments of the present application. Those skilled in the art can, without inventive effort, apply the present application to other similar scenarios based on these drawings. Furthermore, it is also understood that, although the effort involved in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, changes in design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as an insufficiency of the content disclosed in this application.

[0024] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments unless there is a conflict.

[0025] Unless otherwise defined, technical or scientific terms used herein shall have the ordinary meaning as understood by persons of ordinary skill in the art to which this application belongs. The terms "a," "an," "an," "the," and similar expressions used herein do not denote quantitative limitations and may refer to either the singular or the plural. The terms "comprise," "include," "have," and any variations thereof, used herein, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or modules (units) is not limited to the listed steps or units but may also include steps or units not listed, or may include other steps or units inherent to the process, method, product, or apparatus. The terms "connected," "connected," "coupled," and similar expressions used herein are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. As used herein, "plurality" means two or more. "And / or" describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" may mean: A exists alone; A and B exist simultaneously; or B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0026] It should be understood that the terms herein may be technical means for implementing a part of the present invention or other summary technical terms. For example, the terms may include: Spatial model: refers to the independent three-dimensional model unit divided from the BIM model according to the spatial hierarchy (such as room, area), which contains geometric data, attribute information and spatial topology relationships, and is used for local loading and interaction in mixed reality.

[0027] Spatial grid: A regular three-dimensional grid unit formed by dividing the spatial model into preset sizes. Each grid has a unique code and boundary definition, which is used for rapid positioning of spatial data.

[0028] Spatial coordinates: data describing the location of a target point in three-dimensional space.

[0029] Unique identification code: A string identifier generated based on a hash algorithm or hierarchical encoding, with the following structure: Project ID-Floor ID-Space ID-Grid ID, used to uniquely identify a space model or grid unit.

[0030] Grid boundary list: A data structure that stores the boundary parameters of all grid cells and records the minimum or maximum X / Y / Z coordinates of each grid.

[0031] 3D Bounding Box (AABB): An axis-aligned cubic bounding box defined by min / max coordinate values.

[0032] World coordinates: The global coordinate system defined by the BIM model is mapped to the real scene through QR code or feature point calibration.

[0033] In the first aspect, the embodiments of the present application provide a method for collaboration between a BIM model mobile terminal and mixed reality. Figure 1 It is a flow chart of the collaborative method between BIM model mobile terminal and mixed reality, such as Figure 1 As shown, the method includes: Step S101: perform spatial segmentation and spatial grid division processing on the BIM model according to a pre-created index structure, obtain index information including multiple spatial models and multiple spatial grids, and construct a unique identification code of the spatial model based on the index information.

[0034] Step S102: Identify the identification code through the mobile terminal, load the spatial model corresponding to the identification code, obtain the spatial coordinates in the spatial model according to the gesture operation, convert the spatial coordinates into spatial grid index information through the preset conversion rules, and generate a loading instruction for the mixed reality device.

[0035] Step S103: The mixed reality device loads the spatial model corresponding to the spatial grid index information according to the loading instruction, and superimposes the spatial model onto the real building scene through the positioning strategy.

[0036] In summary, the embodiment of the present application provides a method for collaboration between a mobile terminal of a BIM model and mixed reality. By performing spatial segmentation and grid division on the BIM model through a pre-created index structure, the structured organization of building space data is realized, and the retrieval and processing efficiency of large-scale BIM models is improved. The rapid recognition and loading mechanism based on the unique identification code enables the mobile terminal to accurately locate and instantly retrieve the target space model, effectively reducing the amount of data transmission. Through gesture operation and spatial coordinate conversion technology, natural and intuitive human-computer interaction is achieved, which greatly improves the convenience of operation; grid index information is used to generate mixed reality loading instructions to ensure the accuracy and real-time performance of model loading; finally, through a high-precision virtual and real space positioning strategy, the seamless superposition of the BIM model and the real building scene is achieved, solving the problem of low collaboration efficiency in traditional methods. An efficient, accurate and easy-to-use method for collaboration between a mobile terminal of a BIM model and mixed reality is constructed.

[0037] Figure 2 is a flowchart of step S101 according to an exemplary embodiment. Figure 2As shown, step S101, based on the pre-created index structure, performs space segmentation and space grid division on the BIM model, obtains index information including multiple space models and multiple space grids, and constructs a unique identification code for the space model based on the index information. Specifically, the following steps are included: Step S1011: Segment the BIM model according to the spatial index in the index structure to generate multiple spatial models, and obtain index information of the spatial models. This specifically includes the following steps: Step 1: Obtain model geometric vertex coordinate data whose index structure is a spatial index, and determine the three-dimensional bounding box of the spatial model by calculating the extreme values ​​of the coordinates based on the geometric vertex coordinate data.

[0038] Step 2: Based on the three-dimensional bounding box, a geometric screening algorithm is used to retain the building components and cross-space connection points located inside the spatial model to generate multiple initial spatial models. Among them, in multiple spatial models, cross-space components are anchored by connection points to ensure the topological integrity of the cross-space components.

[0039] Step 3: Convert the initial spatial model into a GLB format spatial model through a three-dimensional graphics compression conversion algorithm.

[0040] Step 4: Generate index information of the spatial model based on the index structure through multi-level digital coding rules.

[0041] Optionally, the entire BIM model is cropped based on the room's bounding box, preserving components within each room and cross-space connection points. For cross-space components such as pipes through walls, a specific algorithm is used to maintain their topological integrity, ensuring that the model still reflects actual physical connections after segmentation. The cropped model is converted to glb format to ensure lightweight and compatible models. Bounding box definition: The room bounding box is defined by the minimum or maximum coordinates, as shown in the following formula:

[0042] Where, x min Represents the minimum position of the room bounding box in the x-axis, x max Represents the maximum position of the room bounding box in the x-axis; y min Represents the minimum position of the room bounding box on the y axis, y max Represents the maximum position of the room bounding box on the y axis; z min Represents the minimum position of the room's bounding box in the z-axis, z max Represents the maximum position of the room's bounding box in the z-axis.

[0043] For cross-space components such as pipelines passing through walls, the connection point anchoring method is used to identify the intersection points where the pipelines cross the room boundaries ( C 1 , C 2 , ..., C i ), retaining the geometric parameters of the pipeline segments between the intersections (length, diameter, slope), and associating the pipeline models of adjacent rooms through the intersection coordinates. The formula is:

[0044] Among them, RoomA is room A, RoomB is room B, i is the number of specific intersection points. The connection point anchoring method is an algorithm used to maintain the topological integrity of cross-space components (such as pipes and beams passing through walls) when segmenting the BIM model. This method identifies the connection points where components cross spatial boundaries and preserves their geometry and attribute information, ensuring that the actual physical connection relationships are still accurately reflected after the model is segmented.

[0045] Step S1012: Divide the spatial grid based on the actual size of the spatial model and the preset grid density, and determine the spatial grid index information and the spatial grid bounding box based on the divided spatial grid. This specifically includes the following steps: Step 1: Set the size parameters of the standard grid according to the preset grid division density; Step 2: Based on the geometric boundary data of the spatial model, the actual physical size of the spatial model is obtained through a three-dimensional size measurement algorithm; Step 3: Based on the proportional relationship between the actual physical size and the standard grid size, the number of spatial grid divisions in the spatial model is calculated using a grid coverage algorithm, and grid division is performed based on the number of spatial grid divisions to obtain multiple spatial grids; Step 4: Generate index information of each spatial grid through multi-level coding rules, where the spatial grid index information includes spatial position information; Step 5: Based on the spatial position information, determine the grid bounding box of each grid through a coordinate assignment algorithm.

[0046] Optionally, the room is subdivided into multiple spatial grids, each with independent spatial grid index information and spatial grid bounding box. The generate_space_grid() function is predefined, and the standard grid size is defined using configurable parameters (such as 5m×5m), and the number of grids is dynamically adjusted according to the actual size of the room. Dynamic calculation of grid quantity: Assume the actual size of the room is L x (length),L y (width), the default grid size is S x * S y (For example, 5m×5m), the number of grids is:

[0047] (Round up function to ensure complete coverage of room boundaries) Grid identifier encoding rules: Use a three-level encoding structure:

[0048] Where i [1, n_x] is the row index, j [1, n_y] is the column index, GridID is the spatial grid index information, FloorID is the floor identifier, and RoomID is the room identifier. Grid identifier encoding rules: The boundary coordinates of the (i,j)th grid:

[0049] Furthermore, the generate_space_grid() function is used to automatically divide the room into a regular space grid. Its processing flow includes the following steps: 1. Space size calculation: Based on the room's geometric bounding box, extract its actual physical dimensions (length, width, and height).

[0050] 2. Grid size adaptation: Dynamically adjust the grid division strategy based on the preset standard grid size (such as 5m×5m) and the actual size of the room; 3. Calculation of the number of grids: Based on the ratio between the room size and the standard grid size, the number of grid divisions in each axis is determined using a round-up algorithm. 4. Grid boundary and identifier generation: Based on the row and column layer rules, each grid cell is assigned a unique identifier and its 3D boundary coordinates are calculated; 5. Index mapping construction: Map the generated grid identifiers and boundary information to the spatial index structure to support fast query and positioning.

[0051] Step S1013: Generate a unique identification code for the spatial model through a hash algorithm based on the index information of the spatial model and the spatial grid index information.

[0052] Optionally, a unique QR code is generated for each room, which includes the floor ID, space ID, grid ID, and project code. The data associated with the QR code includes the room-level model URL address (glb) and the spatial grid index, making it easy for mobile devices to quickly access relevant information. Scanning the QR code allows users to directly enter the spatial model of the target room, simplifying on-site operations. Specifically; The structured data fields contained in the QR code are: Data={ProjectCode,FloorNo,RoomID,GLB_URL,GridIndexMap} Among them, ProjectCode is the project identifier, FloorNo is the floor identifier, RoomID is the grid identifier, GLB model, URL address, and GridIndexMap is the mapping table between grid ID (representation) and boundary coordinates.

[0053] Generate a unique QR code identifier through a hash algorithm. The formula is as follows: QR_UUID = Hash(ProjectCode + FloorN} + RoomID) In the formula, the SHA-256 hash function is used to ensure that the QR code in the same room is unique.

[0054] Code conversion: Convert the integrated data into an ISO / IEC 18004 standard two-dimensional code matrix and improve the recognition error tolerance through error correction code algorithms (such as Reed-Solomon code).

[0055] Steps S1011-S1013 spatially segment the BIM model using a hierarchical index structure, achieving logical organization of building data and significantly improving the management efficiency of large-scale models. A grid division method based on actual size and preset density ensures standardized processing of spatial units and optimizes subsequent spatial calculation performance. A hash algorithm is used to generate a unique identification code, establishing a globally unique identity for each spatial model, significantly improving the accuracy of data retrieval. By associating model data, URL addresses, and grid information through identification codes, a complete data access channel is established, enabling rapid acquisition and integration of multi-source information.

[0056] In one embodiment, in step S101, before performing space segmentation and space meshing processing on the BIM model according to the pre-created index structure, the method includes: Analyze the BIM model through 3D boundaries and extract the geometric boundary information of the spatial model from the BIM model. The geometric boundary information includes vertex coordinates and boundary length. According to the positional relationship between components, the spatial correlation between components is analyzed to generate topological relationship data, where the positional relationship includes containment and adjacency relationships; Based on the geometric boundary information and topological relationship, the index structure of the BIM model is generated through the hierarchical relationship of the BIM model. The index structure includes indexes of floors, spaces and grids. Among them, the floor index includes floor identifiers, the space index includes floor identifiers and space identifiers, and the grid index includes floor identifiers, space identifiers and grid identifiers.

[0057] Optionally, Revit space property extraction: Utilize the Autodesk Revit API (e.g., Autodesk.Revit.DB) to read and parse the geometric boundary parameters (e.g., vertex coordinates, boundary lengths, etc.) and spatial topological relationships (e.g., adjacency, containment, etc.) of Room and Space elements in the Revit model. A tree-like index structure is constructed based on the "floor ID → room ID → grid ID" hierarchy. Specifically: The three-dimensional boundary vector analysis method analyzes spatial topological relationships through three-dimensional vector operations: Boundary vector generation: The coordinates of the geometric boundary vertices of the Room / Space element P 1( x 1, y 1,z1), P 2( x 2, y 2,z2)…… P n ( x n , y n ,z n ), calculate the vector difference between adjacent vertices:

[0058] Topological relationship determination: Adjacent relationship: If the boundary vectors of two spaces have collinear reverse parts (the dot product is negative and the modulus length is close), they are considered adjacent. The formula is as follows:

[0059] Containment relationship: If all vertices of space A are within the minimum bounding box of space B, A is considered to be contained by B. The formula is as follows:

[0060] Hierarchical index construction adopts tree encoding rules, and the index value calculation formula is:

[0061] Among them, the coefficients 10^6 and 10^3 are level separation constants and can be adjusted according to the project scale.

[0062] By extracting geometric boundary information from BIM models through 3D boundary analysis technology, accurate digital representation of building component geometry is achieved, laying the data foundation for subsequent spatial analysis. A topological analysis algorithm based on component positional relationships automatically identifies spatial relationships such as containment and adjacency, significantly improving the efficiency of analyzing building spatial relationships. By integrating geometric boundary and topological relationship data to construct a hierarchical index structure, a multi-level spatial index system encompassing floors, rooms, and grids is formed.

[0063] Figure 3 is a flowchart of step S102 according to an exemplary embodiment. Figure 3 As shown, step S102, the mobile terminal identifies the identification code, loads the spatial model corresponding to the identification code, obtains the spatial coordinates in the spatial model according to the gesture operation, converts the spatial coordinates into spatial grid index information according to the preset conversion rules, and generates a loading instruction for the mixed reality device. Specifically, it includes the following steps: Step S1021: Identify the identification code through the mobile terminal and load the space model corresponding to the identification code.

[0064] Optionally, integration with open-source QR code recognition libraries like ZXing or commercial SDKs like Zebra supports offline code scanning. After a user scans a spatial QR code, the system automatically parses the project code, floor number, room number, and other information, and loads the corresponding room-level BIM model and grid index data from the cloud. The data associated with the QR code includes the room-level model URL (glb) and other model URLs, as well as the grid space index map, enabling quick mobile access to relevant information. Scanning a QR code allows users to directly access the spatial model of the target room, streamlining on-site workflows, significantly improving efficiency, and reducing the time required for traditional positioning.

[0065] Step S1022: Acquire spatial coordinates in the spatial model according to the gesture operation.

[0066] Optionally, a graphics engine based on WebGL or UE integrates gesture recognition capabilities and supports common 3D operations: rotation, scaling, component highlighting and selection, etc. Gesture events are captured and processed through front-end JavaScript or UE plug-ins to ensure smooth and natural interaction and a smooth operating experience for users.

[0067] Specifically, users use mobile terminals to scan the spatial QR code deployed at the construction site, and the system quickly parses the information contained therein, such as the project code, floor number, and room number. Based on the parsing results, the mobile terminal quickly loads the corresponding room-level BIM model from the cloud and accurately renders and displays it in a three-dimensional view. It supports rich interactive operations on the model through gesture recognition, including but not limited to: two-finger sliding to rotate the model around the center, allowing users to view the model from different angles; pinching or spreading two fingers to zoom in and out of the model, and flexibly adjusting the model display size; gesture drawing of section planes or clicking buttons to switch section views, clearly presenting the internal structure of the model; clicking on a component to trigger highlighting, pop-up information display, or coordinate picking, making it convenient for users to obtain detailed component information and spatial location.

[0068] Users can click on a specific component (such as an offset duct) to obtain its unique identifier (component ID), or circle an area to record the target spatial coordinates (x, y, z). The system has a built-in spatial coordinate to grid ID mapping algorithm, which is used to determine the spatial grid unit to which the currently selected location belongs. This algorithm achieves rapid positioning from spatial coordinates to logical grids by traversing and judging the room grid bounding box, providing a precise spatial basis for subsequent MR equipment task distribution. In actual application, when construction workers discover a problem in a certain area, they circle the problem area to obtain the coordinates. The system uses this algorithm to quickly determine the corresponding grid ID, laying the foundation for the MR equipment to accurately locate the problem area.

[0069] Step S1023: Convert the spatial coordinates into spatial grid index information using a preset conversion rule. This specifically includes the following steps: Step 1: Based on the spatial coordinates obtained by the mobile terminal, the spatial coordinates of the mobile terminal are converted into the building space coordinates in the building coordinate system through the coordinate system conversion matrix; Step 2: Build a grid boundary list based on the bounding box information of the spatial grid. In the grid boundary list, the initial grid boundary corresponding to the building space coordinates is determined using the spatial hash acceleration algorithm based on the three-level index of row, column, and layer. Step 3: Determine whether the initial mesh boundary satisfies the mesh boundary inequality group. If so, use the initial mesh boundary as the target mesh boundary and ensure that the target mesh boundary is within a predetermined three-dimensional space range. Step 4: Based on the target grid boundary and the index structure, determine the spatial grid index information, where the spatial grid index information includes a grid identifier and a grid bounding box.

[0070] Specifically, coordinate preprocessing: the spatial coordinates (x, y, z) obtained by the mobile terminal must first be converted to the building coordinate system consistent with the grid encoding (to avoid device coordinate system deviation). The formula is: (x', y', z') = T(x, y, z) Where T(·) is the coordinate system transformation matrix (including translation and rotation parameters, obtained through QR code positioning calibration), ensuring that the coordinates and grid boundary parameters are in the same reference.

[0071] According to the boundary parameters of all grids in the known target room (from the spatial grid encoding module), each grid G k , whose boundaries are:

[0072] The specific matching steps are as follows: a. Traverse the boundary data of all grids in the room and build a grid boundary list { G 1 , G 2 ,..., G m}; b. Verify the target coordinates (x', y', z') one by one to see if they meet the grid requirements. G k The set of boundary inequalities for ; c. When a grid G k When all inequalities are satisfied, the coordinate is determined to belong to G k .

[0073] To avoid traversing the entire grid, a three-level index of "row-column-layer" is used for quick positioning: The row index calculation formula is as follows:

[0074] Where x_{\min,\text{room}} is the minimum coordinate of the room in the x direction, S x is the grid size in the x direction.

[0075] The column index calculation formula is as follows:

[0076] The layer index calculation formula (if vertical grid division exists) is as follows:

[0077] By directly locating the candidate grid through (i, j, l) and then verifying whether the boundary matches, the time complexity can be reduced from O(m) to O(1).

[0078] After a successful match, an ID is generated according to the grid's three-level encoding rules: where n y is the total number of grids in the y direction, n z The total number of grid cells in the z direction (default is 1, can be omitted if there is no vertical division).

[0079] For example: If the coordinates (x', y', z') match the grid at row 2 (i=2), column 3 (j=3) of room 5 (RoomID=5) on the 3rd floor (FloorID=3), and n_y=5, then: GridID = "3-5-" + (2 * 5 + 3) = "3-5-13".

[0080] The spatial hash acceleration algorithm in step 2 achieves index mapping with O (1) time complexity. After the spatial hash acceleration algorithm is performed, secondary hashing can be used to avoid hash collisions between grids on different floors and improve mapping accuracy. Specifically: Spatial hash acceleration mapping: It means converting the coordinates into the corresponding row, column, and layer three-level indexes in advance, and then mapping the three-level indexes into unique integers through a hash function to avoid hash collisions and achieve O(1) time complexity queries.

[0081] Secondary hashing: To resolve hash collisions that may occur across grids on different floors (for example, grids on different floors may have the same i, j, and l indexes), the floor ID (or other unique spatial identifier) ​​is appended to the primary hash value to form a string of "hash value_floor ID" (for example, 123456_3 represents a grid on the 3rd floor). This ensures that the hash values ​​of grids across floors are globally unique.

[0082] Spatial hashing acceleration mapping and quadratic hashing are progressively related. The three-level index (i, j, l) is the foundation, used to identify the location of a grid in space. Spatial hashing acceleration mapping optimizes efficiency based on the three-level index. Using a hash function, the three-level index is converted into a single-value hash, avoiding the O(n) complexity of traditional traversal queries. Quadratic hashing complements hash mapping, resolving hash collision issues across spaces (e.g., different floors). Step S1024: Generate loading instructions for the mixed reality device based on the spatial grid index information.

[0083] Optionally, the mobile terminal determines the spatial grid index information based on the user's operation, generates an MR loading instruction, and forwards it to the target MR device via the cloud. Upon receiving the instruction, the MR device can automatically load the corresponding grid model and focus on the specified coordinate point, enabling rapid response and visual confirmation of on-site issues. For example, during construction, a construction worker may discover a deviation in the installation of a component in a certain grid area on the mobile terminal. This operation generates an MR loading instruction and sends it to the MR device. Once the MR device operator arrives at the site wearing the device, the device automatically loads the corresponding grid model and focuses on the problematic coordinate point, allowing for intuitive visualization of the problem and subsequent processing, greatly improving the efficiency and accuracy of problem solving.

[0084] Steps S1021-S1024, quickly identify the unique identification code and accurately load the corresponding spatial model through the mobile terminal, significantly improving the retrieval and loading efficiency of the BIM model and avoiding the waste of resources in loading the entire model. Spatial coordinates are obtained based on natural gesture interaction, achieving an intuitive and efficient human-computer operation experience and lowering the user threshold. The coordinates are converted into grid index information through preset conversion rules to ensure the accuracy and consistency of spatial positioning. Finally, dynamic loading instructions for mixed reality devices are generated based on the grid index, realizing on-demand loading and real-time rendering, greatly reducing data transmission volume and improving response speed, and solving the problem of low collaborative efficiency in traditional methods. An efficient, accurate and user-friendly interaction system is constructed.

[0085] Step S103: The mixed reality device loads the spatial model corresponding to the spatial grid index information according to the loading instruction, and superimposes the spatial model onto the real building scene through the positioning strategy. Specifically, the following steps are included: Step 1: Pre-build the mapping relationship between the feature point set of the real building scene and the feature point set corresponding to the world coordinates to reduce the amount of real-time feature matching calculations.

[0086] Step 2: Use the visual sensor of the mixed reality device to scan the QR code feature points and extract the image coordinates of the spatial model; based on the mapping relationship, establish the correspondence between the image coordinates and the world coordinates.

[0087] Step 3: Based on the correspondence, the rotation matrix and translation vector of the mixed reality device are obtained through the EPnP optimization algorithm. The optimization is performed by minimizing the sum of the reprojection errors of each feature point. The reprojection error represents the sum of the squares of the differences between the image coordinates and the actual projection coordinates.

[0088] Step 4: Verify that the reprojection error meets the preset distance. If so, determine the preliminary positioning of the mixed reality device model and the real building scene.

[0089] Step 5: Based on the rotation matrix and translation vector obtained through preliminary positioning, the depth sensor of the mixed reality device is used to collect the environmental point cloud in the real building scene, and the model point cloud associated with the spatial model is loaded.

[0090] Step 6. Through an iterative optimization process, minimize the sum of the converted Euclidean distances between the environment point cloud points and the model point cloud points to obtain the optimal rotation matrix and translation vector, where the transformation is to apply the current rotation matrix and translation vector to map the model point cloud points to the environment point cloud space.

[0091] Step 7: Stop the iteration when the error change between two adjacent iterations is less than a preset condition, and complete the superposition of the spatial model and the real building scene.

[0092] Optionally, after receiving the MR loading command from the mobile device, the MR device parses the target mesh ID and focus coordinates contained therein. Based on the mesh ID, it downloads only the BIM model file (glTF or Unity AssetBundle format) for the mesh in the corresponding room from the cloud, rather than the entire model layer. This model loading strategy is based on a lightweight transmission protocol and supports resumable downloads and cache preloading mechanisms, significantly reducing data transmission and improving loading speeds.

[0093] In the initial alignment stage: the user uses the MR device to scan the QR code deployed in the building space. The system uses the PnP (Perspective-n-Point) algorithm to combine the QR code image features with real-world coordinates to solve the pose, achieving initial spatial registration of the MR model with a positioning accuracy of up to ±3cm.

[0094] During the fine alignment phase, based on the coarse positioning, the system further calls the ICP (Iterative Closest Point) point cloud registration algorithm according to the focus_coord focus coordinates sent by the mobile terminal. It uses the built-in depth sensor of the MR device to obtain the environmental point cloud data, and compares and corrects it with the target mesh model. Finally, the position of the MR model is adjusted to millimeter-level accuracy, and the positioning error is controlled within ±3mm.

[0095] Voice-driven archiving: MR devices integrate voice recognition interfaces and support keyword-activated activation (e.g., "Record view," "Take photo," "Start annotation"). When the user issues the "Record view" command, the system automatically captures the current AR overlay state and generates structured archival data. This data is uploaded to a cloud database and linked to the corresponding BIM components, problem descriptions, and operator information, forming a complete traceability record of the construction process.

[0096] Specifically, the core principle of the coarse positioning stage (PnP algorithm) is to solve the device pose (rotation matrix R and translation vector t) through the 3D world coordinates of the QR code and the 2D image coordinates taken by the MR device camera.

[0097] Perspective projection model: Let the world coordinates of a feature point on the QR code be P w = (X, Y, Z, 1)^T, and its pixel coordinates on the MR camera image are p = (u, v, 1)^T, which satisfies the following formula:

[0098] Where s is the scale factor, K is the camera intrinsic matrix (known and obtained from device calibration), and t] is the 3×4 pose matrix (R is the 3×3 rotation matrix, and t is the 3×1 translation vector). Pose solution: The EPnP (EfficientPerspective-n-Point) optimization algorithm is used to solve R and t by minimizing the reprojection error. The formula is:

[0099] Where π(.) is the perspective projection function, and n is the number of feature points of the QR code (usually 4 corner points).

[0100] The implementation steps are as follows: a. The MR device scans the QR code and extracts the image coordinates of the four corner points p1 to p4; b. Read the world coordinate P bound to the QR code w 1 ~ P w 4; c. Substitute the above formula to solve the pose matrix [R | t] to complete the initial alignment of the MR model with the real space; d. Positioning accuracy verification: Calculate the reprojection error and ensure that the error is ≤ 3 cm.

[0101] The core principle of the fine alignment stage (ICP algorithm) is to minimize the spatial distance between the environmental point cloud collected by the MR device and the target grid model point cloud through iterative optimization, thereby achieving millimeter-level calibration.

[0102] Assume that the environment point cloud is Q = {q1, q2,..., qm} and the model point cloud is P = {p1, p2,..., pm}. Find the optimal transformation (R, t) by the following formula:

[0103] The iterative convergence condition is: stop when the error change between two adjacent iterations is less than a threshold (such as 0.1mm), which is expressed as follows:

[0104] Implementation steps: Based on the coarse positioning results, the MR device collects the environmental point cloud Q of the target grid area through the depth sensor; Load the target mesh's model point cloud P from the cloud (associated with the mobile terminal's focus coordinates).

[0105] Iterative process: Step 1: Establish the point correspondence between Q and P through nearest neighbor search; Step 2: Solve the above objective function to obtain the current optimal transformation (R, t); Step 3: Apply the transformation to update the point cloud P and calculate the error E; Step 4: If not converged, return to step 1 and repeat the iteration; The final positioning error is controlled within ±3mm, completing the precise alignment of the MR model with the real environment.

[0106] Step S103 significantly reduces the computational complexity of real-time feature matching in mixed reality devices and improves positioning efficiency by pre-building a set of scene feature points and their world coordinate mapping relationships. Based on the QR code feature points and the EPnP optimization algorithm, high-precision initial pose estimation is achieved to ensure rapid alignment of the virtual model with the real scene. Through the reprojection error verification mechanism, reliable positioning results are effectively screened to avoid erroneous superposition. Combining the environmental point cloud collected by the depth sensor with iterative closest point (ICP) optimization, the model alignment accuracy is further improved to ensure the stability of virtual-reality fusion. Finally, through dynamic convergence judgment conditions, computing resource consumption is optimized while ensuring positioning accuracy. This solves the key problems of slow loading, inaccurate alignment, and unstable virtual-reality fusion of BIM models in mixed reality, and achieves efficient, accurate, and stable spatial model superposition.

[0107] In one embodiment, in step S103, after the mixed reality device loads the spatial model corresponding to the spatial grid index information according to the loading instruction and superimposes the spatial model onto the real building scene using the positioning strategy, the method further includes: Step 1: Based on the priority message queue mechanism, the stream processing engine performs real-time transmission and processing of mixed reality device operation data. The data processing priority is determined by a dynamic weight allocation algorithm based on the type and urgency of the operation data.

[0108] Optionally, the core logic of the event-driven data flow algorithm (MR-side response link) ensures that MR operation data is efficiently synchronized to all terminals through priority queues and stream processing rules.

[0109] Specifically, in the message queue priority sorting (Kafka stage), MR-side operation records (such as quality inspection results and annotation data) need to be synchronized first, and a dynamic weight allocation algorithm is used to assign message priorities, which is expressed by the following formula: \text{Priority} = w1 \cdot \text{urgency} + w2 \cdot \text{data_size} Where: \text{urgency} is the urgency level (level 1-5, the default level for recording triggered by voice commands is level 5), \text{data_size} is the normalized value of the data size (0-1), w1=0.7, w2=0.3 are weight coefficients (giving priority to emergency operations).

[0110] Kafka sorts messages based on the \text{Priority} value. High-priority messages (>3.5) skip the normal queue and go directly to the Spark fast processing link.

[0111] Stream processing data cleaning (Spark stage) performs real-time cleaning on the raw data uploaded by the MR end (such as posture parameters and voice annotations). The data integrity check is expressed by the following formula:

[0112] The coordinate normalization transformation (unified to the building coordinate system) is expressed by the following formula:

[0113] In the formula, coord MR :Mixed reality coordinates, offset Room :Room coordinate offset, coord Standard : Standard coordinates.

[0114] Step 2: Based on the spatial grid index, establish the association and binding between the mixed reality device operation record and the BIM component through the three-level mapping relationship, and determine the ownership relationship between the component and the grid based on the spatial inclusion relationship judgment criterion.

[0115] Optionally, the core principle of the data association mapping algorithm (cross-table association based on grid ID) is to achieve precise binding of MR operation records and BIM components through three-level index mapping relationships.

[0116] The binding formula of MR record and grid index is: (One-to-one mapping, achieved through primary key and foreign key constraints) Among them, MR_Record: mixed reality operation record table, Grid: spatial grid basic information table, grid_id: grid table primary key, unique identifier.

[0117] The association between grid index and BIM component is: Grid table.element_ids = {e1,e2...ek} Where ek is the element_id of all BIM components in the grid. element_ids is a field in the grid table that stores all component IDs contained in the current grid. The relationship between components and grids is calculated based on the spatial inclusion relationship:

[0118] Among them, ei: the i-th component of Element, Grudk: the k-th grid of Grid, AABB (ei): AxisAlignedBoundingBox up axis-aligned bounding box, using an axis-aligned cuboid to enclose ei, and the spatial range is determined by the coordinate extreme values ​​of the component geometric fixed points, Grid_k.bounding_box: the bounding box of grid K, :equivalence, : subset,∈: belongs to.

[0119] Step 3: Based on the version control mechanism, incremental data push is triggered through the difference detection algorithm. According to the timestamp arbitration rules, multi-terminal data conflicts are resolved through the weighted fusion algorithm.

[0120] Optionally, the core logic of the real-time synchronization trigger mechanism (WebSocket push) is to trigger the mobile UI update based on the difference detection between the data version number and the timestamp.

[0121] Version control formula After the MR end data is uploaded, the version identifier is automatically generated. The formula is as follows: Version = BaseVersion + UpdateCount BaseVersion is the initial version (1.0), and UpdateCount is the cumulative number of updates.

[0122] Push trigger conditions: When the cloud detects that the MR data version is higher than the mobile cache version, a push is triggered. The formula is as follows:

[0123] Among them, Version_cloud: cloud storage version number, Version_mobile: mobile cache version number, >: version number comparison operator, WenSocket: network communication protocol, Push: data push, Delte_data: incremental data Incremental data extraction rules push only changed fields to reduce the amount of transmission. The formula is as follows: ΔData=diff(MR_Recordnew,MR_Recordold) Among them, ΔData: transmission data, Diff: difference calculation, MR_Recordnew: new version mixed reality operation record, MR_Recordold: old version mixed reality operation record.

[0124] The data consistency guarantee algorithm uses distributed locks and timestamp arbitration to avoid multi-terminal synchronization conflicts. Lock contention resolution: When the MR terminal and the mobile terminal operate on the same grid data at the same time, the priority is determined by the timestamp. The formula is as follows: Winner=argmax(TimestampMR,Timestampmobile) Among them, Winner: operation authority, Argmax: maximum value, TimestampMR: MR operation timestamp, Timestampmobile: mobile terminal operation timestamp.

[0125] Conflict repair: Weighted fusion is used for conflicting data (MR data has a higher weight because it contains field measurement information). The formula is as follows: MergedData=0.7Data_MR+0.3Data_mobile MergedData: fused data, 0.7 / 0.3 weight coefficient, Data_MR: MR data, Data_mobile: mobile data.

[0126] Data association rule: Operation records are associated with the BIM component library through the grid ID: MR_Record table.grid_id →Grid index table →Revit component table.element_id.

[0127] Among them, MR_Record: mr operation record table, grid index table: grid index table, Revit component table: Revit component library table, Grid_id grid unique identifier, Element_id: component unique identifier.

[0128] Through the priority message queue and dynamic weight allocation mechanism, intelligent hierarchical processing of mixed reality device operation data is achieved, ensuring real-time response to high-urgency operations (such as quality inspection annotation) while optimizing system resource allocation. Based on the three-level mapping relationship and bounding box detection of the grid space index, a precise association between operation records and BIM components is established, supporting data consistency maintenance and rapid retrieval across terminals. Through version control and incremental data push, only data fields are changed synchronously, significantly reducing the network transmission load. Combined with timestamp arbitration and weighted fusion algorithms, it effectively resolves conflicts in concurrent operations of multiple terminals, giving priority to retaining MR-side operation results containing on-site measured data. An efficient, reliable, and low-latency multi-terminal data collaboration system has been built to solve the problems of data asynchrony, frequent conflicts, and slow response in traditional BIM mixed reality applications.

[0129] In summary, the embodiment of the present application provides a method for collaboration between a mobile terminal of a BIM model and mixed reality, which performs spatial segmentation and grid division on the BIM model through a pre-created index structure, realizes the structured organization of building space data, and improves the retrieval and processing efficiency of large-scale BIM models. The rapid recognition and loading mechanism based on the unique identification code enables the mobile terminal to accurately locate and instantly retrieve the target space model, effectively reducing the amount of data transmission. Through gesture operation and spatial coordinate conversion technology, natural and intuitive human-computer interaction is achieved, greatly improving the convenience of operation; grid index information is used to generate mixed reality loading instructions to ensure the accuracy and real-time performance of model loading; finally, through a high-precision virtual and real space positioning strategy, the seamless superposition of BIM models and real building scenes is achieved, solving the problem of low collaboration efficiency in traditional methods. An efficient, accurate and easy-to-use method for collaboration between a mobile terminal of a BIM model and mixed reality is constructed.

[0130] Secondly, an embodiment of the present application provides a collaborative system between a BIM model mobile terminal and mixed reality. Figure 4 It is a structural diagram of a collaborative system based on a BIM model mobile terminal and mixed reality. Figure 4 As shown, the device includes: an identification code module 410, a loading instruction module 420 and an overlay module 430, wherein: The identification code module 410 is used to perform spatial segmentation and spatial grid division processing on the BIM model according to a pre-created index structure, obtain index information including multiple spatial models and multiple spatial grids, and construct a unique identification code for the spatial model based on the index information; The loading instruction module 420 is used to identify the identification code through the mobile terminal, load the spatial model corresponding to the identification code, obtain spatial coordinates in the spatial model based on gesture operations, convert the spatial coordinates into spatial grid index information using preset conversion rules, and generate a loading instruction for the mixed reality device; The overlay module 430 is used for the mixed reality device to load the spatial model corresponding to the spatial grid index information according to the loading instruction, and overlay the spatial model onto the real building scene through the positioning strategy.

[0131] In summary, the present application provides a collaborative system between a BIM model mobile terminal and mixed reality. By spatially segmenting and gridding the BIM model through a pre-created index structure, the structured organization of building space data is achieved, and the retrieval and processing efficiency of large-scale BIM models is improved. The rapid recognition and loading mechanism based on the unique identification code enables the mobile terminal to accurately locate and instantly retrieve the target space model, effectively reducing the amount of data transmission. Through gesture operation and spatial coordinate conversion technology, natural and intuitive human-computer interaction is achieved, greatly improving the convenience of operation; grid index information is used to generate mixed reality loading instructions to ensure the accuracy and real-time performance of model loading; finally, through a high-precision virtual and real space positioning strategy, the seamless superposition of BIM models and real building scenes is achieved, solving the problem of low collaboration efficiency in traditional methods. An efficient, accurate and easy-to-use BIM model mobile terminal and mixed reality collaboration method is constructed.

[0132] It should be noted that the collaboration between a mobile BIM model and mixed reality provided in this embodiment is used to implement the above-mentioned implementation method, and the details that have been explained will not be repeated. As used above, the terms "module", "unit", "sub-unit", etc. can be a combination of software and / or hardware that implements the predetermined function. Although the device described in the above embodiment is preferably implemented in software, it is also possible and conceivable to implement it in hardware, or a combination of software and hardware.

[0133] In a third aspect, an embodiment of the present application provides an electronic device, Figure 5 FIG is a block diagram of an electronic device according to an exemplary embodiment. Figure 5 As shown, the electronic device may include a processor 51 and a memory 52 storing computer program instructions.

[0134] Specifically, the processor 51 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0135] The memory 52 may include a large-capacity memory for data or instructions. By way of example, and not limitation, the memory 52 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 52 may include removable or non-removable (or fixed) media. Where appropriate, the memory 52 may be internal or external to the data processing device. In certain embodiments, the memory 52 is non-volatile memory. In certain embodiments, the memory 52 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), where the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data out dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0136] The memory 52 may be used to store or cache various data files that need to be processed and / or used for communication, as well as possible computer program instructions executed by the processor 51 .

[0137] The processor 51 reads and executes computer program instructions stored in the memory 52 to implement any one of the collaborative methods for the BIM model mobile terminal and mixed reality in the above embodiments.

[0138] In one embodiment, a BIM model mobile terminal and mixed reality collaboration device may further include a communication interface 53 and a bus 50. Figure 5 As shown, the processor 51, the memory 52, and the communication interface 53 are connected via a bus 50 and communicate with each other.

[0139] The communication interface 53 is used to enable communication between the various modules, devices, units, and / or devices in the embodiments of the present application. The communication interface 53 can also enable data communication with other components such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations.

[0140] Bus 50 includes hardware, software, or both, and couples components of a BIM model mobile terminal and a mixed reality collaboration device. Bus 50 includes, but is not limited to, at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, bus 50 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Bus 50 may include one or more buses, where appropriate. Although embodiments herein describe and illustrate a particular bus, this application contemplates any suitable bus or interconnect.

[0141] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, a method for collaboration between a mobile BIM model and mixed reality provided in the first aspect is implemented.

[0142] The readable storage medium may include, but is not limited to, a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0143] In a possible embodiment, the present invention can also be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of a collaborative method for a BIM model mobile terminal and mixed reality provided in the first aspect.

[0144] The program code for executing the present invention may be written in any combination of one or more programming languages, and may be executed entirely on the user device, partially on the user device, as a standalone software package, partially on the user device and partially on a remote device, or entirely on the remote device.

[0145] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0146] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A collaborative method between a mobile BIM model and mixed reality, characterized in that: The method comprises: Performing spatial segmentation and spatial grid division processing on the BIM model according to a pre-created index structure to obtain index information including multiple spatial models and multiple spatial grids, and constructing a unique identification code for the spatial model based on the index information; Identifying the identification code through a mobile terminal, loading the spatial model corresponding to the identification code, obtaining spatial coordinates in the spatial model according to a gesture operation, converting the spatial coordinates into spatial grid index information according to a preset conversion rule, and generating a loading instruction for a mixed reality device; The mixed reality device loads the spatial model corresponding to the spatial grid index information according to the loading instruction, and superimposes the spatial model on the real building scene through a positioning strategy.

2. A method for collaboration between a mobile BIM model and mixed reality according to claim 1, characterized in that: According to the pre-created index structure, the BIM model is spatially segmented and spatially meshed to obtain index information including multiple spatial models and multiple spatial meshes. Based on the index information, a unique identification code for the spatial model is constructed, including: Segmenting the BIM model according to the spatial index in the index structure to generate multiple spatial models, and obtaining index information of the spatial models; Dividing a spatial grid based on an actual size of the spatial model and a preset grid density, and determining spatial grid index information and a spatial grid bounding box based on the divided spatial grid; Based on the index information of the spatial model and the spatial grid index information, a spatial model unique identification code is generated by a hash algorithm.

3. The method for coordinating a mobile BIM model with mixed reality according to claim 2, characterized in that: The BIM model is segmented according to the spaces in the index structure to generate multiple space models, and index information of the space models is obtained, including: Acquire model geometric vertex coordinate data whose index structure is a spatial index, and determine a three-dimensional bounding box of the spatial model by calculating coordinate extreme values ​​based on the geometric vertex coordinate data; Based on the three-dimensional bounding box, a geometric screening algorithm is used to retain the building components and cross-space connection points located inside the spatial model to generate multiple initial spatial models, wherein, in the multiple spatial models, cross-space components are anchored by connection points to determine the topological integrity of the cross-space components; Converting the initial spatial model into a spatial model in GLB format using a three-dimensional graphics compression conversion algorithm; According to the index structure, the index information of the spatial model is generated through multi-level digital coding rules.

4. The method for coordinating a mobile BIM model with mixed reality according to claim 2, characterized in that: The dividing the spatial grid based on the actual size of the spatial model and the preset grid density, and determining the spatial grid index information and the spatial grid bounding box based on the divided spatial grid, comprises: Set the size parameters of the standard grid according to the preset grid division density; According to the geometric boundary data of the spatial model, the actual physical size of the spatial model is obtained by a three-dimensional size measurement algorithm; Based on the proportional relationship between the actual physical size and the standard grid size, the number of spatial grid divisions in the spatial model is calculated using a grid coverage algorithm, and grid division is performed based on the number of spatial grid divisions to obtain a plurality of spatial grids; Generate spatial grid index information of each item through multi-level coding rules, wherein the spatial grid index information includes spatial position information; Based on the spatial position information, a grid bounding box of each grid is determined by a coordinate assignment algorithm.

5. The method for coordinating a BIM model mobile terminal with mixed reality according to claim 1, characterized in that: The spatial coordinates are converted into spatial grid index information using a preset conversion rule, including: According to the spatial coordinates obtained by the mobile terminal, the spatial coordinates of the mobile terminal are converted into building spatial coordinates in a building coordinate system through a coordinate system conversion matrix; Constructing a grid boundary list based on the bounding box information of the spatial grid, and determining the initial grid boundary corresponding to the building space coordinates using a spatial hash acceleration algorithm based on the three-level index of row, column, and layer in the grid boundary list; determining whether the initial grid boundary satisfies a grid boundary inequality group, and if so, using the initial grid boundary as a target grid boundary, and determining that the target grid boundary is within a predetermined three-dimensional space range; Based on the target grid boundary and according to the index structure, spatial grid index information is determined, wherein the spatial grid index information includes a grid identifier and a grid bounding box.

6. The method for coordinating a mobile BIM model with mixed reality according to claim 5, characterized in that: In the grid boundary list, determining the grid boundary corresponding to the building space coordinates by three-level indexes of row, column, and layer includes: According to the x-axis coordinate value of the building space coordinate, the row index number is calculated by the ratio of the minimum coordinate in the x-direction of the building space to the x-direction size of the grid; According to the y-axis coordinate value of the building space coordinate, the column index number is calculated by the ratio of the minimum coordinate in the y-direction of the building space to the y-direction size of the grid; According to the z-axis coordinate value of the building space coordinate, the layer index number is calculated by the ratio of the minimum coordinate in the z direction of the building space to the z-direction size of the grid; A grid boundary corresponding to the building space coordinate is determined based on the row index number, the column index number and the layer index number.

7. The method for coordinating a BIM model mobile terminal with mixed reality according to claim 1, characterized in that: The mixed reality device includes a visual sensor, and the positioning strategy is used to superimpose the spatial model onto the real building scene, including: Pre-constructing a real building scene feature point set and a mapping relationship between the feature point set and the world coordinates; Scanning the feature points of the QR code with the visual sensor of the mixed reality device to extract the image coordinates of the spatial model; and establishing a corresponding relationship between the image coordinates and the world coordinates based on the mapping relationship; Based on the correspondence, obtaining a rotation matrix and a translation vector of the mixed reality device through an optimization algorithm, wherein the optimization is performed by minimizing a sum of reprojection errors of each feature point, wherein the reprojection error represents the sum of squares of differences between image coordinates and actual projected coordinates; Verifying that the reprojection error meets a preset distance, and if so, determining a preliminary positioning of the mixed reality device model and the real building scene; Based on the rotation matrix and the translation vector obtained by the preliminary positioning, collecting an environment point cloud in a real building scene through a depth sensor of a mixed reality device, and loading a model point cloud associated with a spatial model; Through an iterative optimization process, the sum of the transformed Euclidean distances between the environment point cloud points and the model point cloud points is minimized to obtain the optimal rotation matrix and translation vector, where the transformation is to map the model point cloud points to the environment point cloud space using the current rotation matrix and translation vector; The superposition of the spatial model and the real building scene is completed by stopping the iteration when the error variation between two adjacent iterations is less than a preset condition.

8. The method for coordinating a BIM model mobile terminal with mixed reality according to claim 1, characterized in that: Before performing space segmentation and space grid division processing on the BIM model according to the pre-created index structure, the method further includes: parsing the BIM model through the three-dimensional boundary, and extracting geometric boundary information of the spatial model from the BIM model, the geometric boundary information including vertex coordinates and boundary length; Analyzing the spatial association between components according to the positional relationship between the components to generate topological relationship data, wherein the positional relationship includes containment and adjacency relationships; Based on the geometric boundary information and topological relationship, an index structure of the BIM model is generated through the hierarchical relationship of the BIM model, wherein the index structure includes indexes of floors, spaces, and grids, wherein the floor index includes a floor identifier, the space index includes the floor identifier and the space identifier, and the grid index includes the floor identifier, the space identifier, and the grid identifier.

9. The method for coordinating a mobile BIM model with mixed reality according to claim 1, characterized in that: After superimposing the spatial model onto the real building scene using a positioning strategy, the method further includes: Based on the priority message queue mechanism, the stream processing engine performs real-time transmission and processing of mixed reality device operation data. The dynamic weight allocation algorithm determines the data processing priority based on the operation data type and urgency. Based on the spatial grid index, the association and binding between the mixed reality device operation record and the BIM component is established through the three-level mapping relationship. The attribution relationship between the component and the grid is determined according to the spatial inclusion relationship judgment criterion. Based on the version control mechanism, incremental data push is triggered by the difference detection algorithm, and multi-terminal data conflicts are resolved through the weighted fusion algorithm according to the timestamp arbitration rules.

10. A collaborative system between a mobile BIM model terminal and mixed reality, characterized in that: The system includes: an identification code module, a loading instruction module and an overlay module, wherein: The identification code module is used to perform spatial segmentation and spatial grid division processing on the BIM model according to a pre-created index structure, obtain index information including multiple spatial models and multiple spatial grids, and construct a unique identification code for the spatial model based on the index information; The loading instruction module is configured to identify the identification code through a mobile terminal, load the spatial model corresponding to the identification code, obtain spatial coordinates in the spatial model according to a gesture operation, convert the spatial coordinates into spatial grid index information according to a preset conversion rule, and generate a loading instruction for the mixed reality device; The overlay module is used for the mixed reality device to load the spatial model corresponding to the spatial grid index information according to the loading instruction, and overlay the spatial model onto the real building scene through a positioning strategy.

11. An electronic device, characterized in that: The method comprises a memory and a processor, a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, a method for collaborating between a mobile terminal of a BIM model and mixed reality as described in any one of claims 1 to 9 is implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a method for collaborating between a BIM model mobile terminal and mixed reality is implemented as described in any one of claims 1 to 9.

Images