A dynamic interaction method and system of building information model based on mixed reality

Abstract

The present application discloses a mixed reality-based building information model dynamic interaction method and system, the method comprising: obtaining geometric information and attribute information of a target building information model, storing the geometric information and attribute information in a custom data structure, and constructing a graphic language transmission format model according to the custom data structure; simplifying the geometric information of the graphic language transmission format model, and constructing a three-level multi-detail level model; using a spatial index to segment the three-level multi-detail level model respectively to obtain a plurality of multi-detail level slices; determining the loading of each tile in each multi-detail level slice according to a view cone and a view cone position; determining the spatial transformation matrix of each tile in a world coordinate system according to a spatial coordinate system of each multi-detail level slice, and obtaining the position of each tile in the world coordinate system, so that a mixed reality device can smoothly load and render building information model data.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a mixed reality-based building information model dynamic interaction method, system, terminal and computer-readable storage medium. Background Art

[0002] The rapid development of the new generation of ICT (information and communications technology) such as "people, big data, things, and cloud" has provided key technical support for the construction of new smart cities. In response to the actual needs of refined urban governance, the use of mixed reality technology to achieve interactive progress management of construction projects under construction will help improve the efficiency of project progress management.

[0003] In the existing technology, a complete physical BIM (Building Information Modeling) system usually has 300,000 to 500,000 components. With such a huge amount of data, directly loading data will put a lot of pressure on the memory and video memory of MR (Mixed Reality) devices, and MR software and hardware devices have performance bottlenecks when it comes to interactive visualization needs for large-scale, ultra-fine building information models. It is still difficult to directly load BIM data through MR devices and achieve smooth loading and rendering, and it is impossible to achieve real-time, continuous, and smooth visualization of complex BIM models, which limits the practical application of related technologies. Summary of the invention

[0004] In view of this, the present application provides a mixed reality-based building information model dynamic interaction method, system, terminal and computer-readable storage medium to solve the problem in the prior art that directly loading BIM data through MR devices cannot achieve smooth loading and rendering difficulties.

[0005] The present application proposes a mixed reality-based building information model dynamic interaction method, the mixed reality-based building information model dynamic interaction method comprising:

[0006] Acquire geometric information and attribute information of a target building information model, store the geometric information and the attribute information in a custom data structure, and construct a graphic language transmission format model according to the custom data structure;

[0007] Simplifying the geometric information of the graphic language transmission format model to obtain an intermediate graphic language transmission format model, and constructing a three-level multi-level detail model according to the intermediate graphic language transmission format model;

[0008] Using spatial indexes to segment the three-level multi-detail model respectively to obtain multiple multi-detail level slices;

[0009] Obtaining a view cone and a view cone position of a mixed reality, and determining the loading of each tile in each of the multi-level-of-detail slices according to the view cone and the view cone position;

[0010] According to the spatial coordinate system of each of the multi-level-of-detail slices, the spatial transformation matrix of each of the tiles in the world coordinate system is determined to obtain the position of each of the tiles in the world coordinate system.

[0011] Optionally, the acquiring of geometric information and attribute information of a target building information model, storing the geometric information and the attribute information in a custom data structure, and constructing a graphic language transmission format model according to the custom data structure specifically includes:

[0012] Acquire data information of a target building information model, and extract geometric information and attribute information of each component and each of the components in the target building information model from the data information; wherein the geometric information includes point data, fragment index data and normal data;

[0013] Using the identity information of each of the components to construct a mapping relationship between the geometric information and the attribute information;

[0014] The geometric information, the attribute information and the mapping relationship are stored in a custom data structure, and a graphic language transmission format model is constructed according to the custom data structure.

[0015] Optionally, storing the geometric information, the attribute information and the mapping relationship in a custom data structure, and constructing a graphic language transmission format model according to the custom data structure specifically includes:

[0016] Preset a custom data structure, and store the geometric information, the attribute information and the mapping relationship in the custom data structure;

[0017] The custom data structure is read, and the first attribute of each of the components in the actual application is expanded, and a graphic language transmission format model is constructed according to the custom data structure and the first attribute.

[0018] Optionally, simplifying the geometric information of the graphic language transmission format model to obtain an intermediate graphic language transmission format model, and constructing a three-level multi-level detail model according to the intermediate graphic language transmission format model specifically includes:

[0019] Respectively optimizing redundant geometric triangles of complex components and simple components of the graphic language transmission format model to obtain an intermediate graphic language transmission format model;

[0020] The relationship between the intermediate graphic language transmission format model and the multi-level-of-detail model is preset, and a three-level multi-level-of-detail model is constructed according to the intermediate graphic language transmission format model and the relationship.

[0021] Optionally, using the spatial index to segment the three-level multi-detail model to obtain multiple multi-detail level slices specifically includes:

[0022] Establishing a unified octree index for the multi-level-of-detail models at different levels;

[0023] The multi-level-of-detail model is segmented according to the octree index to obtain multi-level-of-detail slices at different levels, and the multi-level-of-detail slices at different levels are called for instantiation rendering.

[0024] Optionally, the obtaining of a view cone and a view cone position of a mixed reality, and determining the loading of each tile in each of the multi-level-of-detail slices according to the view cone and the view cone position, specifically includes:

[0025] The view cone and view cone position of the mixed reality are obtained, and whether each tile is loaded is determined according to the intersection of the view cone and the bounding box of the multi-level-of-detail slice, and an intersection set between the tiles and a buffer set of each tile are created.

[0026] Optionally, the obtaining of the view cone and the view cone position of the mixed reality, judging whether each tile is loaded according to the intersection of the view cone and the bounding box of the multi-level-of-detail slice, and creating an intersection set between the tiles and a buffer set of each tile specifically includes:

[0027] Obtaining a view cone and a view cone position of a mixed reality, obtaining a sequence of intersection judgments between the view cone and the bounding box of the multi-level-of-detail slice, searching for a corresponding index block from an index list according to the sequence, passing the index block to a request queue, and creating a request set;

[0028] Receive a message from the request queue, determine whether it is a duplicate, and then enable the coroutine to load the request content and create a download set;

[0029] After receiving the downloaded block content, read the cone intersection result, obtain the intersection index, load the block object currently existing in the cone, and obtain the loading set;

[0030] The intersection set is operated according to the request set, the download set and the load set, and the loading of each tile in each of the multi-level-of-detail slices is determined by adopting a loop feedback update method;

[0031] A buffer set is created to record the multi-level-of-detail slices adjacent to each multi-level-of-detail slice intersecting with an edge, and the multi-level-of-detail slices of the buffer set are loaded while the intersecting multi-level-of-detail slices are loaded.

[0032] The present application also proposes a mixed reality-based dynamic interaction system for building information models, the mixed reality-based dynamic interaction system for building information models comprising:

[0033] A graphic language transmission format model reconstruction module is used to obtain geometric information and attribute information of a target building information model, store the geometric information and the attribute information in a custom data structure, and construct a graphic language transmission format model according to the custom data structure;

[0034] A multi-level-of-detail model stratification module is used to simplify the geometric information of the graphic language transmission format model to obtain an intermediate graphic language transmission format model, and to construct a three-level multi-level-of-detail model according to the intermediate graphic language transmission format model;

[0035] A multi-level-of-detail model segmentation module is used to segment the multi-level-of-detail models of three levels respectively using spatial indexes to obtain multiple multi-level-of-detail slices;

[0036] A dynamic loading module, used for obtaining a view cone and a view cone position of a mixed reality, and determining the loading of each tile in each of the multi-level-of-detail slices according to the view cone and the view cone position;

[0037] The virtual mapping module is used to determine the spatial transformation matrix of each tile in the world coordinate system according to the spatial coordinate system of each of the multi-level-of-detail slices, and obtain the position of each tile in the world coordinate system.

[0038] The present application also proposes a terminal, which includes: a memory, a processor, and a mixed reality-based building information model dynamic interaction program stored in the memory and executable on the processor, wherein the mixed reality-based building information model dynamic interaction program, when executed by the processor, implements the steps of the mixed reality-based building information model dynamic interaction method.

[0039] The present application also proposes a computer-readable storage medium, which stores a mixed reality-based building information model dynamic interaction program. When the mixed reality-based building information model dynamic interaction program is executed by a processor, the steps of the mixed reality-based building information model dynamic interaction method are implemented.

[0040] The beneficial effects of the present application are as follows: Different from the prior art, the present application obtains the geometric information and attribute information of the target building information model, stores the geometric information and attribute information in a custom data structure, and constructs a graphic language transmission format model according to the custom data structure, which can avoid the problem of too many drawing calls in the rendering stage and reduce the CPU (Central Processing Unit, i.e., central processing unit) drawing scheduling pressure, reducing the occurrence of screen freezes; secondly, the present application simplifies the geometric information of the graphic language transmission format model to obtain an intermediate graphic language transmission format model, constructs a three-level multi-detail level model according to the intermediate graphic language transmission format model, and uses spatial indexes to segment the three-level multi-detail level models respectively to obtain multiple multi-detail level slices to achieve real-time rendering; in addition, the present application obtains the view cone and view cone position of the mixed reality, determines the loading of each tile in each multi-detail level slice according to the view cone and view cone position, and the mixed reality device smoothly loads the building information model data to achieve a smooth transition; in addition, the present application determines the spatial transformation matrix of each tile in the world coordinate system according to the spatial coordinate system of each multi-detail level slice, obtains the position of each tile in the world coordinate system, realizes the unified virtual-real mapping of the spatial model coordinates and the physical world coordinates, realizes model superposition, and enables the mixed reality device to smoothly render the building information model data.

[0041] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0043] Figure 1 It is a flow chart of a preferred embodiment of the dynamic interaction method of building information model based on mixed reality of the present application;

[0044] Figure 2 This is a BIM data processing flow chart of an embodiment of a building information model dynamic interaction method based on mixed reality in the present application;

[0045] Figure 3 It is a dynamic loading principle diagram of an embodiment of the dynamic interaction method of building information model based on mixed reality of the present application;

[0046] Figure 4 It is the virtual-reality mapping principle diagram and practice diagram of the dynamic interaction method of building information model based on mixed reality in this application;

[0047] Figure 5 It is a building model attribute update diagram of an embodiment of the building information model dynamic interaction method based on mixed reality of the present application;

[0048] Figure 6 It is a schematic diagram of the principle of a preferred embodiment of the dynamic interactive system of building information model based on mixed reality of the present application;

[0049] Figure 7 A schematic diagram of the operating environment of a preferred embodiment of the terminal of the present application. DETAILED DESCRIPTION

[0050] In order to enable those skilled in the art to better understand the technical solution of the present application, the mixed reality-based building information model dynamic interaction method, system, terminal and computer-readable storage medium provided by the present application are further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It can be understood that the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative efforts are within the scope of protection of the present application.

[0051] The terms "first", "second", etc. in this application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0052] The present application provides a mixed reality-based building information model dynamic interaction method, system, terminal and computer-readable storage medium to solve the problem in the prior art that direct loading of BIM data through MR devices cannot achieve smooth loading and rendering difficulties.

[0053] See also Figures 1 to 5 , Figure 1 It is a flow chart of a preferred embodiment of the dynamic interaction method of building information model based on mixed reality of the present application; Figure 2This is a BIM data processing flow chart of an embodiment of a building information model dynamic interaction method based on mixed reality in the present application; Figure 3 It is a dynamic loading principle diagram of an embodiment of the dynamic interaction method of building information model based on mixed reality of the present application;

[0054] Figure 4 It is the virtual-reality mapping principle diagram and practice diagram of the dynamic interaction method of building information model based on mixed reality in this application; Figure 5 This is a building model attribute update diagram of an embodiment of the building information model dynamic interaction method based on mixed reality in the present application.

[0055] This application proposes a dynamic interaction method of building information model based on mixed reality, such as Figure 1 As shown, the dynamic interaction method of building information model based on mixed reality includes the following steps:

[0056] Step S100: obtaining geometric information and attribute information of a target building information model, storing the geometric information and the attribute information in a custom data structure, and constructing a graphic language transmission format model according to the custom data structure.

[0057] Specifically, Figure 2 As shown, before model processing, it is first necessary to obtain the data information of the target building information model, extract the geometric information and attribute information of the building information model, store the geometric information and attribute information in a custom data structure, and build a GLTF (Graphics Language Transmission Forma) model according to the custom data structure. This can avoid the problem of too many drawing calls in the rendering stage, reduce the pressure of CPU (Central Processing Unit) drawing scheduling, and reduce the occurrence of screen freezes. Among them, the GLTF model refers to a 3D (three-dimensional) model file in the gltf format.

[0058] The step S100: obtaining geometric information and attribute information of a target building information model, storing the geometric information and the attribute information in a custom data structure, and constructing a graphic language transmission format model according to the custom data structure, specifically includes:

[0059] Acquire data information of a target building information model, and extract geometric information and attribute information of each component and each of the components in the target building information model from the data information; wherein the geometric information includes point data, fragment index data and normal data;

[0060] Using the identity information of each of the components to construct a mapping relationship between the geometric information and the attribute information;

[0061] The geometric information, the attribute information and the mapping relationship are stored in a custom data structure, and a graphic language transmission format model is constructed according to the custom data structure.

[0062] Specifically, the data information of the target building information model is obtained, and the geometric information and attribute information of each component and each component in the building information model are extracted; wherein the geometric information includes point data, fragment index data, normal data and other geometry-related data, and the ID (Identity document) information of each component is used to build a mapping relationship between the geometric information and the attribute information, and the geometric information, attribute information and mapping relationship are stored in a custom data structure, and a GLTF model is built according to the custom data structure to facilitate model grading in subsequent steps.

[0063] The step of storing the geometric information, the attribute information and the mapping relationship in a custom data structure and constructing a graphic language transmission format model according to the custom data structure specifically includes:

[0064] Preset a custom data structure, and store the geometric information, the attribute information and the mapping relationship in the custom data structure;

[0065] The custom data structure is read, and the first attribute of each of the components in the actual application is expanded, and a graphic language transmission format model is constructed according to the custom data structure and the first attribute.

[0066] Specifically, the geometric information, attribute information and mapping relationship are stored in a custom data structure, the custom data structure is read, and the first attribute of each component in the actual application is expanded, wherein the first attribute is a relevant attribute of each component, which may be maintenance or not, maintenance date, maintenance content, etc. A GLTF model is constructed according to the custom data structure and the first attribute to facilitate model grading in subsequent steps.

[0067] Step S200: simplifying the geometric information of the graphic language transmission format model to obtain an intermediate graphic language transmission format model, and constructing a three-level multi-detail level model according to the intermediate graphic language transmission format model.

[0068] Specifically, the geometric information of the GLTF model is simplified to obtain an intermediate GLTF model, and a three-level LOD model (Levels of Detail) is constructed based on the intermediate GLTF model, and instanced rendering is performed in units of spatial slices.

[0069] The step S200: simplifying the geometric information of the graphic language transmission format model to obtain an intermediate graphic language transmission format model, and constructing a three-level multi-level detail model according to the intermediate graphic language transmission format model, specifically includes:

[0070] Respectively optimizing redundant geometric triangles of complex components and simple components of the graphic language transmission format model to obtain an intermediate graphic language transmission format model;

[0071] The relationship between the intermediate graphic language transmission format model and the multi-level-of-detail model is preset, and a three-level multi-level-of-detail model is constructed according to the intermediate graphic language transmission format model and the relationship.

[0072] Specifically, the number of redundant triangles of complex components of the GLTF model, the number of redundant triangles of simple components and the number of other redundant triangles that do not affect the expression of components are optimized respectively to obtain an intermediate GLTF model, and the relationship between the intermediate graphics language transmission format model and the multi-level of detail model is preset. Three-level LOD models are constructed according to the intermediate GLTF model and the relationship to obtain LOD models of different levels.

[0073] Step S300: using the spatial index to segment the three-level LOD models respectively to obtain a plurality of LOD slices.

[0074] Specifically, a unified spatial index is used to segment the LOD models of the three levels respectively, to obtain multiple LOD slices, to obtain LODs of different blocks, and thus to generate the used 3DTiles.

[0075] The step S300: using the spatial index to segment the three-level LOD models respectively to obtain multiple LOD slices specifically includes:

[0076] Establishing a unified octree index for the multi-level-of-detail models at different levels;

[0077] The multi-level-of-detail model is segmented according to the octree index to obtain multi-level-of-detail slices at different levels, and the multi-level-of-detail slices at different levels are called for instantiation rendering.

[0078] Specifically, if the previous steps are stored as a single GLTF data structure, it will cause too many drawing calls during the rendering stage, resulting in excessive pressure on the CPU drawing scheduling, screen freezes, etc. Therefore, after the GLTF model is built, a unified octree index is established for LOD models of different levels. At the same time, the model is simplified and LOD is created according to actual needs. The LOD model is divided according to the octree index to obtain LOD slices of different levels, and LOD slices of different levels are called for instantiation rendering in a timely manner.

[0079] Step S400: Obtain a view cone and a view cone position of mixed reality, and determine the loading of each tile in each of the multi-level-of-detail slices according to the view cone and the view cone position.

[0080] Specifically, the view cone and view cone position of the mixed reality are obtained, and the loading and unloading of each tile in each LOD slice are determined according to the view cone and the view cone position. The mixed reality device smoothly loads the building information model data to achieve a smooth transition.

[0081] The step S400 of obtaining a view cone and a view cone position of mixed reality, and determining the loading of each tile in each of the multi-level-of-detail slices according to the view cone and the view cone position, specifically includes:

[0082] The view cone and view cone position of the mixed reality are obtained, and whether each tile is loaded is determined according to the intersection of the view cone and the bounding box of the multi-level-of-detail slice, and an intersection set between the tiles and a buffer set of each tile are created.

[0083] Specifically, the view cone and view cone position of the mixed reality are obtained, and whether each tile is loaded is determined based on the intersection of the bounding box of the view cone and the LOD slice, and an intersection set between tiles and a buffer set for each tile are created. If the view cone and the tile intersect, the intersecting tiles are placed in the intersection set, and the buffer set stores the intersecting tiles, i.e., brother tiles.

[0084] The step of obtaining the view cone and view cone position of the mixed reality, determining whether each tile is loaded according to the intersection of the view cone and the bounding box of the multi-level-of-detail slice, and creating an intersection set between the tiles and a buffer set of each tile specifically includes:

[0085] Obtaining a view cone and a view cone position of a mixed reality, obtaining a sequence of intersection judgments between the view cone and the bounding box of the multi-level-of-detail slice, searching for a corresponding index block from an index list according to the sequence, passing the index block to a request queue, and creating a request set;

[0086] Receive a message from the request queue, determine whether it is a duplicate, and then enable the coroutine to load the request content and create a download set;

[0087] After receiving the downloaded block content, read the cone intersection result, obtain the intersection index, load the block object currently existing in the cone, and obtain the loading set;

[0088] The intersection set is operated according to the request set, the download set and the load set, and the loading of each tile in each of the multi-level-of-detail slices is determined by adopting a loop feedback update method;

[0089] A buffer set is created to record the multi-level-of-detail slices adjacent to each multi-level-of-detail slice intersecting with an edge, and the multi-level-of-detail slices of the buffer set are loaded while the intersecting multi-level-of-detail slices are loaded.

[0090] This example uses a queue-based request approach for dynamic loading, uses request sets, download sets, and load sets to operate on intersecting sets, and uses a loop feedback update approach to determine model loading, running once per frame.

[0091] Specifically, Figure 3 As shown, the view cone and view cone position of the mixed reality are obtained, the order of intersection judgment of the bounding box of the view cone and the multi-level of detail slice is obtained, the corresponding index block is found from the index list according to the order, the index block is passed to the request queue, a request set is created, and a message from the request queue is received. After determining whether it is repeated, the coroutine is enabled to load the request content, a download set is created, and after receiving the download block content, the view cone intersection result is read, and the intersection index is obtained to load the block object currently existing in the view cone to obtain the loading set. The intersection set is operated according to the request set, the download set and the loading set, and the loop feedback update method is used to determine the loading of each tile in each LOD slice, create a buffer set, record the LOD slices adjacent to each edge intersecting LOD slice, load the LOD slices of the buffer set while loading the intersecting LOD slices, and use the scale distance accurate to decimeters to determine the level of loading indoor model slices.

[0092] In the process of using MR visualization, the cone changes very quickly. If only the intersecting content is loaded during the block loading process, it may cause jamming and empty screens. In order to avoid this, a buffer set is created based on the above three sets, and the slices adjacent to each edge intersecting slice are recorded. The LOD of the buffer set slice is loaded while the intersecting slice is loaded. Among them, low-level LOD can be used when loading the model to optimize the loading jamming problem during the cone change process. Using a buffer set can prevent the cone from changing too quickly and causing jamming or empty screens, and achieve a smooth transition.

[0093] Step S500: According to the spatial coordinate system of each of the multi-level-of-detail slices, the spatial transformation matrix of each of the tiles in the world coordinate system is determined to obtain the position of each of the tiles in the world coordinate system.

[0094] Specifically, according to the spatial coordinate system of each multi-level-of-detail slice (tile model coordinate system, QR code coordinate system, MR device coordinate system, and world coordinate system), the QR code is scanned by a mixed reality device to determine the spatial transformation matrix of the tile in the world coordinate system, and the position of the tile in the world coordinate system is obtained, thereby achieving a unified virtual-to-real mapping effect of the spatial model coordinates and the physical world coordinates, realizing model superposition, and allowing the mixed reality device to smoothly render the building information model data.

[0095] Wherein, the step S500: according to the spatial coordinate system of each of the multi-level-of-detail slices, scanning the QR code by a mixed reality device, determining the spatial transformation matrix of the tile in the world coordinate system, and obtaining the position of the tile in the world coordinate system, specifically includes:

[0096] According to the spatial coordinate system of each of the multi-level-of-detail slices, a coordinate transformation method based on a two-dimensional code is adopted, and the upper left corner of the two-dimensional code is used as the origin of the model coordinates;

[0097] By scanning the QR code with a mixed reality device, the spatial transformation matrix between the tile in the rendering engine coordinate system, the HoloLens coordinate system, the QR code coordinate system and the model coordinate system is determined, and the position of the tile in the world coordinate system is obtained to achieve the superposition of the virtual model and the physical building.

[0098] Specifically, Figure 4 As shown, you first need to set the initial position of the model in the rendering engine. Due to the difference between the coordinate systems of Untiy and HoloLens, it will eventually be used on HoloLens, so the right-hand coordinate system is used to set it, and this position is used as a link between the virtual model and the real world. In the real building, find the position with the same name and use the upper left corner of the QR code as the origin to create a logo.

[0099] In actual application, the virtual model is actually aligned with the position in the rendering engine coordinates. The transformation matrix is ​​obtained by scanning the QR code, and then the transformation operation is performed to achieve the matching operation. The specific formula is as follows:

[0100] M WM =M WH ·M HN ·M NM ;

[0101] Among them, M WMRepresents the transformation matrix of the tile in the world coordinate system, which is also the posture of M in W space. WH Represents the transformation matrix of HoloLens (MR device) in the world coordinate system, M HN Represents the transformation matrix of the QR code under HoloLens, M NM Represents the transformation matrix of the tile in the QR code coordinate system, and finally obtains the transformation matrix of the model coordinates under the QR code to the world coordinates of the rendering engine:

[0102]

[0103] Among them, θ x Represents the rotation angle around the X axis, θ y Represents the rotation angle around the Y axis, θ z Indicates the rotation angle around the Z axis, t x represents the displacement in the X-axis direction, t y represents the displacement in the Y-axis direction, t z Indicates the displacement in the Z-axis direction.

[0104] By scanning the QR code alignment mark to obtain the transformation matrix, all subsequent loads will be transformed with the same transformation matrix at the same time.

[0105] Step S600: storing a copy of the attribute information on the server side, setting a mutual mapping relationship between the copy and the attribute information, and implementing real-time update of model attributes according to the mutual mapping relationship.

[0106] Specifically, in terms of dynamic updates, this example adopts a cloud-based attribute update method, in which a mapping relationship between BIM model attribute information and its copy is stored in the cloud. The management end can modify the cloud information and map it to the model. At the same time, the field end can modify the model attributes and update the attribute copy to provide basic information for subsequent maintenance work.

[0107] The step S600: storing a copy of the attribute information on the server side, setting a mutual mapping relationship between the copy and the attribute information, and implementing real-time update of the model attributes according to the mutual mapping relationship, specifically includes:

[0108] JSON is used to record the attribute information of the BIM model and stored on the server side. The JSON (JavaScript Object Notation, a lightweight data exchange format) file stores the mapping relationship between attributes and components. Changes in back-end requirements can be mapped to the virtual model through JSON, and modifications to model attributes on the field side can be recorded through JSON.

[0109] Specifically, Figure 5 As shown in the figure, after reading the BIM data, the attribute data is organized in JSON format. Each component has a unique index, and the corresponding relationship of (key, value) can also be customized. After the work order is set up in the back end, it will be mapped to the corresponding attribute / maintenance attribute through the index. When accessing the component attributes, the field end can directly read the attribute information under the index; at the same time, the field end can modify the attributes, including maintenance content, maintenance records, etc.

[0110] See also Figure 6 to Figure 7 , Figure 6 It is a schematic diagram of the principle of a preferred embodiment of the dynamic interactive system of building information model based on mixed reality of the present application; Figure 7 A schematic diagram of the operating environment of a preferred embodiment of the terminal of the present application.

[0111] In some embodiments, Figure 6 As shown, based on the above-mentioned mixed reality-based building information model dynamic interaction method, the present application also proposes a mixed reality-based building information model dynamic interaction system, and the mixed reality-based building information model dynamic interaction system includes:

[0112] A graphic language transmission format model reconstruction module 51 is used to obtain geometric information and attribute information of a target building information model, store the geometric information and the attribute information in a custom data structure, and construct a graphic language transmission format model according to the custom data structure;

[0113] A multi-level-of-detail model stratification module 52 is used to simplify the geometric information of the graphic language transmission format model to obtain an intermediate graphic language transmission format model, and to construct a three-level multi-level-of-detail model according to the intermediate graphic language transmission format model;

[0114] A multi-level-of-detail model segmentation module 53 is used to segment the multi-level-of-detail models of three levels respectively using a spatial index to obtain a plurality of multi-level-of-detail slices;

[0115] A dynamic loading module 54, used for obtaining a view cone and a view cone position of a mixed reality, and determining the loading of each tile in each of the multi-level-of-detail slices according to the view cone and the view cone position;

[0116] The virtual mapping module 55 is used to determine the spatial transformation matrix of each tile in the world coordinate system according to the spatial coordinate system of each of the multi-level-of-detail slices, and obtain the position of each tile in the world coordinate system.

[0117] In some embodiments, Figure 7As shown, based on the above-mentioned mixed reality-based building information model dynamic interaction method and system, the present application also proposes a terminal accordingly, the terminal comprising: a memory 20, a processor 10, a display 30, Figure 7 Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0118] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed in the terminal, such as the program code of the installation terminal, etc. The memory 20 may also be used to temporarily store data that has been output or is to be output.

[0119] In one embodiment, a mixed reality-based building information model dynamic interaction program 40 is stored in the memory 20, and the mixed reality-based building information model dynamic interaction program 40 can be executed by the processor 10, thereby realizing the mixed reality-based building information model dynamic interaction method in the present application.

[0120] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing the mixed reality-based building information model dynamic interaction method.

[0121] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.

[0122] The present application also proposes a computer-readable storage medium, which stores a mixed reality-based building information model dynamic interaction program. When the mixed reality-based building information model dynamic interaction program is executed by a processor, the steps of the mixed reality-based building information model dynamic interaction method as described above are implemented.

[0123] In summary, the present application obtains the geometric information and attribute information of the target building information model, stores the geometric information and attribute information in a custom data structure, and constructs a graphics language transmission format model according to the custom data structure, so as to avoid the problem of too many drawing calls in the rendering stage, reduce the pressure of CPU drawing scheduling, and reduce the occurrence of screen freezes; secondly, the present application simplifies the geometric information of the graphics language transmission format model to obtain an intermediate graphics language transmission format model, constructs a three-level multi-detail level model according to the intermediate graphics language transmission format model, and uses spatial indexes to respectively perform The application obtains multiple multi-level slices by segmenting the rows to achieve real-time rendering; in addition, the application obtains the cone and cone position of the mixed reality, determines the loading of each tile in each multi-level slice according to the cone and cone position, and the mixed reality device smoothly loads the building information model data to achieve smooth transition; in addition, the application determines the spatial transformation matrix of each tile in the world coordinate system according to the spatial coordinate system of each multi-level slice, obtains the position of each tile in the world coordinate system, realizes the unified virtual-real mapping of the spatial model coordinates and the physical world coordinates, realizes the model superposition, and enables the mixed reality device to smoothly render the building information model data. The MR dynamic loading and updating algorithm provided by the present invention realizes the dynamic loading and attribute updating of the BIM model under the existing MR hardware conditions.

[0124] It should be noted that the various optional implementations introduced in the embodiments of the present application can be implemented in combination with each other or can be implemented separately, and the embodiments of the present application are not limited to this.

[0125] In the description of the present application, it should be understood that the terms "upper", "lower", "left", "right", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, and a specific orientation structure and operation. Therefore, it cannot be understood as a limitation on the present application. In addition, "first" and "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present application, unless otherwise specified, "multiple" means two or more.

[0126] In the description of this application, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", etc. should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0127] The above embodiments are described with reference to the accompanying drawings, and other different forms and embodiments are also feasible without departing from the principles of the present application, so the present application should not be constructed as a limitation of the embodiments proposed herein. More specifically, these embodiments are provided so that the present application will be perfect and complete, and the scope of the present application will be conveyed to those skilled in the art. In the accompanying drawings, the component sizes and relative sizes may be exaggerated for clarity. The terms used herein are only based on the purpose of describing specific embodiments and are not intended to be limiting. The terms "comprising" and / or "including" when used in this specification indicate the presence of the features, integers, components and / or components, but do not exclude the presence or increase of one or more other feature integers, components, components and / or their groups. Unless otherwise indicated, when stated, the numerical range includes the upper and lower limits of the range and any sub-ranges therebetween.

[0128] The above descriptions are only some embodiments of the present application, and do not limit the protection scope of the present application. Any equivalent device or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A dynamic interaction method of building information model based on mixed reality, characterized in that: include: Acquire geometric information and attribute information of a target building information model, store the geometric information and the attribute information in a custom data structure, and construct a graphic language transmission format model according to the custom data structure; Simplifying the geometric information of the graphic language transmission format model to obtain an intermediate graphic language transmission format model, and constructing a three-level multi-level detail model according to the intermediate graphic language transmission format model; The step of simplifying the geometric information of the graphic language transmission format model to obtain an intermediate graphic language transmission format model, and constructing a three-level multi-level detail model according to the intermediate graphic language transmission format model specifically includes: Respectively optimizing redundant geometric triangles of complex components and simple components of the graphic language transmission format model to obtain an intermediate graphic language transmission format model; Presetting the relationship between the intermediate graphic language transmission format model and the multi-level-of-detail model, and constructing a three-level multi-level-of-detail model according to the relationship between the intermediate graphic language transmission format model and the multi-level-of-detail model; Using spatial indexes to segment the three-level multi-detail model respectively to obtain multiple multi-detail level slices; The use of spatial indexes to segment the three-level multi-detail level models to obtain multiple multi-detail level slices specifically includes: Establishing a unified octree index for the multi-level-of-detail models at different levels; The multi-level-of-detail model is segmented according to the octree index to obtain multi-level-of-detail slices of different levels, and the multi-level-of-detail slices of different levels are called for instantiation rendering; Obtaining a view cone and a view cone position of a mixed reality, and determining the loading of each tile in each of the multi-level-of-detail slices according to the view cone and the view cone position; The obtaining of the view cone and the view cone position of the mixed reality, and determining the loading of each tile in each of the multi-level-of-detail slices according to the view cone and the view cone position, specifically includes: Obtaining a view cone and a view cone position of mixed reality, determining whether each tile is loaded according to the intersection of the view cone and the bounding box of the multi-level-of-detail slice, and creating an intersection set between the tiles and a buffer set of each tile; The obtaining of the view cone and view cone position of the mixed reality, judging whether each tile is loaded according to the intersection of the view cone and the bounding box of the multi-level-of-detail slice, and creating an intersection set between the tiles and a buffer set of each tile, specifically includes: Obtaining a view cone and a view cone position of a mixed reality, obtaining a sequence of intersection judgments between the view cone and the bounding box of the multi-level-of-detail slice, searching for a corresponding index block from an index list according to the sequence, passing the index block to a request queue, and creating a request set; Receive a message from the request queue, determine whether it is a duplicate, and then enable the coroutine to load the request content and create a download set; After receiving the downloaded block content, read the cone intersection result, obtain the intersection index, load the block object currently existing in the cone, and obtain the loading set; The intersection set is operated according to the request set, the download set and the load set, and the loading of each tile in each of the multi-level-of-detail slices is determined by adopting a loop feedback update method; Creating a buffer set, recording the multi-level-of-detail slices adjacent to each multi-level-of-detail slice intersecting with an edge, and loading the multi-level-of-detail slices of the buffer set while loading the intersecting multi-level-of-detail slices; According to the spatial coordinate system of each of the multi-level-of-detail slices, the spatial transformation matrix of each of the tiles in the world coordinate system is determined to obtain the position of each of the tiles in the world coordinate system.

2. The dynamic interaction method of building information model based on mixed reality according to claim 1, characterized in that: The step of acquiring geometric information and attribute information of a target building information model, storing the geometric information and the attribute information in a custom data structure, and constructing a graphic language transmission format model according to the custom data structure specifically includes: Acquire data information of a target building information model, and extract geometric information and attribute information of each component and each of the components in the target building information model from the data information; wherein the geometric information includes point data, fragment index data and normal data; Using the identity information of each of the components to construct a mapping relationship between the geometric information and the attribute information; The geometric information, the attribute information and the mapping relationship are stored in a custom data structure, and a graphic language transmission format model is constructed according to the custom data structure.

3. The dynamic interaction method of building information model based on mixed reality according to claim 2 is characterized in that: The step of storing the geometric information, the attribute information and the mapping relationship in a custom data structure and constructing a graphic language transmission format model according to the custom data structure specifically includes: Preset a custom data structure, and store the geometric information, the attribute information and the mapping relationship in the custom data structure; The custom data structure is read, and the first attribute of each of the components in the actual application is expanded, and a graphic language transmission format model is constructed according to the custom data structure and the first attribute.

4. A dynamic interactive system of building information model based on mixed reality, characterized in that: The mixed reality-based building information model dynamic interaction system includes: A graphic language transmission format model reconstruction module is used to obtain geometric information and attribute information of a target building information model, store the geometric information and the attribute information in a custom data structure, and construct a graphic language transmission format model according to the custom data structure; A multi-level-of-detail model stratification module is used to simplify the geometric information of the graphic language transmission format model to obtain an intermediate graphic language transmission format model, and to construct a three-level multi-level-of-detail model according to the intermediate graphic language transmission format model; The step of simplifying the geometric information of the graphic language transmission format model to obtain an intermediate graphic language transmission format model, and constructing a three-level multi-level detail model according to the intermediate graphic language transmission format model specifically includes: Respectively optimizing redundant geometric triangles of complex components and simple components of the graphic language transmission format model to obtain an intermediate graphic language transmission format model; Presetting the relationship between the intermediate graphic language transmission format model and the multi-level-of-detail model, and constructing a three-level multi-level-of-detail model according to the relationship between the intermediate graphic language transmission format model and the multi-level-of-detail model; A multi-level-of-detail model segmentation module is used to segment the multi-level-of-detail models of three levels respectively using spatial indexes to obtain multiple multi-level-of-detail slices; The use of spatial indexes to segment the three-level multi-detail level models to obtain multiple multi-detail level slices specifically includes: Establishing a unified octree index for the multi-level-of-detail models at different levels; The multi-level-of-detail model is segmented according to the octree index to obtain multi-level-of-detail slices of different levels, and the multi-level-of-detail slices of different levels are called for instantiation rendering; A dynamic loading module, used for obtaining a view cone and a view cone position of a mixed reality, and determining the loading of each tile in each of the multi-level-of-detail slices according to the view cone and the view cone position; The obtaining of the view cone and the view cone position of the mixed reality, and determining the loading of each tile in each of the multi-level-of-detail slices according to the view cone and the view cone position, specifically includes: Obtaining a view cone and a view cone position of mixed reality, determining whether each tile is loaded according to the intersection of the view cone and the bounding box of the multi-level-of-detail slice, and creating an intersection set between the tiles and a buffer set of each tile; The obtaining of the view cone and view cone position of the mixed reality, judging whether each tile is loaded according to the intersection of the view cone and the bounding box of the multi-level-of-detail slice, and creating an intersection set between the tiles and a buffer set of each tile, specifically includes: Obtaining a view cone and a view cone position of a mixed reality, obtaining a sequence of intersection judgments between the view cone and the bounding box of the multi-level-of-detail slice, searching for a corresponding index block from an index list according to the sequence, passing the index block to a request queue, and creating a request set; Receive a message from the request queue, determine whether it is a duplicate, and then enable the coroutine to load the request content and create a download set; After receiving the downloaded block content, read the cone intersection result, obtain the intersection index, load the block object currently existing in the cone, and obtain the loading set; The intersection set is operated according to the request set, the download set and the load set, and the loading of each tile in each of the multi-level-of-detail slices is determined by adopting a loop feedback update method; Creating a buffer set, recording the multi-level-of-detail slices adjacent to each multi-level-of-detail slice intersecting with an edge, and loading the multi-level-of-detail slices of the buffer set while loading the intersecting multi-level-of-detail slices; The virtual mapping module is used to determine the spatial transformation matrix of each tile in the world coordinate system according to the spatial coordinate system of each of the multi-level-of-detail slices, and obtain the position of each tile in the world coordinate system.

5. A terminal, characterized in that: The terminal includes: a memory, a processor, and a mixed reality-based building information model dynamic interaction program stored in the memory and executable on the processor. When the mixed reality-based building information model dynamic interaction program is executed by the processor, the steps of the mixed reality-based building information model dynamic interaction method according to any one of claims 1 to 3 are implemented.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a mixed reality-based building information model dynamic interaction program, and when the mixed reality-based building information model dynamic interaction program is executed by the processor, the steps of the mixed reality-based building information model dynamic interaction method according to any one of claims 1 to 3 are implemented.

Images