Prefabricated beam-column component detection system and construction control method based on mixed reality
Through the mixed reality technology prefabricated beam and column component detection system, the Hololens2 equipment and image recognition algorithm are used to solve the problems of complexity and relying on manual labor in traditional methods, and high-precision beam and column component recognition and construction guidance are achieved, and building quality is improved. Construction efficiency.
Patent Information
- Application Number
- CN202410003919.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-03
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-01-03
AI Technical Summary
Traditional beam and column component identification and construction methods are complex and rely on manual operations, which are easily affected by errors, making structural stability and safety difficult to ensure.
Adopting a prefabricated beam and column component detection system based on mixed reality technology, using Hololens2 equipment and image recognition algorithm, real-time, non-contact component recognition and construction guidance are achieved through image acquisition, feature recognition, coordinate conversion and virtual and real superposition systems, real-time, non-contact component recognition and construction guidance are achieved, reducing manual labor Intervention to improve identification accuracy and construction efficiency.
It has achieved high-precision identification and construction guidance of beam and column components, reduced labor and time costs, reduced construction risks, improved building quality and construction efficiency, and promoted technological progress in the field of prefabricated building construction.
Smart Images

Figure CN117745710B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image recognition technology, and in particular to a detection and control technology for assembled beam-column components based on mixed reality. Background Art
[0002] In recent years, the development of Mixed Reality (MR) technology has allowed users to interact with virtual objects in the real world, superimposing virtual information on the real world, and providing users with enhanced perception and interactive experience. The mixed reality Hololens2 device has built-in sensors, computing power, and head-mounted displays that can track the user's head movements and environment in real time, and can superimpose virtual information on the real world. The use of this device can bring great potential to the construction field for real-time acquisition and processing of information on building components.
[0003] With the continuous development of the construction industry and technological advancement, prefabricated construction has become a popular construction method due to its higher efficiency, lower cost and better quality control compared to traditional on-site construction. In prefabricated buildings, beam and column components are an important part of building the building skeleton, and their accurate identification, positioning and construction and installation are essential to ensure the stability and safety of the structure. However, traditional beam and column component identification and construction methods face a series of challenges, including complex measurement and positioning processes, high-risk working environments, and manual reliance on operations, which are susceptible to errors. Summary of the invention
[0004] The purpose of the present invention is to provide a prefabricated beam-column component detection system and a construction control method based on mixed reality.
[0005] The present invention is an assembled beam-column component detection system and a construction control method based on mixed reality. The assembled beam-column component detection system based on mixed reality technology includes a mixed reality Hololens2 device 1, an image acquisition system 2, a point marking system 3, a computer PC4, an image processing system 5, and a virtual-reality superposition system 6; the image acquisition system 2 is composed of an RGB camera 2-1, a sensor unit 2-2, an internal storage unit 2-3 and a data transmission module 2-4, and each module is centrally mounted on the mixed reality Hololens2 device 1; the point marking system 3 is composed of a non-marking module 3-1 and a two-dimensional code marking module 3-2; the image processing system 5 is composed of a feature recognition module 5-1, a coordinate conversion module 5-2, an image reconstruction module 5-3, a data storage module 5-4 and a data comparison module 5-5; the virtual-reality superposition system 6 is composed of a user interface UI6-1, an assembly guide animation 6-2 and a spatial coordinate anchor 6-3; the The image acquisition system 2 acquires images through the HoloLens 2 device 1 integrated with the scaling factor, equipped with a high-quality RGB camera 2-1 and a sensor unit 2-2, to capture the required photos or videos in the construction environment; wherein, the scaling factor is used to increase the range of the acquired image to avoid the unusability of the acquired information; the built-in depth sensor and IMU sensor are used to acquire the depth value and attitude angle of the pixel point; after the shooting is completed, the photo or video is saved to the internal storage space 2-3 of the HoloLens 2 device 1, and the corresponding data file is generated; the data transmission module 2-4 uses the TCP / IP network transmission protocol for real-time communication to transmit the acquired data to the computer PC4; during data acquisition, the measuring points are manually selected from multiple angles, different directions, complex backgrounds, ambient lighting and other conditions to increase the number of data sets; the acquisition process enables the researcher mode of the HoloLens 2 device 1 to access the data of each sensor.
[0006] The above-mentioned construction control method of the prefabricated beam-column component detection system based on mixed reality is adopted. The user interface UI6-1 is made using Unity and C# programming language, and capture, upload, and calibration operation buttons are added; the corresponding buttons are selected for operation according to image acquisition and construction requirements; assembly guidance animation 6-2, including component assembly sequence, connection method, fixing method, safety precautions and quality requirements; visual presentation of building information, reduction of manual intervention and provision of real-time construction guidance, and reduction of safety risks on the construction site; spatial coordinate anchoring 6-3 uses coordinate inverse transformation to process world coordinate information and convert it into the spatial coordinate system information of HoloLens2 device 1.
[0007] The present invention has the following advantages over the prior art: the present invention uses mixed reality technology to achieve real-time, non-contact identification and construction guidance of prefabricated beam and column components. Compared with traditional manual measurement and marking methods, the mixed reality Hololens2 device and image recognition algorithm can identify and locate components with high precision, which helps to ensure the correct docking and installation of components, improve building quality and construction efficiency, and reduce manpower and time costs and the risk of human errors. The present invention uses the deep learning Faster R-CNN detection algorithm to identify prefabricated beam and column components. Through the feature recognition module and the coordinate conversion module, it can efficiently and accurately identify and extract component information and transmit it to the cloud database in real time. By constructing the prefabricated building construction drawings as BIM models, combining them with the actual construction process helps to realize digital construction management. The information and location data of the beam and column components are stored in the cloud database, providing a complete recording and tracking function for the construction process. The present invention uses the mixed reality Hololens2 device and the guidance animation produced by Unity to display construction guidance information in real time, which has significant potential in improving the efficiency, quality and sustainability of building construction, and helps to promote technological progress and innovation in the field of prefabricated building construction. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] Figure 1 It is a structural block diagram of a prefabricated beam-column component detection system and a construction control method based on mixed reality according to an embodiment of the present invention; Figure 2 It is a structural block diagram of an image acquisition system in an embodiment of the present invention; Figure 3 It is a structural block diagram of a point marking system in an embodiment of the present invention; Figure 4 is a structural block diagram of an image processing system in an embodiment of the present invention; Figure 5 is a flowchart of a feature recognition module in an embodiment of the present invention; Figure 6 It is a flowchart of the virtual-reality superposition system in an embodiment of the present invention. DETAILED DESCRIPTION
[0009] The present invention is an assembled beam-column component detection system and a construction control method based on mixed reality. The assembled beam-column component detection system based on mixed reality technology includes a mixed reality Hololens2 device 1, an image acquisition system 2, a point marking system 3, a computer PC4, an image processing system 5, and a virtual-reality superposition system 6; the image acquisition system 2 is composed of an RGB camera 2-1, a sensor unit 2-2, an internal storage unit 2-3 and a data transmission module 2-4, and each module is centrally mounted on the mixed reality Hololens2 device 1; the point marking system 3 is composed of a non-marking module 3-1 and a two-dimensional code marking module 3-2; the image processing system 5 is composed of a feature recognition module 5-1, a coordinate conversion module 5-2, an image reconstruction module 5-3, a data storage module 5-4 and a data comparison module 5-5; the virtual-reality superposition system 6 is composed of a user interface UI6-1, an assembly guide animation 6-2 and a spatial coordinate anchor 6-3; the The image acquisition system 2 acquires images through the HoloLens 2 device 1 integrated with the scaling factor, equipped with a high-quality RGB camera 2-1 and a sensor unit 2-2, to capture the required photos or videos in the construction environment; wherein, the scaling factor is used to increase the range of the acquired image to avoid the unusability of the acquired information; the built-in depth sensor and IMU sensor are used to acquire the depth value and attitude angle of the pixel point; after the shooting is completed, the photo or video is saved to the internal storage space 2-3 of the HoloLens 2 device 1, and the corresponding data file is generated; the data transmission module 2-4 uses the TCP / IP network transmission protocol for real-time communication to transmit the acquired data to the computer PC4; during data acquisition, the measuring points are manually selected from multiple angles, different directions, complex backgrounds, ambient lighting and other conditions to increase the number of data sets; the acquisition process enables the researcher mode of the HoloLens 2 device 1 to access the data of each sensor.
[0010] The detection system described above, the point marking system 3 includes a markless module 3-1 and a two-dimensional code marking module 3-2, which can manually select measuring points for information collection; the markless module 3-1 is suitable for data collection under general circumstances, and can manually select feature points with better feature values of measuring points; the two-dimensional code marking module 3-2 uses a two-dimensional code containing additional information of the prefabricated structural component as a marking point.
[0011] In the detection system described above, the image processing system 5 includes a feature recognition module 5-1, a coordinate conversion module 5-2, an image reconstruction module 5-3, a data storage module 5-4 and a data comparison module 5-5; the feature recognition module 5-1 contains a recognition algorithm, processes image information in real time, detects and recognizes image feature points or marking points, and extracts component point information; for feature point markings, a deep learning Faster R-CNN (Faster Region-based Convolutional Neural Network) detection algorithm is used to extract important feature information; for QR code marking points, the algorithm is used in conjunction with the ZXing library to extract the data content of the QR code in the image.
[0012] The detection system described above, the virtual-reality overlay system 6 uses C# programming language and Unity to create an information panel of prefabricated structural components, and creates a construction assembly guidance animation; based on the mixed reality toolkit MRTK and OpenXR, an interactive user interface UI6-1 is created, operation buttons are added, and a construction assembly guidance animation 6-2 is designed; the cloud data storage module 5-4 is called in real time, and the MR model produced by Unity is packaged and sent to Visual Studio, and then deployed to the application of the mixed reality HoloLens2 device 1; at the same time, the world coordinate information is processed using the reverse coordinate transformation algorithm and converted into the spatial coordinate system information of the HoloLens2 device 1; then it is transmitted to the HoloLens2 device 1 to complete the display of the virtual component information on the device; the MR model is approximately proportionally overlaid on the assembly position of the actual beam-column component through spatial coordinate anchoring 6-3 to achieve spatial mapping, virtual-reality overlay, and use guidance animation to guide engineering assembly construction.
[0013] In the detection system described above, the coordinate conversion module 5-2 reads the data stream transmitted by the HoloLens2 device 1, and converts the original coordinate information into world coordinate information on the computer PC; by calculating the data of the sensor unit 2-2 assembled on the HoloLens2 device 1, the original coordinates are mapped into the world coordinate system.
[0014] The detection system described above uses the image reconstruction module 5-3 to import the construction drawings of prefabricated buildings into the Revit software, adjusts and optimizes the drawings according to actual requirements, and constructs a visual BIM model; adds relevant information such as the component template and construction process of the prefabricated building to the BIM model; then uploads the above information to the data storage module 5-4, imports the relevant data and performs appropriate data structure and table design, and calls the data comparison module 5-5 to compare the database component information to determine the component type.
[0015] The above-mentioned construction control method of the prefabricated beam-column component detection system based on mixed reality is adopted. The user interface UI6-1 is made using Unity and C# programming language, and capture, upload, and calibration operation buttons are added; the corresponding buttons are selected for operation according to image acquisition and construction requirements; assembly guidance animation 6-2, including component assembly sequence, connection method, fixing method, safety precautions and quality requirements; visual presentation of building information, reduction of manual intervention and provision of real-time construction guidance, and reduction of safety risks on the construction site; spatial coordinate anchoring 6-3 uses coordinate inverse transformation to process world coordinate information and convert it into the spatial coordinate system information of HoloLens2 device 1.
[0016] The construction control method of the assembled beam-column component detection system based on mixed reality described above is based on the component detection system and is used to indicate and guide the construction on the site; the specific steps are: the image acquisition system 2 integrated in the mixed reality Hololens2 device 1 is used to collect the working condition image of the site; at the same time, according to the marking module of the point marking system 3, the measuring points are manually selected to collect component information; the data transmission TCP / IP network transmission protocol is used for real-time communication to transmit the collected data to the computer PC4;
[0017] Secondly, the recognition algorithm of the image processing system 5 is used to process the image information in real time, detect and recognize the image feature point module or the marking point module, and simultaneously reconstruct the three-dimensional visualization BIM model of the prefabricated building, extract the image information and the corresponding BIM template information, upload and store them to the cloud database, and compare the component data to determine the component type;
[0018] Finally, the virtual-reality superposition system 6 is used to produce the prefabricated structural component information UI, which is used to display the component comparison results and produce the construction assembly guidance animation; the comparison results of the cloud database are retrieved in real time, packaged and sent to the mixed reality HoloLens2 device 1, and the virtual model is superimposed on the real scene to achieve space mapping, virtual-reality superposition, and guide the engineering assembly construction.
[0019] The following will be combined with the accompanying drawings to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0020] Figure 1 The present invention is a block diagram of a prefabricated beam-column component detection system and a construction control method based on mixed reality according to an embodiment of the present invention. The system comprises: a mixed reality device Hololens 2 (1), an image acquisition system 2, a point marking system 3, a computer PC 4, an image processing system 5 and a virtual-reality superposition system 6.
[0021] The method comprises: first, collecting data on the construction site of the assembled beam-column component by using an image acquisition system 2;
[0022] Second: manually select the marking points through the point marking system 3;
[0023] Third: The image processing system 5 uses an image recognition algorithm to process image information in real time, identify image feature points or marking points, and extract component feature point information.
[0024] Fourth: The virtual component information on the equipment is displayed through the virtual-reality superposition system 6, space mapping and virtual-reality superposition are realized, and the guiding animation is used to guide the engineering assembly construction.
[0025] Specifically, the image acquisition system 2 integrated in the mixed reality Hololens2 device 1 is used to take photos or record videos of the scene, and the image information of the beam and column components is initially obtained as the original data set. At the same time, according to the marking module of the point marking system 3, the measuring points are manually selected for information collection. The real-time Socket communication between the HoloLens2 device 1 and the computer PC4 is carried out using the TCP / IP network transmission protocol, and the data collected by the HoloLens2 device 1 is transmitted to the computer PC4. The recognition algorithm of the image processing system (5) is used to process the image information in real time, detect and identify the image feature point module or the marking point module, and extract the component feature point information, and upload and store it to the Azure SQL Database cloud database synchronously. The original coordinate information transmitted from the HoloLens2 device 1 to the computer PC4 is converted into world coordinate information by coordinate conversion, and uploaded and stored in the cloud database. A three-dimensional visual BIM model is constructed according to the prefabricated building construction drawing, and the beam and column component template data, construction process and other information are stored in the database, and the database component information is called for comparison to determine the component type. The virtual-reality overlay system 6 includes C# programming language and Unity to create an information panel for assembled structural components, display component comparison result information, and create a construction assembly guidance animation. The cloud database comparison information is retrieved in real time and packaged and sent to the mixed reality HoloLens2 device 1, and the virtual model is superimposed on the real scene to achieve space mapping, virtual-reality overlay, and guide engineering assembly construction.
[0026] Figure 2 1 is a structural block diagram of an image acquisition system in an embodiment of the present invention, which mainly includes: an RGB camera 2-1, a sensor unit 2-2, an internal storage unit 2-3 and a data transmission module 2-4.
[0027] Use a high-quality RGB camera 2-1 equipped with a fusion scaling factor to collect images and take photos or videos of the real construction environment. The scaling factor can be adjusted according to the range of the collected image to avoid the collected information being unusable.
[0028] Sensor unit 2-2 integrates depth sensor, IMU sensor, hand tracking and eye movement sensor. The depth sensor is used to collect pixel depth values and estimate the distance of the component relative to the camera. The IMU sensor is used to collect device attitude angle information, and the depth sensor and IMU sensor are used to collect data original coordinate information. Hand tracking and eye movement sensors are used to track fingers or apply holographic images. By looking at the selected holographic image, functions such as air tapping, dragging and moving, and adjusting window size can be completed.
[0029] The internal storage space 2-3 is used to save photos or videos to the HoloLens 2 device 1 and generate corresponding data files after image acquisition is completed.
[0030] The data transmission module 2 - 4 uses the TCP / IP network transmission protocol to perform real-time Socket communication between the HoloLens 2 device 1 and the computer PC4, and transmits the data collected by the HoloLens 2 device 1 to the computer PC4.
[0031] Figure 3 The structure block diagram of the point marking system in the embodiment of the present invention mainly includes: a non-marking module 3-1 and a two-dimensional code marking module 3-2. The non-marking module 3-1 is suitable for data collection under general conditions, and can manually select feature points with better feature values of the measuring points. The two-dimensional code marking module 3-2 uses a two-dimensional code containing additional information of the assembled structural component as a marking point.
[0032] Figure 4 1 is a structural block diagram of an image processing system in an embodiment of the present invention, which mainly includes: a feature recognition module 5-1, a coordinate conversion module 5-2, an image reconstruction module 5-3, a data storage module 5-4 and a data comparison module 5-5.
[0033] The feature recognition module 6-1 mainly uses labelImg to mark various feature points or marker points, and uses the Faster R-CNN deep learning neural network to train with various feature points and marker points as data sets. After achieving the desired effect, the image to be detected is tested to identify, track and extract coordinates of the feature points or marker points.
[0034] The coordinate conversion module 5 - 2 maps the original coordinates into the world coordinate system by calculating the data of the sensor unit 2 - 2 .
[0035]
[0036] The image reconstruction module (5-3) imports the prefabricated building construction drawings into the Revit software, adjusts and optimizes the drawings according to actual requirements, and constructs a visual BIM model. Adds relevant information such as the component template and construction process of the prefabricated building to the BIM model. This includes template information such as component type, size, material, connection method, and operation procedures such as construction steps and construction sequence.
[0037] The data storage module 5-4 uses Azure SQL Database cloud database to store beam and column component template data, construction process and other information, and imports relevant data to perform appropriate data structure and table design. Cloud storage facilitates multiple users to access and edit data at any time and realize collaborative work.
[0038] The data comparison module 5-5 uses the database component information to compare the beam-column component template data to determine the component type.
[0039] Figure 5 This is a flowchart of the feature recognition module in an embodiment of the present invention. The feature recognition module uses the Faster R-CNN detection algorithm, and this embodiment is the output result of the training model. For the QR code markers, the algorithm uses the ZXing library to extract the data content of the QR code in the image.
[0040] First, input the image and preprocess it. The input image is scaled to the network input size through the VGG16 model. After 13 conv layers, 4 pooling layers, and 13 relu layers, the features in the input image are extracted and a feature map is generated for use by subsequent algorithms.
[0041] After convolution, the RPN module uses binary classification to give a foreground score to each feature map anchor, and uses regression to calculate the fine-tuning parameters between each anchor and its corresponding true target box GT. Based on the foreground score and fine-tuning parameters, the prediction box ROI is calculated and the appropriate RP is selected to obtain the effective anchor confidence and the coordinate coefficient with the prediction box.
[0042] The feature map and the Proposal calculated by the prediction box are used through ROIpooling to obtain a fixed-size prediction target feature map. The fixed-size feature map is sent to the full connection layer to predict bbox_pred and cls_prob in softmax for classification.
[0043] Figure 6 It is a flowchart of the virtual-reality superposition system in the embodiment of the present invention, which mainly includes: user interface UI6-1, assembly guide animation 6-2 and space coordinate anchoring 6-3.
[0044] The user interface UI6-1 is based on the Mixed Reality Toolkit (MRTK) and OpenXR, and is made using Unity and C# programming languages. It adds buttons for capture, upload, and calibration. Select the corresponding button for operation according to image acquisition and construction requirements.
[0045] Assembly guidance animation 6-2 includes: component assembly sequence, connection method, fixing method, safety precautions and quality requirements, etc. Visualize building information, reduce manual intervention and provide real-time construction guidance, and reduce safety risks on the construction site.
[0046] The spatial coordinate anchoring 6-3 uses the coordinate inverse transformation to process the world coordinate information and convert it into the spatial coordinate system information of the HoloLens2 device 1. Then it is transmitted to the HoloLens2 device 1 to complete the display of the virtual component information on the device. Using the spatial anchor point, the MR model is approximately proportionally superimposed on the assembly position of the actual beam and column components to achieve spatial mapping and virtual-real superposition.
[0047] Although the present application provides an embodiment or flowchart of a prefabricated beam-column component detection system and a construction control method based on mixed reality, as well as methods and steps of the embodiment or flowchart, more or fewer operation steps may be included based on conventional or non-creative means. The steps in the embodiment or flowchart are only one implementation method and do not represent all implementation methods.
[0048] Each embodiment in this specification is described in a related manner, and each embodiment focuses on its unique features. For the same or similar parts, reference can be made to each other. The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the scope of protection of the present invention.
Claims
1. The prefabricated beam and column component detection system based on mixed reality is characterized by: The system comprises a mixed reality Hololens 2 device (1), an image acquisition system (2), a point marking system (3), a computer PC (4), an image processing system (5), and a virtual-reality superposition system (6); the image acquisition system (2) is composed of an RGB camera (2-1), a sensor unit (2-2), an internal storage unit (2-3), and a data transmission module (2-4), and each module is centrally mounted on the mixed reality Hololens 2 device (1); the point marking system (3) is composed of a non-marking module (3-1) and a QR code marking module (3-2); the image processing system (5) is composed of a non-marking module (3-1) and a QR code marking module (3-2); The system (5) is composed of a feature recognition module (5-1), a coordinate conversion module (5-2), an image reconstruction module (5-3), a data storage module (5-4) and a data comparison module (5-5); the virtual-reality superposition system (6) is composed of a user interface UI (6-1), an assembly guide animation (6-2) and a spatial coordinate anchor (6-3); the image acquisition system (2) acquires images by using a HoloLens2 device (1) with a fusion scale factor, equipped with a high-quality RGB camera (2-1) and a sensor unit (2-2), to capture images in the construction environment. The required photo or video is taken; wherein the scaling factor is used to increase the range of the collected image to avoid the unusability of the collected information; the built-in depth sensor and IMU sensor are used to collect pixel depth values and attitude angles; after the shooting is completed, the photo or video is saved to the internal storage space (2-3) of the HoloLens2 device (1) to generate a corresponding data file; the data transmission module (2-4) uses the TCP / IP network transmission protocol for real-time communication to transmit the collected data to the computer PC (4); when collecting data, the collected data is collected from multiple angles, different directions, complex backgrounds, and ambient lighting conditions. Manually select measuring points to increase the number of data sets; during the acquisition process, the researcher mode of the HoloLens2 device (1) is enabled to access the data of each sensor; the image processing system (5) includes a feature recognition module (5-1), a coordinate conversion module (5-2), an image reconstruction module (5-3), a data storage module (5-4) and a data comparison module (5-5); the feature recognition module (5-1) contains a recognition algorithm to process image information in real time, detect and recognize image feature points or marking points, and extract component point information; for feature point marking, a deep learning Faster R-CNN (FasterRegion-based Convolutional Neural Network) detection algorithm is used to extract important feature information; for QR code marking points, the algorithm is used in conjunction with the ZXing library to extract the data content of the QR code in the image; the coordinate conversion module (5-2) reads the data stream transmitted by the HoloLens2 device (1), and converts the original coordinate information into world coordinate information on the computer PC (4);By calculating the data of the sensor unit (2-2) installed on the HoloLens2 device (1), the original coordinates are mapped into the world coordinate system; the image reconstruction module (5-3) is used to import the prefabricated building construction drawings into the Revit software, and the drawings are adjusted and optimized according to actual requirements to build a visual BIM model; the component templates and construction process related information of the prefabricated building are added to the BIM model; the above information is then uploaded to the data storage module (5-4), the relevant data is imported and the appropriate data structure and table design is performed, and the data comparison module (5-5) is called to compare the database component information to identify the component type; the virtual-reality superposition system (6) uses the C# programming language and Unity to create the prefabricated structure component information panel and create the construction assembly guide animation; based on the mixed reality toolkit (MRTK) and OpenXR, an interactive user interface UI (6-1) is created, operation buttons are added, and a construction assembly guide animation (6-2) is designed; the cloud data storage module (5-4) is called in real time, and the MR model created by Unity is packaged and sent to VisualStudio, Then deploy it to the application of the mixed reality HoloLens2 device (1); at the same time, use the reverse coordinate transformation algorithm to process the world coordinate information and convert it into the spatial coordinate system information of the HoloLens2 device (1); then transmit it to the HoloLens2 device (1) to complete the display of the virtual component information on the device; through the spatial coordinate anchoring (6-3), the MR model is approximately proportionally superimposed on the assembly position of the actual beam and column components to achieve spatial mapping, virtual and real superposition, and use the guide animation to guide the engineering assembly construction; the user interface UI (6-1) is made using Unity and C# programming languages, and the capture, upload, and calibration operation buttons are added; according to the image acquisition and construction requirements, the corresponding button is selected for operation; the assembly guide animation (6-2) includes the component assembly sequence, connection method, fixing method, safety precautions and quality requirements; the building information is visualized, reducing manual intervention and providing real-time construction guidance, reducing the safety risks of the construction site; the spatial coordinate anchoring (6-3) uses the coordinate reverse transformation to process the world coordinate information and convert it into the spatial coordinate system information of the HoloLens2 device (1). ; 2. The assembled beam-column component detection system based on mixed reality according to claim 1, characterized in that: The point marking system (3) comprises a non-marking module (3-1) and a two-dimensional code marking module (3-2), and can manually select measuring points for information collection; the non-marking module (3-1) is suitable for data collection under general circumstances, and can manually select characteristic points with better characteristic values of the measuring points; the two-dimensional code marking module (3-2) uses a two-dimensional code containing additional information of the assembled structural component as a marking point.
3. The construction control method of the assembled beam-column component detection system based on mixed reality according to claim 1, characterized in that: The construction control method is based on a component detection system and is used to indicate and guide construction on site. The specific steps are: using an image acquisition system (2) integrated in a mixed reality Hololens2 device (1) to acquire working condition images of the site; at the same time, manually selecting measurement points to acquire component information according to a marking module of a point marking system (3); using a data transmission TCP / IP network transmission protocol for real-time communication, the acquired data is transmitted to a computer PC (4); Secondly, the recognition algorithm of the image processing system (5) is used to process the image information in real time, detect and recognize the image feature point module or the marker point module, and simultaneously reconstruct the three-dimensional visual BIM model of the prefabricated building, extract the image information and the corresponding BIM template information, upload them to the cloud database for storage, and compare the component data to identify the component type; Finally, the virtual-reality superposition system (6) is used to produce the prefabricated structural component information UI, which is used to display the component comparison results and produce the construction assembly guidance animation; The comparison results of the cloud database are retrieved in real time, packaged and sent to the mixed reality HoloLens2 device (1), and the virtual model is superimposed on the real scene to achieve spatial mapping, virtual-real superposition, and guide engineering assembly construction.
Citation Information
Patent Citations
Augmented reality-based construction progress visual management system and method
CN106897817A
Virtual technology-based construction method and system
CN108053486A
Virtual content control method and device, terminal equipment and storage medium
CN111766936A
Medical auxiliary three-dimensional model positioning matching method and system, equipment and medium
CN113842227A
Railway signal indoor equipment maintenance system based on augmented reality
CN113989462A
