Component-level construction progress detection method and system based on panoramic vision and medium

Abstract

The invention discloses a component-level construction progress detection method and system based on panoramic vision and a medium, and the method comprises the steps: carrying out the visual inertia three-dimensional reconstruction of initial panoramic image data, and obtaining a three-dimensional dense point cloud map; based on the three-dimensional dense point cloud map, performing global path planning and local path dynamic adjustment to obtain a target inspection path; collecting a target panoramic image according to the target inspection path, and processing the target panoramic image by using a semantic segmentation algorithm to obtain a semantic segmentation result; the semantic segmentation result is mapped to the three-dimensional dense point cloud map, a target three-dimensional dense point cloud map with component semantics is obtained, the construction progress is obtained based on the target three-dimensional dense point cloud map and the BIM model, and space alignment of the BIM model and the target three-dimensional dense point cloud map is completed. According to the method, the intelligent detection of the construction progress is realized by fusing the three-dimensional reconstruction, path planning and semantic segmentation of multiple sensors and combining the spatial alignment BIM model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of construction progress detection, and in particular to a component-level construction progress detection method and system based on panoramic vision and a medium. BACKGROUND

[0002] In the field of building engineering management, with the continuous expansion of building scale and the increasing complexity of construction, how to realize real-time and accurate monitoring of construction progress has become the key to ensuring timely delivery of projects, controlling costs and ensuring engineering quality. Traditional construction progress detection technologies, such as methods based on sensor identification, track progress by pasting two-dimensional codes or RFID tags on the surface of components, although they provide a direct means of progress monitoring, but their low efficiency of manual deployment, easy damage of tags and data loss caused by signal interference in complex environments make it difficult to meet the detection needs of large construction sites with high frequency and high precision. At the same time, although the technology based on point cloud three-dimensional reconstruction can construct detailed three-dimensional models for progress analysis, it is difficult to collect data in complex indoor environments and time-consuming to process, which cannot meet the requirements of real-time updating and dynamic monitoring of construction progress. In addition, the traditional visual analysis method using ordinary cameras for image acquisition has the problems of limited monitoring range and difficulty in realizing component-level fine detection when facing complex building space structures, and the recognition accuracy for different construction stages and different types of components is low, which cannot meet the needs of modern building engineering fine management.

[0003] Therefore, the prior art still needs to be improved and improved. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a component-level construction progress detection method, system and medium based on panoramic vision to solve the problems of low efficiency of manual deployment, easy damage of tags, poor adaptability in complex environments and low recognition accuracy in traditional construction progress detection technologies.

[0005] In order to solve the above technical problems, the technical solutions adopted by the present application are as follows:

[0006] In a first aspect, the present application provides a component-level construction progress detection method based on panoramic vision, wherein the method comprises:

[0007] performing visual-inertial three-dimensional reconstruction on the initial panoramic image data to obtain a three-dimensional dense point cloud map;

[0008] based on the three-dimensional dense point cloud map, performing global path planning and local path dynamic adjustment to obtain a target inspection path;

[0009] According to the target inspection path, a target panoramic image is collected, and a semantic segmentation algorithm is used to process the target panoramic image to obtain a semantic segmentation result;

[0010] The semantic segmentation result is mapped to the three-dimensional dense point cloud map to obtain a target three-dimensional dense point cloud map with component semantics, and based on the target three-dimensional dense point cloud map and a BIM model, a construction progress is obtained, wherein the BIM model is spatially aligned with the target three-dimensional dense point cloud map.

[0011] In an implementation manner, the visual-inertial three-dimensional reconstruction of the initial panoramic image data to obtain the three-dimensional dense point cloud map comprises:

[0012] The initial panoramic image data is processed by using a feature extraction algorithm, a feature point matching algorithm, an essential matrix estimation algorithm, and an epipolar geometry constraint to obtain an initial sparse point cloud map;

[0013] The camera pose data and the motion attitude data in the initial sparse point cloud map are tightly coupled and optimized to obtain a high-precision sparse point cloud map;

[0014] The high-precision sparse point cloud map is processed by using a multi-view stereo algorithm to generate point cloud data, and based on the point cloud data, filtering processing and a moving least squares surface reconstruction are performed to obtain a three-dimensional dense point cloud map.

[0015] In an implementation manner, the target inspection path is obtained by performing global path planning and local path dynamic adjustment based on the three-dimensional dense point cloud map, comprising:

[0016] The three-dimensional dense point cloud map is subjected to target node recognition, and based on the target node, global path planning is performed to obtain an initial inspection path;

[0017] The initial inspection path is subjected to local path dynamic adjustment to obtain a target inspection path.

[0018] In an implementation manner, the target node recognition is performed on the three-dimensional dense point cloud map, and based on the target node, the global path planning is performed to obtain the initial inspection path, comprising:

[0019] The three-dimensional dense point cloud map is subjected to region division, and based on the divided regions, target nodes are obtained;

[0020] A field of view coverage model is constructed for the target nodes by using panoramic camera parameters, and based on the field of view coverage model, coverage efficiency of the target nodes on undetected regions is calculated;

[0021] Based on the coverage efficiency and path length, global path planning is performed to obtain an initial inspection path.

[0022] In an implementation manner, the local path dynamic adjustment on the initial inspection path to obtain a target inspection path comprises:

[0023] If a dynamic obstacle is detected, an obstacle detection algorithm based on an optical flow method is used to determine an obstacle position and a motion trend;

[0024] Based on the obstacle position and the motion trend, an activity area of the dynamic obstacle is determined;

[0025] If the activity area is within the range of the initial inspection path, a local safety path is re-planned based on a remaining uncovered area with a current position of a collection device as a new starting point to obtain the target inspection path.

[0026] In an implementation manner, the target panoramic image is processed by using a semantic segmentation algorithm to obtain a semantic segmentation result, wherein the semantic segmentation algorithm is an improved U-Net semantic segmentation algorithm, and the improved U-Net semantic segmentation algorithm comprises:

[0027] A multi-scale feature fusion module is introduced in an encoder stage of the U-Net semantic segmentation algorithm, multi-scale feature extraction is performed on the target panoramic image, and global context information is fused to obtain enhanced high-level semantic features;

[0028] A cross-attention mechanism is introduced in a decoder stage of the U-Net semantic segmentation algorithm, attention weights are calculated for the high-level semantic features and output features of a current level of the decoder, and selective fusion is performed on the features based on the attention weights to obtain a semantic segmentation result.

[0029] In an implementation manner, the semantic segmentation result is mapped to the three-dimensional dense point cloud map to obtain a target three-dimensional dense point cloud map with member semantics, and a construction progress is obtained based on the target three-dimensional dense point cloud map and a BIM model, wherein the BIM model and the target three-dimensional dense point cloud map are spatially aligned, and the method comprises:

[0030] The semantic segmentation result is mapped to the three-dimensional dense point cloud map by using a back projection algorithm and a point cloud fusion technology to obtain a target three-dimensional dense point cloud map with member semantics;

[0031] Based on the target three-dimensional dense point cloud map and a BIM model, an amount of construction and a total amount of design are obtained;

[0032] Based on the amount of construction and the total amount of design, a construction progress is obtained.

[0033] In a second aspect, an embodiment of the present application further provides a member-level construction progress detection system based on panoramic vision, wherein the system comprises:

[0034] a three-dimensional dense point cloud map acquisition module configured to perform visual-inertial three-dimensional reconstruction on the initial panoramic image data to obtain a three-dimensional dense point cloud map;

[0035] a target inspection path acquisition module configured to perform global path planning and local path dynamic adjustment based on the three-dimensional dense point cloud map to obtain a target inspection path;

[0036] a semantic segmentation result acquisition module configured to collect a target panoramic image according to the target inspection path and process the target panoramic image by using a semantic segmentation algorithm to obtain a semantic segmentation result;

[0037] a construction progress acquisition module configured to map the semantic segmentation result to the three-dimensional dense point cloud map to obtain a target three-dimensional dense point cloud map with component semantics, and obtain construction progress based on the target three-dimensional dense point cloud map and a BIM model, wherein the BIM model is spatially aligned with the target three-dimensional dense point cloud map.

[0038] In a third aspect, an embodiment of the present application further provides a terminal, wherein the terminal comprises a memory, a processor, and a panoramic vision-based component-level construction progress detection program stored in the memory and executable on the processor; when the panoramic vision-based component-level construction progress detection program is executed by the processor, the steps of the panoramic vision-based component-level construction progress detection method in any of the above solutions are implemented.

[0039] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein a panoramic vision-based component-level construction progress detection program is stored on the computer-readable storage medium; when the panoramic vision-based component-level construction progress detection program is executed by a processor, the steps of the panoramic vision-based component-level construction progress detection method in any of the above solutions are implemented.

[0040] Beneficial effects: The present application provides a component-level construction progress detection method based on panoramic vision, compared with the prior art, the present application firstly utilizes a panoramic camera to collect initial panoramic image data, performs visual inertial three-dimensional reconstruction based on the initial panoramic image data, obtains a three-dimensional dense point cloud map, and in this step, combines inertial measurement unit (IMU) data to overcome the problem that a single sensor is easily disturbed in a complex construction site, thereby realizing high-precision and robust spatial modeling and positioning; on this basis, further through global path planning and local path dynamic adjustment, comprehensively considering the overall structure of the environment and real-time change factors, an optimal and highly adaptive target inspection path is generated, which improves the inspection efficiency and the autonomous decision-making ability of the system. Then, according to the target inspection path, a target panoramic image is collected, and a semantic segmentation algorithm is used to process the target panoramic image to obtain a semantic segmentation result, realizing pixel-level recognition and classification of various components, equipment and obstacles in the construction site, and providing accurate and structured semantic information support for subsequent spatial analysis. Subsequently, the semantic segmentation result is mapped to the three-dimensional dense point cloud map to obtain a target three-dimensional dense point cloud map with component semantics, which not only retains the real geometric structure of the construction site, but also enhances the information expression ability of the map, laying a foundation for the digital presentation of the construction state. Next, based on the target three-dimensional dense point cloud map and the BIM model, the construction progress is obtained, wherein the BIM model and the target three-dimensional dense point cloud map complete spatial alignment, this step can automatically identify the difference between the actual construction progress and the design model, and then realize accurate evaluation and visual display of the construction progress. In summary, the present application realizes intelligent detection and dynamic update of construction progress by fusing multi-sensor three-dimensional reconstruction, path planning and semantic segmentation, and combining a spatially aligned BIM model, significantly improving the efficiency, accuracy and intelligent level of construction management, and having good engineering application value and popularization prospect. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 The flowchart of the specific implementation mode of the component-level construction progress detection method based on panoramic vision provided by the embodiment of the present application is provided.

[0042] Figure 2 The flowchart of obtaining a three-dimensional dense point cloud map in the component-level construction progress detection method based on panoramic vision provided by the embodiment of the present application is provided.

[0043] Figure 3 The flowchart of obtaining a target inspection path in the component-level construction progress detection method based on panoramic vision provided by the embodiment of the present application is provided.

[0044] Figure 4 The flowchart of obtaining a semantic segmentation result in the component-level construction progress detection method based on panoramic vision provided by the embodiment of the present application is provided.

[0045] Figure 5 The flow chart of obtaining construction progress in the construction progress detection method at the component level based on panoramic vision provided by the embodiment of the present application.

[0046] Figure 6 The principle block diagram of the construction progress detection system at the component level based on panoramic vision provided by the embodiment of the present application.

[0047] Figure 7 The principle block diagram of the internal structure of the terminal provided by the embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical scheme and effect of the present application more clear and definite, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0049] In the field of building engineering management, with the continuous expansion of building scale and the increasing complexity of construction, how to realize real-time and accurate monitoring of construction progress has become the key to ensuring timely delivery of projects, controlling costs and ensuring engineering quality. Traditional construction progress detection technologies, such as methods based on sensor identification, track progress by pasting two-dimensional codes or RFID tags on the surface of components, although providing a direct means of progress monitoring, but its low efficiency of manual deployment, label easy to fall off and damage, and data loss caused by signal interference in complex environment, etc. It is difficult to adapt to the detection needs of large construction site with high frequency and high precision. At the same time, although the technology based on point cloud three-dimensional reconstruction can construct detailed three-dimensional model for progress analysis, but in the complex indoor shielding environment, data acquisition is difficult and processing is time-consuming, which cannot meet the requirements of real-time update and dynamic monitoring of construction progress. In addition, the traditional visual analysis method using ordinary camera for image acquisition has the problems of limited monitoring range and difficulty in realizing component-level fine detection when facing complex building space structure, and the recognition accuracy of different construction stages and different types of components is low, which is difficult to meet the needs of modern building engineering fine management.

[0050] To solve the above problems, the embodiment provides a component-level construction progress detection method based on panoramic vision, in specific implementation, the embodiment first collects initial panoramic image data by using a panoramic camera, performs visual inertial three-dimensional reconstruction based on the initial panoramic image data, obtains a three-dimensional dense point cloud map, and in this step, inertial measurement unit (IMU) data is combined to overcome the problem that a single sensor is easily disturbed in a complex construction site, so that high-precision and robust spatial modeling and positioning are realized; on this basis, an optimal and highly adaptive target inspection path is generated by further global path planning and local path dynamic adjustment, considering the overall structure of the environment and real-time change factors, so that the inspection efficiency and the autonomous decision-making ability of the system are improved. Then, target panoramic images are collected according to the target inspection path, and the target panoramic images are processed by using a semantic segmentation algorithm to obtain a semantic segmentation result, so that pixel-level recognition and classification of various components, equipment and obstacles in the construction site are realized, and accurate and structured semantic information support is provided for subsequent spatial analysis. Subsequently, the semantic segmentation result is mapped to the three-dimensional dense point cloud map to obtain a target three-dimensional dense point cloud map with component semantics, which not only retains the real geometric structure of the construction site, but also enhances the information expression ability of the map, and lays a foundation for digital presentation of the construction state. Next, based on the target three-dimensional dense point cloud map and a BIM model, construction progress is obtained, wherein the BIM model and the target three-dimensional dense point cloud map are spatially aligned, this step can automatically identify the difference between the actual construction progress and the design model, and then realize accurate evaluation and visual display of the construction progress. In summary, the application realizes intelligent detection and dynamic update of construction progress by fusing three-dimensional reconstruction of multiple sensors, path planning and semantic segmentation, and combining a spatially aligned BIM model, which significantly improves the efficiency, accuracy and intelligent level of construction management, and has good engineering application value and promotion prospect.

[0051] The component-level construction progress detection method based on panoramic vision provided by the embodiment can be applied to an intelligent terminal, such as a smart phone. Figure 1 As shown in the embodiment, the method comprises the following steps:

[0052] Step S100, performing visual inertial three-dimensional reconstruction on the initial panoramic image data to obtain a three-dimensional dense point cloud map.

[0053] In this embodiment, first, the initial panoramic image data collected by the panoramic camera is used to extract ORB and other feature points, and a sparse SLAM map is initialized through feature matching and epipolar geometry constraints to obtain the preliminary scene structure and camera pose; at the same time, high-frequency motion attitude data provided by an inertial measurement unit (IMU) is fused, a sliding window-based tightly coupled optimization algorithm is used to construct a joint optimization objective function containing visual re-projection error and IMU pre-integral error, and a nonlinear optimization method is used to realize joint refinement of the camera pose and the map point, thereby significantly improving the positioning accuracy and operation robustness of the system in complex construction environments such as light changes and texture missing, and forming a high-precision sparse point cloud map. On this basis, the optimized key frame pose is used as a guide, a multi-view stereo (MVS) algorithm is applied for dense reconstruction, a dense point cloud is generated through block matching and depth map fusion, and a moving least squares (MLS) surface reconstruction is combined with filtering processing to finally obtain a three-dimensional dense point cloud map with a spatial accuracy of better than 5 cm. This three-dimensional reconstruction method not only fully utilizes the complementary advantages of visual and inertial sensors, effectively suppresses the cumulative error and drift problem of a single sensor, but also solves the inherent scale uncertainty problem of a monocular SLAM system by introducing a reference object with accurate dimensions for scale calibration, thereby ensuring that the reconstructed map has a real physical scale. The generated high-fidelity three-dimensional dense point cloud map completely and accurately restores the geometric structure and spatial relationship of the construction site, and provides a solid and reliable three-dimensional spatial reference and data support for subsequent intelligent path planning, component semantic segmentation, BIM model alignment, and construction progress quantitative analysis, thereby significantly improving the automation, refinement, and intelligence level of the entire intelligent inspection system.

[0054] Specifically, the step S100 comprises the following steps:

[0055] Step S101, processing the initial panoramic image data by using a feature extraction algorithm, a feature point matching algorithm, an essential matrix estimation algorithm, and an epipolar geometry constraint to obtain an initial sparse point cloud map.

[0056] Step S102, tightly coupling and optimizing the camera pose data and the motion attitude data in the initial sparse point cloud map to obtain a high-precision sparse point cloud map.

[0057] Step S103, processing the high-precision sparse point cloud map by using a multi-view stereo algorithm to generate point cloud data, and performing filtering processing and moving least squares surface reconstruction based on the point cloud data to obtain a three-dimensional dense point cloud map.

[0058] In an implementation manner, as Figure 2As shown, first, a mobile inspection device integrating a panoramic camera (resolution ≥ 8K) and an inertial measurement unit (IMU, sampling frequency ≥ 200Hz) is deployed at the construction site for data collection. The panoramic camera acquires 360° panoramic images at fixed time intervals (e.g., 5 seconds / frame), which can comprehensively capture the visual texture information of the construction area, providing rich visual features for subsequent mapping. At the same time, the IMU collects high-frequency motion attitude data in real time, which is used to support high-precision calculation of the device pose, thereby improving the overall positioning accuracy and robustness. This multi-sensor fusion data collection method not only improves the environmental perception ability, but also provides high-quality input data for subsequent SLAM (Simultaneous Localization And Mapping) mapping. Subsequently, multiple standard reference objects with precise dimensions (e.g., square calibration stickers with a side length of 0.3 meters) are arranged at the construction site to ensure that they appear in images from different angles. Feature matching algorithms are used to identify the positions of these reference objects in each frame of image, and combined with camera calibration parameters and triangulation principles, the three-dimensional coordinates of the reference objects in the SLAM map are calculated, and then the map scale factor is derived. This process effectively solves the problem of monocular SLAM not being able to recover the true scale, laying the foundation for constructing a three-dimensional map with real spatial scale. Next, feature extraction algorithms (e.g., Oriented FAST and Rotated BRIEF, ORB) are used to extract feature points from the image sequence collected by the panoramic camera, and feature point matching algorithms (e.g., FLANN fast approximate nearest neighbor search algorithm) are used to complete feature matching between adjacent frames. Based on the matching results, essential matrix estimation algorithms (e.g., five-point method) are used to solve the essential matrix, and the relative pose of the camera is recovered based on epipolar geometry constraints to initialize the sparse SLAM point cloud local map, obtaining the initial sparse point cloud map. This step aims to build an initial sparse map structure, including key frames and their corresponding feature point three-dimensional coordinates, providing a geometric framework for subsequent optimization. On this basis, the camera pose data in the initial sparse point cloud map and the motion attitude data collected by the IMU are tightly coupled and optimized to obtain a high-precision sparse point cloud map. Specifically, a sliding window-based tightly coupled optimization algorithm is used to construct a joint optimization function containing visual re-projection errors and IMU inertial errors: where μ i is the feature point observation coordinate, π is the projection function, K is the camera intrinsic parameter, T i is the camera pose transformation matrix, c i is the feature point three-dimensional coordinate; ω j , α k are the angular velocity and acceleration measured by the IMU, For the predicted value, p is the Huber kernel function. By optimizing this function, accurate estimation of the global pose is achieved, which significantly improves the stability and accuracy of the SLAM system, constructs a high-precision sparse SLAM map, and obtains a high-precision sparse point cloud map. Further, the high-precision sparse point cloud map is processed by using a multi-view stereo algorithm to generate point cloud data, and based on the point cloud data, filtering processing and moving least squares surface reconstruction are performed to obtain a three-dimensional dense SLAM map, i.e., a three-dimensional dense point cloud map. Specifically, based on the high-precision sparse SLAM map, a Multi-View Stereo (MVS) algorithm is used for dense mapping. The key frames in the high-precision sparse SLAM map are used as a reference to process each panoramic image in blocks. The PatchMatch algorithm is used to find the best matching block in the adjacent key frames. Combining the multi-view geometric constraint and the photometric consistency constraint, the depth information is estimated pixel by pixel to generate dense point cloud data. Then, the point cloud data is filtered to remove outliers, and the Moving Least Squares (MLS) method is used for surface reconstruction to finally obtain a high-precision three-dimensional dense SLAM map with a spatial accuracy better than 5 cm. The three-dimensional dense SLAM map not only preserves the real geometric structure of the construction scene, but also provides a high-fidelity spatial model for path planning and component identification.

[0059] In summary, the embodiment significantly improves the accuracy and robustness of the SLAM system through tight coupling optimization of high-resolution panoramic cameras and IMUs. The introduction of a standard reference object enables real-scale map recovery, enhancing the application value of the map. Based on multi-view stereo reconstruction technology, a high-precision dense map is generated, providing a reliable spatial foundation for path planning and component identification. Overall, the embodiment realizes high-precision, full-scale three-dimensional dense mapping based on tight coupling of panoramic vision and inertial measurement units, laying a precise spatial perception foundation for subsequent intelligent inspection path planning and component-level semantic analysis.

[0060] Step S200, based on the three-dimensional dense point cloud map, global path planning and local path dynamic adjustment are performed to obtain a target inspection path.

[0061] In this embodiment, global path planning is first performed based on the 3D dense point cloud map. The reconstructed initial trajectory is then nonlinearly optimized to further eliminate accumulated errors, improving the smoothness and consistency of the overall path. This results in higher navigation stability and path predictability for the robot or mobile inspection device during task execution. Next, local path dynamic adjustments are made based on the global path. By incorporating real-time perceived environmental changes or obstacle information, the path is corrected and adaptively adjusted online, ensuring that the inspection path meets task requirements while maintaining good safety and flexibility. In summary, this method, through a phased optimization strategy, improves the accuracy of 3D reconstruction and the reliability of path planning, achieving efficient adaptability of the inspection path in dynamic environments. This provides solid technical support for the intelligent and practical application of autonomous inspection systems.

[0062] Specifically, step S200 includes the following steps:

[0063] Step S201: Identify target nodes in the three-dimensional dense point cloud map and perform global path planning based on the target nodes to obtain an initial inspection path;

[0064] Step S202: Perform local path dynamic adjustment on the initial inspection path to obtain the target inspection path.

[0065] In one implementation, such as Figure 3 As shown, firstly, the 3D dense point cloud map (3D dense SLAM map) is divided into regions, and target nodes are obtained based on these regions. Specifically, the map is divided into sub-regions according to spatial functional zones (such as floors, work areas, etc.), and the starting position of the mobile inspection equipment is used as the root node. The geometric center of the uncovered area or key construction parts (such as beam-column joints under construction, wall corners) in each sub-region are used as target nodes. This method helps improve the targeting and efficiency of path planning, ensuring that key areas are covered by inspection. Next, a field-of-view coverage model is constructed for the target nodes using panoramic camera parameters, and the coverage efficiency of the target nodes for undetected areas is calculated based on the field-of-view coverage model. To evaluate the coverage efficiency of each target node, a virtual sphere field-of-view coverage model is constructed based on the panoramic camera's field-of-view parameters (360° horizontally, 180° vertically), with each target node as the center and a maximum detection distance (e.g., 10 meters) set. The coverage area of ​​the field-of-view model for undetected areas of each target node is calculated, and its coverage efficiency is evaluated. With the target node as the center and radius R as the maximum detection distance, the specific calculation of coverage efficiency is as follows: Let the total area of ​​the field of view model be A. total The area covered by the undetected area is A. covered The uncovered area is A. uncovered For a spherical model, the formula for the total area covered by its field of view is: Atotal = 4πR 2 wherein the coverage area A covered After identifying the component regions by the image semantic segmentation algorithm, the pixel area is counted and converted into the actual area. The coverage area of the visual field model of each target node on the undetected region is calculated to evaluate the coverage efficiency λ, and the formula is as follows: Subsequently, global path planning is performed based on the coverage efficiency and path length to obtain an initial inspection path. The path length is the total path distance from the starting point to the target node or between one target node and another target node. Specifically, in combination with the A* or Dijkstra algorithm, the optimization goal of maximizing the coverage efficiency and minimizing the path length is used for global path planning. The target node with high coverage efficiency and short path is preferentially selected as an intermediate path point, the optimal collision-free path is calculated, and the initial inspection path is obtained. During path execution, the system updates the undetected region state in real time to avoid repeated coverage and improve inspection efficiency. Then, the initial inspection path is locally adjusted dynamically to obtain a target inspection path. Specifically, if a dynamic obstacle is detected, an obstacle detection algorithm based on the optical flow method is used to determine the obstacle position and motion trend. Based on the obstacle position and the motion trend, the activity area of the dynamic obstacle is determined. If the activity area is within the range of the initial inspection path, the current position of the collection device is taken as a new starting point, and a local safe path is re-planned based on the remaining uncovered area to obtain the target inspection path. At the same time, the camera field of view coverage efficiency of the target inspection path is ensured to be not less than 80% of that of the initial inspection path, ensuring the continuity and safety of the inspection task. Finally, the planned target inspection path is visually presented on the three-dimensional map in different color lines. The user can click any path point by mouse, and the system can display the panoramic image, component segmentation result and construction progress quantitative data at the corresponding time according to the pose information of the mobile inspection device corresponding to the point. This interactive visualization module not only improves the user's understanding and control ability of the inspection path, but also provides intuitive and efficient auxiliary decision support for construction management.

[0066] In summary, the embodiment divides the space region in the three-dimensional dense point cloud map and determines the key target node, constructs a virtual coverage model combining the omnidirectional field of view characteristics of the panoramic camera, quantitatively evaluates the detection coverage efficiency of each node, and performs global path planning with the dual objectives of maximizing the coverage efficiency and minimizing the path length, to generate an efficient and collision-free initial inspection path. Then, in the execution process, the light flow method is fused to detect dynamic obstacles in real time, the potential interference is judged based on the position and motion trend of the dynamic obstacles, and the local path is dynamically re-planned to avoid the activity area of the dynamic obstacles, so as to ensure that the camera field of view coverage efficiency is not less than 80% of the original path while ensuring the safety and continuity of the inspection. Finally, the optimized target inspection path is visualized on the three-dimensional map, and the user can interactively click on any position of the path to call the corresponding panoramic image, component segmentation result and progress data, realizing intelligent and adaptive planning and dynamic adjustment of the construction inspection path, significantly improving the comprehensiveness of the inspection coverage, the robustness of the path execution and the convenience of human-computer interaction, and providing strong technical support for efficient, accurate and safe inspection of the construction site.

[0067] Step S300, according to the target inspection path, a target panoramic image is collected, and a semantic segmentation algorithm is used to process the target panoramic image to obtain a semantic segmentation result.

[0068] In the embodiment, first, according to the target inspection path, a target panoramic image is collected by using a panoramic camera. This step ensures the systematicness and comprehensiveness of image acquisition through a reasonable target inspection path, thereby effectively covering the key parts of the detected area and improving the accuracy and reliability of subsequent analysis. Then, the collected target panoramic image is processed by using an improved U-Net semantic segmentation algorithm. This process can accurately classify each pixel in the image according to its object category, thereby generating a segmentation result with semantic information. The application of the semantic segmentation algorithm realizes the fine identification and differentiation of different objects in a complex scene. Overall, the embodiment combines a structured image acquisition process with advanced image understanding technology, significantly improves the intelligent level and automation capability of the inspection system, improves work efficiency, reduces subjective errors and cost consumption caused by human participation, and has good practicality and popularization value.

[0069] Specifically, the step S300 includes the following steps:

[0070] Step S301, according to the target inspection path, a target panoramic image is collected;

[0071] Step S302, a multi-scale feature fusion module is introduced in the encoder stage of the U-Net semantic segmentation algorithm, multi-scale feature extraction is performed on the target panoramic image, and global context information is fused to obtain enhanced high-level semantic features;

[0072] Step S303, introducing a cross-attention mechanism in the decoder stage of the U-Net semantic segmentation algorithm, calculating attention weights for the high-level semantic features and the output features of the current level of the decoder, and selectively fusing the features based on the attention weights to obtain a semantic segmentation result.

[0073] In an implementation manner, as shown in Figure 4 As shown in the figure, first, a target panoramic image is collected according to the target inspection path. During the movement of the mobile inspection device along the target inspection path, the panoramic camera continuously collects construction site images, thereby obtaining panoramic visual data with wide coverage and rich information. In order to improve the image quality and provide high-quality input for subsequent semantic segmentation, the collected target panoramic image is sequentially subjected to non-local mean denoising processing. This method can effectively suppress Gaussian noise and salt and pepper noise, retain image edges and details, and enhance the structural integrity of the image. Subsequently, the intrinsic parameters and distortion parameters of the panoramic camera are obtained based on Zhang Zhengyou calibration method, the geometric distortion correction of the target panoramic image is completed, the spatial accuracy of the target panoramic image is ensured, and the recognition error caused by lens distortion is avoided. On this basis, the adaptive histogram equalization algorithm is further used to enhance the contrast of the target panoramic image, the brightness distribution of the local region is dynamically adjusted, the visibility and texture detail performance of the target panoramic image are significantly improved, and thus a solid foundation is laid for subsequent image analysis tasks. In order to reduce the influence of polar distortion on the target panoramic image, the target panoramic image is converted from equidistant cylindrical projection to spherical coordinate system during the target panoramic image correction stage, and the formula is as follows: where (μ, v) is the pixel coordinate of the target panoramic image, Let W and H be the width and height of the target panoramic image, respectively, in spherical coordinates. This transformation process is based on the mapping relationship between pixel coordinates and spherical coordinates, and combined with the parameter settings of the image width and height, to achieve reasonable reprojection of the target panoramic image, thereby alleviating the deformation problem caused by traditional projection methods in polar regions and improving the overall spatial consistency and usability of the target panoramic image. Based on this, a U-Net segmentation algorithm improved by a multi-scale cross-attention network is proposed. This algorithm significantly improves the segmentation accuracy of components in complex construction scenarios by innovatively constructing an encoder-decoder architecture based on U-Net. Specifically, a multi-scale feature fusion module is introduced into the encoder stage of the U-Net semantic segmentation algorithm to extract multi-scale features from the target panoramic image and fuse global contextual information to obtain enhanced high-level semantic features. Specifically, the global contextual information of the target panoramic image is extracted using a pyramid pooling module (PPM) with pooling kernels of different sizes, capturing multi-level feature expressions from local details to global semantics, enhancing the model's ability to perceive targets at different scales. This is particularly suitable for recognizing building components of varying sizes and complex shapes in construction sites. The formula is as follows:

[0074] F pool =Concat(Pool1(F),Pool2(F),Pool3(F),…,Pool n (F)), where Pool i Let F represent the pooling operation at the i-th scale, and F be the input feature vector. Simultaneously, a cross-attention mechanism is introduced in the decoder stage of the U-Net semantic segmentation algorithm. Attention weights are calculated between the high-level semantic features and the output features of the current level of the decoder, and features are selectively fused based on these attention weights to obtain the semantic segmentation result. Specifically, a cross-attention mechanism is introduced in the decoder stage to construct a bidirectional attention bridge between the encoder and decoder, achieving feature enhancement through the following formula:

[0075] A enc→dec =σ(W q (F enc )·W k (F dec ) T ), F enhanced =A enc→dec ·W v (F dec )+F dec Among them, W q W k W v Let F be the learnable weight matrix, σ be the softmax function, and F be the weight matrix. enc F is the feature vector output by the encoder.dec The feature vector is the output of the decoder. This mechanism enables the decoder to selectively focus on key features in the encoder, enhancing the feature representation of small target components such as pipes and embedded parts, and improving the segmentation accuracy. This mechanism not only improves the feature interaction efficiency of the model, but also enhances the robustness to occlusion, background interference and other situations. In addition, in the design of the loss function, a multi-task hybrid loss function is used to balance the segmentation accuracy of different component types. The loss function formula is: The Dice loss can effectively deal with the class imbalance problem and improve the model's recognition ability for the less frequent class. The formula is: where X and Y are the predicted segmentation result and the true segmentation label respectively. The Focal loss focuses on difficult classification samples and controls the contribution difference between easy and difficult classification samples through a modulation factor, so that the model pays more attention to the boundary area that is difficult to distinguish. The formula is: where α t ∈[0,1], p t is the predicted probability of the model for positive samples, and τ is the modulation factor. The boundary loss further optimizes the segmentation accuracy of the component edge. The distance transformation method based on Euclidean distance is used to directly measure the geometric difference between the true and predicted boundaries, thereby improving the performance of the model in component contour extraction. The formula is: where and are the true and predicted component boundaries, respectively, composed of a series of pixel points, and d is the Euclidean distance.

[0076] In summary, the method proposed in this embodiment has significant overall advantages in complex construction environments. By combining high-quality image acquisition and multi-stage preprocessing procedures, the clarity, contrast and geometric accuracy of the input data are effectively improved. The target panoramic image projection conversion strategy alleviates the polar distortion problem and enhances the rationality of spatial expression. The improved U-Net segmentation network structure significantly improves the model's ability to recognize and segment various components, especially small target components, in complex backgrounds through multi-scale feature fusion and cross-attention mechanisms. At the same time, the design of the multi-task hybrid loss function further optimizes the model's segmentation performance in terms of class imbalance and edge details. The overall scheme realizes the whole process optimization from image acquisition to semantic understanding, providing efficient and accurate technical support for intelligent inspection and component recognition in construction sites.

[0077] Step S400, map the semantic segmentation result to the three-dimensional dense point cloud map to obtain a target three-dimensional dense point cloud map with component semantics, and obtain the construction progress based on the target three-dimensional dense point cloud map and the BIM model, wherein the BIM model and the target three-dimensional dense point cloud map are spatially aligned.

[0078] In the embodiment, first, the semantic segmentation result is mapped to a three-dimensional dense point cloud map, i.e., mapped to a three-dimensional dense SLAM map, so as to obtain a target three-dimensional dense SLAM map, i.e., a target three-dimensional dense point cloud map. The function of this step is to fuse the image-level semantic information (such as object category, region division, etc.) with the high-precision spatial structure constructed by SLAM, so that the three-dimensional map not only contains geometric information, but also has semantic understanding ability, thereby improving the interpretability and practicality of the map. On this basis, combined with the target three-dimensional dense point cloud map and the BIM model, construction progress information is further obtained. Since the BIM model and the target three-dimensional dense point cloud map have completed spatial alignment, accurate matching and comparative analysis between the two can be realized, ensuring that the information extracted from the actual construction site can be accurately located in the design model, effectively supporting the automatic identification and progress evaluation of the construction state. Overall, this method fully utilizes the real-time and robustness advantages of SLAM technology in environment modeling, and combines the design information integrity and semantic expression ability of the BIM model, realizes efficient and accurate acquisition of construction progress, and provides reliable data support and technical support for building project management.

[0079] Specifically, the step S300 comprises the following steps:

[0080] Step S401, mapping the semantic segmentation result to the three-dimensional dense point cloud map by a back projection algorithm and a point cloud fusion technology to obtain a target three-dimensional dense point cloud map with component semantics.

[0081] Step S402, obtaining the constructed quantity and the design total quantity based on the target three-dimensional dense point cloud map and the BIM model.

[0082] Step S403, obtaining the construction progress based on the constructed quantity and the design total quantity.

[0083] In an implementation manner, as Figure 5As shown, firstly, the semantic segmentation result is mapped to the 3D dense point cloud map, i.e., mapped to the 3D dense SLAM map, using a back-projection algorithm and point cloud fusion technology, to obtain a target 3D dense point cloud map with component semantics, i.e., a target 3D dense SLAM map with component semantics. This step aims to accurately restore the semantic information in the 2D image to 3D space, thereby achieving spatial positioning and identification of components at the construction site. The back-projection algorithm can back-project pixels into 3D space based on camera pose, while point cloud fusion technology helps improve the density and accuracy of the SLAM map, providing a high-quality data foundation for subsequent BIM model alignment. The advantage of this process is that it enhances the semantic expressive power of the SLAM map, enabling it to not only possess geometric structural information but also include high-level semantic information such as component categories, providing accurate spatial semantic data support for construction progress monitoring. Subsequently, the BIM model and the target 3D dense point cloud map (target 3D dense SLAM map) are spatially aligned. Specifically, by matching manually selected feature points, the coordinate transformation matrix T between the BIM model and the target 3D dense point cloud map is calculated. BIM2SLAM This method maps the semantic tags and center points of components in the target 3D dense point cloud map to the geometric parameters (such as center points) and construction sequence information of components in the BIM model, forming a four-dimensional data structure of "component ID and semantics - spatial coordinates - progress status - timestamp". This achieves accurate registration of two heterogeneous models in a unified coordinate system. The key function of this step is to establish spatial consistency between the design model (BIM) and the field measured data (SLAM), allowing direct comparison and analysis of the actual construction status with the design model. Its advantages include avoiding misjudgments and deviations caused by coordinate inconsistencies in traditional methods, improving the accuracy and reliability of construction progress assessment, achieving refined progress management at the component level, and supporting data analysis and visualization based on multiple dimensions such as component type and construction stage, greatly improving the granularity and flexibility of construction management. Based on this, the completed construction volume and the total design volume are obtained from the target 3D dense point cloud map and the BIM model, and the construction progress is calculated accordingly. Specifically, for planar components (such as walls), the completed area S is calculated based on the target 3D dense point cloud map and semantic segmentation results. completed The total design area S is obtained based on the BIM model. total Through the formula: Calculate the actual construction progress; for linear components (such as beams and columns), calculate the constructed length L. completed Compared with the total design length L in BIM total The formula used is: Calculate the actual construction progress; for complex structural components (such as stairs), decompose them into multiple sub-components, calculate the progress of each sub-component, and then take a weighted average to get the overall actual progress P ′ The role of this step is to adopt different progress calculation strategies for different component types, ensuring the scientificity and rationality of component progress evaluation. Its advantage is to balance the diversity of component morphology and the adaptability of calculation method, improving the comprehensiveness and accuracy of overall progress evaluation. Further, the planned start time t start and the planned completion time t end of the component are extracted from the BIM model, and the corresponding planned progress P is calculated based on the current time t current , the formula is as follows:

[0084] The role of this step is to build a time-based construction plan model, allowing actual progress to be dynamically compared and analyzed with the plan. Its advantage is to realize the time sequence management of construction progress, supporting real-time monitoring of the time arrangement and execution of construction tasks. The deviation ΔP = P ′ -P between actual progress and planned progress, when ΔP exceeds the set threshold (such as 10%), the system triggers the early warning mechanism, and notifies the management personnel through multiple channels such as SMS, APP push, PC pop-up window, etc. The role of this mechanism is to timely discover the construction lag risk and assist managers to take targeted measures for correction. Its advantage is to improve the response speed and decision-making efficiency of construction management, and reduce the risk of project delay. In addition, when the user clicks on any point on the SLAM path, the system quickly retrieves the corresponding panoramic image based on the pose information of the mobile inspection device corresponding to the point, and marks each type of component with different colors and lines based on the semantic segmentation result. At the same time, the component name, BIM design parameters, current actual progress, planned progress, and progress deviation are displayed in real time in the image pop-up window, supporting graphical display in the form of Gantt chart, progress bar, etc., and supporting users to view the construction progress changes at different stages in the form of time axis. The function of this function is to provide an intuitive and visual construction progress analysis interface to help managers quickly understand the on-site construction status and its deviation from the plan. Its advantage is to significantly improve the readability and operational convenience of construction progress analysis through multi-modal data fusion and interactive visualization, thereby assisting managers to conduct efficient and accurate construction progress analysis and decision-making.

[0085] Overall, the embodiment implementation realizes a high-precision and fine-grained construction progress intelligent monitoring system by integrating SLAM, BIM and semantic segmentation technology. Its core advantage lies in realizing the spatial alignment and information mapping of the construction site and the BIM design model, improving the automation and visualization level of construction progress management. Through the back projection and point cloud fusion technology, the semantic information is mapped to the three-dimensional space, enhancing the semantic expression ability of the SLAM map; based on the artificial selection of feature points to establish the coordinate transformation matrix, the spatial consistency of SLAM and BIM is ensured, providing a basis for component-level comparison. The constructed four-dimensional data structure supports dynamic tracking and management of construction status, improving the management granularity. For different component types, the corresponding progress calculation method is adopted, improving the accuracy of the evaluation results. At the same time, combined with the time information, the actual and planned progress is compared, and when the deviation exceeds the limit, multi-channel early warning is triggered, improving the response efficiency of construction risks. Finally, through panoramic images and visual display, the management personnel can intuitively master the construction progress and problem nodes. Overall, this method realizes the accurate perception, dynamic analysis and intelligent feedback of construction progress, providing effective support for digital management of engineering projects.

[0086] In summary, the embodiment first extracts ORB feature points from the initial panoramic image data collected by the panoramic camera, initializes the sparse SLAM map through feature matching and epipolar geometry constraints, and obtains the preliminary scene structure and camera pose; at the same time, the high-frequency motion attitude data provided by the inertial measurement unit (IMU) is fused, a sliding window-based tightly coupled optimization algorithm is adopted, a joint optimization objective function containing visual re-projection error and IMU pre-integral error is constructed, and the camera pose and map points are refined through nonlinear optimization method, which significantly improves the positioning accuracy and running robustness of the system in complex construction environments such as light change and texture loss, forming a high-precision sparse point cloud map. On this basis, guided by the optimized key frame pose, a multi-view stereo (MVS) algorithm is applied for dense reconstruction, a dense point cloud is generated through block matching and depth map fusion, and finally a three-dimensional dense point cloud map with a spatial accuracy better than 5 cm is obtained through filtering processing and moving least squares (MLS) surface reconstruction. This three-dimensional reconstruction method not only fully utilizes the complementary advantages of visual and inertial sensors, effectively suppresses the cumulative error and drift problem of a single sensor, but also solves the inherent scale uncertainty problem of monocular SLAM system by introducing a reference object with accurate scale for scale calibration, ensuring that the reconstructed map has real physical scale. The generated high-fidelity three-dimensional dense point cloud map completely and accurately restores the geometric structure and spatial relationship of the construction site, providing a solid and reliable three-dimensional spatial reference and data support for subsequent intelligent path planning, component semantic segmentation, BIM model alignment and construction progress quantitative analysis, significantly improving the automation, refinement and intelligence level of the entire intelligent inspection system.

[0087] As Figure 6 shown in the embodiment, the embodiment also provides a component-level construction progress detection system based on panoramic vision, which comprises a three-dimensional dense point cloud map acquisition module 10, a target inspection path acquisition module 20, a semantic segmentation result acquisition module 30 and a construction progress acquisition module 40. Specifically, the three-dimensional dense point cloud map acquisition module 10 is configured to perform visual-inertial three-dimensional reconstruction on initial panoramic image data to obtain a three-dimensional dense point cloud map. The target inspection path acquisition module 20 is configured to perform global path planning and local path dynamic adjustment based on the three-dimensional dense point cloud map to obtain a target inspection path. The semantic segmentation result acquisition module 30 is configured to collect a target panoramic image according to the target inspection path, and process the target panoramic image by using a semantic segmentation algorithm to obtain a semantic segmentation result, wherein the semantic segmentation algorithm is an improved U-Net semantic segmentation algorithm. The construction progress acquisition module 40 is configured to map the semantic segmentation result to the three-dimensional dense point cloud map to obtain a target three-dimensional dense point cloud map with component semantics, and obtain construction progress based on the target three-dimensional dense point cloud map and a BIM model, wherein the BIM model is spatially aligned with the target three-dimensional dense point cloud map.

[0088] In an implementation manner, the three-dimensional dense point cloud map acquisition module 10 comprises:

[0089] An initial sparse point cloud map acquisition unit is configured to process the initial panoramic image data by using a feature extraction algorithm, a feature point matching algorithm, an essential matrix estimation algorithm and epipolar geometry constraints to obtain an initial sparse point cloud map.

[0090] A high-precision sparse point cloud map acquisition unit is configured to perform tight coupling optimization on camera pose data and motion attitude data in the initial sparse point cloud map to obtain a high-precision sparse point cloud map.

[0091] A three-dimensional dense point cloud map acquisition unit is configured to process the high-precision sparse point cloud map by using a multi-view stereo algorithm to generate point cloud data, and perform filtering processing and mobile least square surface reconstruction based on the point cloud data to obtain a three-dimensional dense point cloud map.

[0092] In an implementation manner, the target inspection path acquisition module 20 comprises:

[0093] An initial inspection path acquisition unit is configured to perform target node identification on the three-dimensional dense point cloud map, and perform global path planning based on the target node to obtain an initial inspection path.

[0094] An initial inspection path acquisition unit is configured to perform local path dynamic adjustment on the initial inspection path to obtain a target inspection path.

[0095] In an implementation manner, the initial inspection path acquisition unit comprises:

[0096] A target node acquisition subunit is configured to perform regional division on the three-dimensional dense point cloud map and obtain target nodes based on the divided regions.

[0097] A coverage efficiency acquisition subunit is configured to construct a field of view coverage model for the target nodes by using panoramic camera parameters, and calculate coverage efficiency of the target nodes on undetected regions based on the field of view coverage model.

[0098] An initial inspection path acquisition subunit is configured to perform global path planning based on the coverage efficiency and path length to obtain an initial inspection path.

[0099] In an implementation manner, the target inspection path acquisition unit comprises:

[0100] A position and motion trend acquisition subunit is configured to, if a dynamic obstacle is detected, determine the position and motion trend of the obstacle by using an obstacle detection algorithm based on an optical flow method.

[0101] An active region acquisition subunit is configured to determine an active region of the dynamic obstacle based on the position of the obstacle and the motion trend.

[0102] A target inspection path acquisition subunit is configured to, if the active region is within the range of the initial inspection path, take the current position of the collection device as a new starting point, re-plan a local safe path based on the remaining uncovered region, and obtain a target inspection path.

[0103] In an implementation manner, the semantic segmentation result acquisition module 30 comprises:

[0104] A target panoramic image acquisition unit is configured to collect a target panoramic image according to the target inspection path.

[0105] A high-level semantic feature acquisition unit is configured to introduce a multi-scale feature fusion module in an encoder stage of a U-Net semantic segmentation algorithm, perform multi-scale feature extraction on the target panoramic image, and fuse global context information to obtain enhanced high-level semantic features.

[0106] A semantic segmentation result acquisition unit is configured to introduce a cross-attention mechanism in a decoder stage of the U-Net semantic segmentation algorithm, calculate attention weights for the high-level semantic features and output features of the current level of the decoder, and perform selective fusion of the features based on the attention weights to obtain a semantic segmentation result.

[0107] In one implementation, the construction progress acquisition module 40 includes:

[0108] The target 3D dense SLAM map acquisition unit is used to map the semantic segmentation result to the 3D dense point cloud map through back projection algorithm and point cloud fusion technology to obtain the target 3D dense point cloud map with component semantics.

[0109] The unit for obtaining the amount of work completed and the total design quantity is used to obtain the amount of work completed and the total design quantity based on the target three-dimensional dense point cloud map and BIM model.

[0110] The construction progress acquisition unit is used to obtain the construction progress based on the amount of work already completed and the total design quantity.

[0111] The working principle of each module in the component-level construction progress detection system based on panoramic vision in this embodiment is the same as that of each step in the above method embodiment, and will not be repeated here.

[0112] Based on the above embodiments, the present invention also provides a terminal, the principle block diagram of which can be as follows: Figure 7 As shown. The terminal may include one or more processors 100 ( Figure 7 (Only one is shown in the image), a memory 101, and a computer program 102 stored in the memory 101 and executable on one or more processors 100, such as a component-level construction progress detection program based on panoramic vision. When one or more processors 100 execute the computer program 102, they can implement the various steps in the embodiment of the component-level construction progress detection method based on panoramic vision. Alternatively, when one or more processors 100 execute the computer program 102, they can implement the functions of each module / unit in the embodiment of the component-level construction progress detection method based on panoramic vision, which is not limited here.

[0113] In one embodiment, the processor 100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0114] In an embodiment, the storage 101 can be an internal storage unit of the electronic device, for example, a hard disk or a memory of the electronic device. The storage 101 can also be an external storage device of the electronic device, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, or the like equipped on the electronic device. Further, the storage 101 can include both the internal storage unit and the external storage device of the electronic device. The storage 101 is used to store a computer program and other programs and data required by the terminal. The storage 101 can also be used to temporarily store data that has been output or will be output.

[0115] Those skilled in the art can understand that, Figure 7 The person skilled in the art can understand that the principle block diagram shown in the above-mentioned embodiments is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the terminal to which the scheme of the present application is applied. The specific terminal can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0116] The person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiments. Any reference to memory, storage, operating database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0117] It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit the same; and although the present application has been described in detail with reference to the foregoing embodiments, it should be appreciated by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features thereof can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A component-level construction progress detection method based on panoramic vision, characterized in that, The method includes: Visual inertial 3D reconstruction is performed on the initial panoramic image data to obtain a 3D dense point cloud map; Based on the aforementioned three-dimensional dense point cloud map, global path planning and local path dynamic adjustment are performed to obtain the target inspection path; A panoramic image of the target is acquired according to the target inspection path, and the panoramic image of the target is processed using a semantic segmentation algorithm to obtain the semantic segmentation result; The semantic segmentation results are mapped to the three-dimensional dense point cloud map to obtain a target three-dimensional dense point cloud map with component semantics. Based on the target three-dimensional dense point cloud map and the BIM model, the construction progress is obtained, wherein the BIM model and the target three-dimensional dense point cloud map are spatially aligned.

2. The component-level construction progress detection method based on panoramic vision according to claim 1, characterized in that, The step of performing visual-inertial 3D reconstruction on the initial panoramic image data to obtain a 3D dense point cloud map includes: The initial panoramic image data is processed using feature extraction algorithms, feature point matching algorithms, essential matrix estimation algorithms, and epipolar geometric constraints to obtain an initial sparse point cloud map. The camera pose data and motion attitude data in the initial sparse point cloud map are tightly coupled and optimized to obtain a high-precision sparse point cloud map. The high-precision sparse point cloud map is processed using a multi-view stereo algorithm to generate point cloud data. Based on the point cloud data, filtering and moving least squares surface reconstruction are performed to obtain a three-dimensional dense point cloud map.

3. The component-level construction progress detection method based on panoramic vision according to claim 2, characterized in that, The process of performing global path planning and dynamic local path adjustment based on the three-dimensional dense point cloud map to obtain the target inspection path includes: Target nodes are identified in the three-dimensional dense point cloud map, and global path planning is performed based on the target nodes to obtain an initial inspection path; The initial inspection path is dynamically adjusted locally to obtain the target inspection path.

4. The component-level construction progress detection method based on panoramic vision according to claim 3, characterized in that, The step of identifying target nodes in the 3D dense point cloud map and performing global path planning based on the target nodes to obtain an initial inspection path includes: The 3D dense point cloud map is divided into regions, and the target nodes are obtained based on the divided regions; A field-of-view coverage model is constructed for the target node using panoramic camera parameters, and the coverage efficiency of the target node for undetected areas is calculated based on the field-of-view coverage model. Global path planning is performed based on the coverage efficiency and path length to obtain the initial inspection path.

5. The component-level construction progress detection method based on panoramic vision according to claim 4, characterized in that, The step of dynamically adjusting the initial inspection path to obtain the target inspection path includes: If a dynamic obstacle is detected, an obstacle detection algorithm based on optical flow is used to determine the obstacle's position and movement trend. Based on the obstacle's location and its movement trend, the activity area of ​​the dynamic obstacle is determined; If the activity area is within the range of the initial inspection path, then the current position of the data acquisition device is used as the new starting point, and a new local safety path is planned based on the remaining uncovered area to obtain the target inspection path.

6. The component-level construction progress detection method based on panoramic vision according to claim 5, characterized in that, The target panoramic image is processed using a semantic segmentation algorithm to obtain a semantic segmentation result. The semantic segmentation algorithm is an improved U-Net semantic segmentation algorithm, comprising: A multi-scale feature fusion module is introduced in the encoder stage of the U-Net semantic segmentation algorithm to extract multi-scale features from the target panoramic image and fuse global context information to obtain enhanced high-level semantic features. A cross-attention mechanism is introduced in the decoder stage of the U-Net semantic segmentation algorithm. Attention weights are calculated for the high-level semantic features and the output features of the current level of the decoder. Based on the attention weights, the features are selectively fused to obtain the semantic segmentation result.

7. The component-level construction progress detection method based on panoramic vision according to claim 6, characterized in that, The semantic segmentation result is mapped to the 3D dense point cloud map to obtain a target 3D dense point cloud map with component semantics. Based on the target 3D dense point cloud map and the BIM model, the construction progress is obtained. The BIM model and the target 3D dense point cloud map are spatially aligned, including: By using back projection algorithm and point cloud fusion technology, the semantic segmentation result is mapped to the three-dimensional dense point cloud map to obtain a target three-dimensional dense point cloud map with component semantics. Based on the target 3D dense point cloud map and BIM model, the completed construction volume and the total design volume are obtained; Based on the amount of work already completed and the total design quantity, the construction schedule is obtained.

8. A component-level construction progress detection system based on panoramic vision, characterized in that, The system includes: The 3D dense point cloud map acquisition module is used to perform visual inertial 3D reconstruction on the initial panoramic image data to obtain a 3D dense point cloud map. The target inspection path acquisition module is used to perform global path planning and local path dynamic adjustment based on the three-dimensional dense point cloud map to obtain the target inspection path. The semantic segmentation result acquisition module is used to acquire a panoramic image of the target based on the target inspection path, and process the panoramic image of the target using a semantic segmentation algorithm to obtain a semantic segmentation result. The construction progress acquisition module is used to map the semantic segmentation results to the three-dimensional dense point cloud map to obtain a target three-dimensional dense point cloud map with component semantics, and to obtain the construction progress based on the target three-dimensional dense point cloud map and the BIM model, wherein the BIM model is spatially aligned with the target three-dimensional dense point cloud map.

9. A terminal, characterized in that, The terminal includes a memory, a processor, and a component-level construction progress detection program based on panoramic vision stored in the memory and executable on the processor. When the processor executes the component-level construction progress detection program based on panoramic vision, it implements the steps of the component-level construction progress detection method based on panoramic vision as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a component-level construction progress detection program based on panoramic vision. When the component-level construction progress detection program based on panoramic vision is executed by a processor, it implements the steps of the component-level construction progress detection method based on panoramic vision as described in any one of claims 1-7.

Images