High-rise building curtain wall construction progress monitoring method and system based on multi-view images

Abstract

The invention discloses a high-rise building curtain wall construction progress monitoring method and system based on a multi-view image, and belongs to the field of curtain wall construction progress monitoring, and the method comprises the following steps: S1, employing a network dome camera and a total station to obtain the multi-view image covering the full facade of a building and the real coordinates of ground control points; s2, generating a building appearance model by using a Poisson surface reconstruction algorithm based on the multi-view image and the real coordinates of the ground control points; s3, virtual camera parameters are configured based on the building appearance model, an image-building mapping relation is established, and a plurality of layered and blocked wall units are obtained; and S4, obtaining the construction progress and generating a visual report. By adopting the high-rise building curtain wall construction progress monitoring method and system based on the multi-view image, through the multi-view image fusion three-dimensional reconstruction and intelligent segmentation algorithm, the image distortion influence can be eliminated, and low-cost, efficient and accurate high-rise building curtain wall construction progress quantification and non-construction area positioning are realized.

Description

Technical Field

[0001] This invention relates to the field of curtain wall construction progress monitoring technology, and in particular to a method and system for monitoring the construction progress of high-rise building curtain walls based on multi-view images. Background Technology

[0002] With the acceleration of urbanization, super high-rise buildings have become a core component of urban skylines. As the core facade structure of super high-rise buildings, curtain walls not only bear protective functions such as windproofing, waterproofing, and heat insulation, but also possess architectural decoration and energy-saving attributes. Their construction quality and progress directly determine the overall delivery cycle of the building and its safety and aesthetics after commissioning. However, super high-rise building curtain walls generally have a large area (single buildings often exceed 10,000m). 2 The project is characterized by its complex spatial distribution (including irregular facades and staggered structures) and high construction height. These characteristics make the need for quantifying curtain wall installation progress and accurately locating unfinished units increasingly urgent in traditional construction management, especially in scenarios involving simultaneous construction across multiple sections. Refined progress control has become crucial for efficient project advancement.

[0003] Currently, curtain wall construction progress monitoring is mainly divided into two categories: traditional manual inspection and technology-assisted monitoring. Traditional manual inspection relies on staff to count the installed curtain wall units and measure the dimensions of unconstructed areas. This not only involves high risks of working at heights and high labor costs, but also makes it difficult to guarantee the accuracy of progress data due to the limited field of vision of ultra-high-rise curtain walls and the ease with which manual counting can miss some details.

[0004] Technically assisted monitoring typically employs sensors, visual recognition, and other technologies to achieve automated monitoring. Sensor-based assisted monitoring methods require assigning a unique identification code (such as an RFID tag) to each curtain wall unit and deploying sensor receiving units at the construction site to identify the information (e.g., patent CN112437408A). While this can accurately correspond to the installation status of a single curtain wall unit, the number of curtain wall units in ultra-high-rise buildings often reaches thousands or even tens of thousands. The material and deployment costs of sensors and identification codes increase exponentially. Furthermore, subsequent troubleshooting and replacement of sensor malfunctions require a simultaneous halt to curtain wall construction, further increasing management costs and the risk of project delays. Consequently, the methods lack economic viability and practicality.

[0005] Visual recognition-based assisted monitoring methods generally use single images for identification. However, due to the large height and wide span of super high-rise facades, single images are prone to significant angular distortion due to perspective effects, resulting in errors exceeding 20% ​​in the calculation of curtain wall panel area and making it impossible to accurately quantify progress. Some solutions improve recognition accuracy by capturing images of local curtain wall units and manually correcting distortion (such as patent CN113239806A), but this method can only obtain information of local areas, and the manual correction process requires additional manpower. More importantly, local images lack correlation with the overall building engineering coordinate system, making it impossible to locate the specific floor or facade position of unconstructed curtain wall units, which is difficult to support global construction scheduling. Other solutions use image-assisted laser positioning technology to identify and locate construction components (such as the paper "Research on Long-Distance Photoelectric Recognition Installation Positioning System for Large Prefabricated Buildings"), but its core is adapted to the "single component, single installation" scenario. However, super high-rise curtain wall construction often involves the simultaneous installation of multiple units, resulting in low recognition efficiency and inability to process in batches, making it difficult to meet the progress monitoring needs of large-scale curtain wall construction. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for monitoring the construction progress of high-rise building curtain walls based on multi-view images, thereby solving the aforementioned technical problems.

[0007] To achieve the above objectives, this invention provides a method for monitoring the construction progress of high-rise building curtain walls based on multi-view images, comprising the following steps: S1. Fix the network PTZ camera to the front end of the tower crane boom, and set up at least four non-collinear ground control points within the shooting range of the network PTZ camera. Then use the network PTZ camera and total station to obtain multi-view images covering the entire facade of the building and the real coordinates of the ground control points. S2. Based on multi-view images and the real coordinates of ground control points, a normalized 3D point cloud is generated using the COLMAP algorithm, and then a real-scale point cloud is obtained through coordinate transformation. Finally, the main building is separated from the real-scale point cloud, and a building appearance model is generated using the Poisson surface reconstruction algorithm. S3. Based on the building appearance model, configure the virtual camera parameters, generate a distortion-free front view of the curtain wall through the ray tracing algorithm, establish the image-building mapping relationship, and divide the distortion-free front view of the curtain wall into layers and blocks according to the floor to obtain multiple wall units of the layered blocks. S4. For each wall unit, the best viewing angle image is selected from multi-view images using evaluation indicators. Then, the image segmentation algorithm is used to identify the pixel area of ​​the installed curtain wall in the best viewing angle image, thereby obtaining the construction progress and generating a visualization report.

[0008] Preferably, step S2 specifically includes the following steps: S21. Preprocessing: Collect multi-view images covering different seasons, weather and lighting conditions, and perform grayscale conversion and contrast enhancement processing on the multi-view images in sequence. S22. Mark the positions of ground control points in the preprocessed multi-view image and obtain the centroid coordinates of the marked area, and use them as the pixel coordinates of the ground control points. S23. Using the pixel coordinates of ground control points as constraints, generate a normalized 3D point cloud using the COLMAP algorithm. S24. Coordinate Transformation: Determine the relative coordinates of each ground control unit from the normalized 3D point cloud. And combined with the actual coordinates of ground control points Solve for the coordinate transformation matrix : (1); In the formula, Represents the actual coordinates of ground control points homogeneous coordinates; Relative coordinates of the ground control instrument homogeneous coordinates, and , These represent the pixel coordinates of the ground control points in the a-view image and the b-view image, respectively. These represent the camera extrinsic parameters for the image from viewpoint a and viewpoint b, respectively. These represent the camera intrinsic parameters for the image from viewpoint a and viewpoint b, respectively. Represents the DLT algorithm; S25. Use formula (1) to obtain the true coordinates of each ground control point. The homogeneous coordinates are obtained to obtain the point cloud at the true scale; S26. Based on the orthogonal projection of point clouds, the main body of a single building is separated from the real-scale point cloud to obtain the point cloud of the main body of the building; S27. Configure the Poisson surface reconstruction parameters and run the Poisson surface reconstruction algorithm to convert the discrete building point cloud into a building mesh model. S28. Attach the texture information from the multi-view images to the mesh facets of the building mesh model to restore the building's appearance and obtain the building appearance model.

[0009] Preferably, step S23 specifically includes the following steps: S231. Organize the pixel coordinates of the ground control points and the real-world coordinates into a text file that conforms to COLMAP requirements; S232. Import the multi-view image marked with the coordinates of the ground control points into COLMAP; S233. After setting the number of feature points, use the SIFT algorithm to extract features and obtain the coordinates of the feature points. S234. Use the FLANN matching algorithm to perform cross-image matching of feature points in different images and output feature matching pairs. S235. The Bundle Adjustment algorithm is used to minimize the deviation between the theoretical corresponding position of the ground control point pixel coordinates and the actual coordinates of the ground control point in order to optimize the camera's intrinsic and extrinsic parameters. S236. Based on the optimized camera intrinsic and extrinsic parameters and feature matching pairs, the three-dimensional coordinates of feature points are calculated using a triangulation algorithm to generate a sparse three-dimensional point cloud. S237. Call the built-in MVS algorithm of COLMAP to calculate the three-dimensional spatial depth corresponding to each pixel in each view image, and fuse the depth maps of multi-view images to obtain a normalized dense three-dimensional point cloud. S238. The dense 3D point cloud is sequentially denoised and smoothed to obtain a normalized 3D point cloud.

[0010] Preferably, step S3 specifically includes the following steps: S31. Configure virtual camera parameters based on the building appearance model; S32. Divide the building facade model into triangular facets, each of which is configured with three vertex coordinates and material properties; and emit light rays from the virtual camera, using an octree structure to find the intersection points of the light rays and the triangular facets to obtain the front view of the building facade. S33. Set the starting point of the light source to the optical center of the virtual camera. The direction vector is The equation of the ray is as follows: (2); In the formula, Indicates the intersection of the light ray and the triangular facet. The position vector; The position vector representing the optical center of the virtual camera; Indicates a scalar parameter; S34. Solve the equations of formula (2) and the plane equation of the triangular facet to find the intersection point of the light ray and the triangular facet. ; S35, Based on intersection points The material properties of the triangular facet are used to calculate the pixel color of the corresponding pixel in the front view of the building facade using an improved Phong lighting model. : (3); In the formula, , and These represent ambient light intensity, diffuse light intensity, and specular light intensity, respectively. , and These are coefficients representing ambient light intensity, diffuse light intensity, and specular light intensity, respectively. Represents the normal vector of a surface patch; Indicates the specular reflectance index; Represents the direction vector of the light source; This represents the direction vector of the reflected ray; Represents the observation direction vector; S36. Repeat steps S33-S35 until all pixels of the front view of the building facade are traversed to obtain the complete front view; S36. Determine the image scale based on the actual dimensions of the building and the pixel dimensions of the complete front view. : (4); In the formula, and These represent the actual height and width of the building, respectively. and These represent the height and width of the complete front view, respectively. S37, Based on image ratio Establish the following image-building mapping relationship: (5); In the formula, Indicates the virtual camera's orientation; These represent the virtual camera in and Focal length in direction; These represent the center coordinates of the complete front view; Represents the three-dimensional coordinates of the building; Represents the two-dimensional coordinates of the image; S38. Vertical Stratification: Based on standard floor height and actual building height. and the height of the complete front view Determine the pixel height of each layer to obtain the sub-image of each layer; S39. Horizontal segmentation: Divide each sub-image layer into blocks in the horizontal direction according to the set pixel values ​​to obtain wall units.

[0011] Preferably, in step S31, the following virtual camera parameters are configured: The virtual camera is a pinhole camera model; the optical center of the virtual camera is placed at a distance of 1.5 times the building height from the building facade to ensure coverage of the entire building facade; the deviation between the virtual camera's viewing direction and the building facade normal is ≤±3°; the virtual camera's extrinsic parameters are represented by quaternions for the rotation matrix, the direction of the rotation matrix is ​​consistent with the building facade normal, and the translation vector points to the position of the optical center in the world coordinate system.

[0012] Preferably, in step S4, the suitability of the included angle and the suitability of the distance are used as evaluation indicators.

[0013] Preferably, step S4 specifically includes the following steps: S41. Assume the normal to the building facade is in the world coordinate system as... The actual image viewpoint direction vector is The included angle is calculated using the Eigen library. and the suitability of the included angle : (6); (7); Distance suitability The expression is as follows: (8); In the formula, This indicates the distance between the network PTZ camera and the building in the actual captured image; S42, Based on the suitability of the included angle and distance suitability Determine the overall score : (9); In the formula, and All represent weights; S43, Overall score of all multi-view images Sort the images from highest to lowest score, and consider the highest-scoring image as the wall unit. The optimal viewing angle is denoted as the optimal viewing angle image. ; S44. For each image with the best viewing angle The YOLOv8 algorithm was used to identify the curtain wall components, and only the wall units were retained. The distribution of curtain wall components within a wall unit is obtained by analyzing the curtain wall pixels within the area. S45, All images from the best viewing angles The pixels of the identified curtain wall components are mapped onto the front view of the building facade. All curtain wall pixels are merged to obtain the distribution of installed curtain walls on the facade, reflecting the location of uninstalled curtain wall components, as well as the curtain wall construction progress of the entire building, each building facade, and each floor. (10); (11); (12); In the formula, Indicates the inverse operation; This indicates the construction progress of the curtain wall on each building facade or each floor. Represents the pixels belonging to the curtain wall components on the front view of the building facade. , Represents the pixels belonging to each floor or each building facade on the front view of the building's exterior. ; This indicates the overall construction progress of the building's curtain wall. Indicates the first Construction progress of the curtain wall of each building facade; This indicates the total number of building facades.

[0014] A system for implementing a method for monitoring the construction progress of high-rise building curtain walls based on multi-view images includes: The hardware layout and data acquisition module is used to fix the network PTZ camera to the front end of the tower crane boom and arrange at least four non-collinear ground control points within the shooting range of the network PTZ camera. Then, the network PTZ camera and the total station are used to acquire multi-view images covering the entire facade of the building and the real coordinates of the ground control points. The 3D reconstruction module for the main building is used to generate a normalized 3D point cloud based on multi-view images and the real coordinates of ground control points using the COLMAP algorithm. Then, a real-scale point cloud is obtained through coordinate transformation. Finally, the main building is separated from the real-scale point cloud, and a Poisson surface reconstruction algorithm is used to generate the building appearance model. The wall unit division module is used to configure virtual camera parameters based on the building appearance model, generate a distortion-free curtain wall front view through ray tracing algorithm, establish an image-building mapping relationship, and divide the distortion-free curtain wall front view into layers and blocks according to the floor to obtain multiple wall units in layers and blocks. The curtain wall construction progress recognition module is used to select the best viewing angle image from multiple wall units with the help of evaluation indicators, and then use image segmentation algorithm to identify the installed curtain wall pixel area in the best viewing angle image, thereby obtaining the construction progress and generating a visualization report.

[0015] Therefore, the beneficial effects of the above-mentioned method and system for monitoring the construction progress of high-rise building curtain walls based on multi-view images in this invention are as follows: 1. High accuracy: By combining 3D reconstruction, image correction and 2D-3D mapping relationship, the influence of image distortion is eliminated, and the accurate quantitative calculation of curtain wall construction progress and accurate 3D spatial positioning of unconstructed areas are achieved. 2. High degree of automation: Through automated algorithms such as optimal selection of multi-view images and YOLOv8 intelligent segmentation, manual inspection is replaced, which greatly improves the efficiency of curtain wall progress monitoring; 3. Controllable cost: Only conventional equipment such as high-definition network PTZ cameras and total stations are used, without the need for additional special sensors (such as RFID tags), which reduces equipment investment and management costs; 4. Strong visualization: It can generate visual reports including progress bars, color-coded charts, progress trend graphs, etc., providing intuitive and clear decision-making basis for project management; 5. Wide applicability: Relying on the tower crane's full-view image acquisition capability and flexible algorithm logic, it is suitable for monitoring the curtain wall construction progress of various high-rise buildings, including super high-rise buildings and irregular shapes.

[0016] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0017] Figure 1 This is a flowchart of the high-rise building curtain wall construction progress monitoring method based on multi-view images according to the present invention; Figure 2 This is a ground control point layout diagram for the high-rise building curtain wall construction progress monitoring method based on multi-view images according to the present invention. Figure 3 This is a schematic diagram of image-building mapping for the high-rise building curtain wall construction progress monitoring method based on multi-view images of the present invention; Figure 4 This is an example of optimal image selection for the high-rise building curtain wall construction progress monitoring method based on multi-view images of the present invention; Figure 5 This is a schematic diagram of the curtain wall component identification method for high-rise building curtain wall construction progress monitoring based on multi-view images according to the present invention. Figure 6 This is a visualization example of the high-rise building curtain wall construction progress monitoring method based on multi-view images according to the present invention; Figure 7 Another visualization example of the high-rise building curtain wall construction progress monitoring method based on multi-view images of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the embodiments of the present invention and are not intended to limit the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of this application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout.

[0019] It should be noted that the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as a process, method, system, product, or server that includes a series of steps or units, not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or device.

[0020] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0021] like Figures 1-7 As shown, the method for monitoring the construction progress of high-rise building curtain walls based on multi-view images includes the following steps: S1. Fix the network PTZ camera to the front end of the tower crane boom, and set up at least four non-collinear ground control points within the shooting range of the network PTZ camera. Then use the network PTZ camera and total station to obtain multi-view images covering the entire facade of the building and the real coordinates of the ground control points. In this embodiment, the network PTZ camera mentioned in step S1 has a resolution of 4K (3840×2160), a field of view of 80°-100°, and a frame rate of 30fps, ensuring that the image clarity and coverage are suitable for shooting high-rise building curtain walls. After installing it at the front end of the tower crane boom, its angle is adjusted to ensure that the viewing angle can cover the entire building, thereby obtaining comprehensive architectural image information.

[0022] Furthermore, the shooting strategy for network PTZ cameras is as follows: For conventional buildings: Control the tower crane to rotate 1-2 revolutions, and simultaneously adjust the tilt angle of the network PTZ camera to ensure the acquisition of 360° multi-view images of the building; For super high-rise buildings (height > 100m): Shoot in three segments (upper 1 / 3, middle 1 / 3, lower 1 / 3), and adjust the camera field of view for each segment separately to avoid blurring of high-rise / low-rise images caused by single-revolution shooting. At this time, it is also necessary to stitch the segmented partial images into a whole image covering the entire facade of the building, and remove redundant areas by cropping to ensure that the pixel size of the stitched image corresponds to the actual size of the building; Shooting frequency: Determine according to the construction progress (e.g., acquire one set of multi-angle images every 30 minutes), and manually trigger additional acquisition at key construction nodes (e.g., completion of the curtain wall of a certain floor).

[0023] Ground control points (GCPs) are precisely measured using a total station with an accuracy within ±1 cm. The size of the markers is determined based on a 1:100 ratio with the building height. For example, for a 100m building, the marker size can be 1m x 1m (the specific size can be adjusted according to actual conditions and is not limited here). The markers must be visible, easily identifiable (e.g., high-contrast color), fixed in position, and at varying heights. For rectangular buildings, GCPs are placed at equal intervals of 1 / 5 of the building's side length along the four corners, long sides, and short sides. For irregular buildings, GCPs are placed at key curvature changes in the outline, as well as at protruding and recessed sections.

[0024] S2. Based on multi-view images and the real coordinates of ground control points, a normalized 3D point cloud is generated using the COLMAP algorithm, and then a real-scale point cloud is obtained through coordinate transformation. Finally, the main building is separated from the real-scale point cloud, and a building appearance model is generated using the Poisson surface reconstruction algorithm. Step S2 specifically includes the following steps: S21. Preprocessing: Collect multi-view images covering different seasons, weather and lighting conditions, and perform grayscale conversion and contrast enhancement processing on the multi-view images in sequence. In step S21, grayscale processing is used to convert the color multi-view image into a grayscale image, reducing the interference of color information on GCPs feature recognition and ensuring that subsequent feature extraction focuses on the core features of GCPs such as shape and contrast.

[0025] Contrast enhancement refers to using an adaptive histogram equalization algorithm to process grayscale images, thereby improving the contrast between GCPs and the background (such as the edge clarity of GCP markers with yellow backgrounds and black borders) and preventing GCPs from becoming blurred due to uneven lighting (such as cloudy days or backlighting).

[0026] In step S22, the marking results of the same GCPs (such as GCPs-1) in images from different viewpoints are verified and compared, and the pixel coordinates are recorded.

[0027] S22. Mark the positions of ground control points in the preprocessed multi-view image and obtain the centroid coordinates of the marked area, and use them as the pixel coordinates of the ground control points. S23. Using the pixel coordinates of ground control points as constraints, generate a normalized 3D point cloud using the COLMAP algorithm. Step S23 specifically includes the following steps: S231. Organize the pixel coordinates of the ground control points and the real-world coordinates into a text file that conforms to COLMAP requirements; S232. Import the multi-view image marked with the coordinates of the ground control points into COLMAP; S233. After setting the number of feature points, use the SIFT algorithm to extract features and obtain the coordinates of the feature points. S234. Use the FLANN matching algorithm to perform cross-image matching of feature points in different images and output feature matching pairs. S235. The Bundle Adjustment algorithm is used to minimize the deviation between the theoretical corresponding position of the ground control point pixel coordinates and the actual coordinates of the ground control point in order to optimize the camera's intrinsic and extrinsic parameters. S236. Based on the optimized camera intrinsic and extrinsic parameters and feature matching pairs, the three-dimensional coordinates of feature points are calculated using a triangulation algorithm to generate a sparse three-dimensional point cloud. S237. Call the built-in MVS algorithm of COLMAP to calculate the three-dimensional spatial depth corresponding to each pixel in each view image, and fuse the depth maps of multi-view images to obtain a normalized dense three-dimensional point cloud. S238. The dense 3D point cloud is sequentially denoised and smoothed to obtain a normalized 3D point cloud.

[0028] S24. Coordinate Transformation: Determine the relative coordinates of each ground control unit from the normalized 3D point cloud. And combined with the actual coordinates of ground control points Solve for the coordinate transformation matrix : (1); In the formula, Represents the actual coordinates of ground control points homogeneous coordinates; Relative coordinates of the ground control instrument homogeneous coordinates, and , These represent the pixel coordinates of the ground control points in the a-view image and the b-view image, respectively. These represent the camera extrinsic parameters for the image from viewpoint a and viewpoint b, respectively. These represent the camera intrinsic parameters for the image from viewpoint a and viewpoint b, respectively. Represents the DLT algorithm; S25. Use formula (1) to obtain the true coordinates of each ground control point. The homogeneous coordinates are obtained to obtain the point cloud at the true scale; S26. Based on the orthogonal projection of point clouds, the main body of a single building is separated from the real-scale point cloud to obtain the point cloud of the main body of the building; S27. Configure the Poisson surface reconstruction parameters and run the Poisson surface reconstruction algorithm to convert the discrete building point cloud into a building mesh model. In step S27, the Poisson surface reconstruction parameters can be roughly set within the range of 5-10 based on the building's appearance characteristics, the octree depth can be roughly set within the range of 6-10, and the sample point density can be approximately 1000-2000 points / cubic meter. The calculation of the building's length, width, and height is performed by analyzing the coordinate range of the building's appearance model in three-dimensional space. In the horizontal direction, the farthest points of the building along the X and Y axes are determined, and the difference between them represents the building's length and width. In the vertical direction, the highest and lowest points of the building along the Z axis are found, and the difference between them represents the building's height.

[0029] S28. Attach the texture information from the multi-view images to the mesh facets of the building mesh model to restore the building's appearance and obtain the building appearance model.

[0030] S3. Based on the building appearance model, configure the virtual camera parameters, generate a distortion-free front view of the curtain wall through the ray tracing algorithm, establish the image-building mapping relationship, and divide the distortion-free front view of the curtain wall into layers and blocks according to the floor to obtain multiple wall units of the layered blocks. Step S3 specifically includes the following steps: S31. Configure virtual camera parameters based on the building appearance model; In step S31, configure the following virtual camera parameters: The virtual camera is a pinhole camera model; the optical center of the virtual camera is placed at a distance of 1.5 times the building height from the building facade to ensure coverage of the entire building facade; the deviation between the virtual camera's viewing direction and the building facade normal is ≤±3°; the virtual camera's extrinsic parameters are represented by quaternions for the rotation matrix, the direction of the rotation matrix is ​​consistent with the building facade normal, and the translation vector points to the position of the optical center in the world coordinate system.

[0031] S32. Divide the building facade model into triangular facets, each of which is configured with three vertex coordinates and material properties; and emit light rays from the virtual camera, using an octree structure to find the intersection points of the light rays and the triangular facets to obtain the front view of the building facade. S33. Set the starting point of the light source to the optical center of the virtual camera. The direction vector is The equation of the ray is as follows: (2); In the formula, Indicates the intersection of the light ray and the triangular facet. The position vector; The position vector representing the optical center of the virtual camera; Indicates a scalar parameter; S34. Solve the equations of formula (2) and the plane equation of the triangular facet to find the intersection point of the light ray and the triangular facet. ; S35, Based on intersection points The material properties of the triangular facet are used to calculate the pixel color of the corresponding pixel in the front view of the building facade using an improved Phong lighting model. : (3); In the formula, , and These represent ambient light intensity, diffuse light intensity, and specular light intensity, respectively. , and These are coefficients representing ambient light intensity, diffuse light intensity, and specular light intensity, respectively. Represents the normal vector of a surface patch; Indicates the specular reflectance index; Represents the direction vector of the light source; This represents the direction vector of the reflected ray; Represents the observation direction vector; S36. Repeat steps S33-S35 until all pixels of the front view of the building facade are traversed to obtain the complete front view; S36. Determine the image scale based on the actual dimensions of the building and the pixel dimensions of the complete front view. : (4); In the formula, and These represent the actual height and width of the building, respectively. and These represent the height and width of the complete front view, respectively. S37, Based on image ratio Establish the following image-building mapping relationship: (5); In the formula, Indicates the virtual camera's orientation; These represent the virtual camera in and Focal length in direction; These represent the center coordinates of the complete front view; Represents the three-dimensional coordinates of the building; Represents the two-dimensional coordinates of the image; S38. Vertical Stratification: Based on standard floor height and actual building height. and the height of the complete front view Determine the pixel height of each layer to obtain the sub-image of each layer; S39. Horizontal segmentation: Divide each sub-image layer into blocks in the horizontal direction according to the set pixel values ​​to obtain wall units.

[0032] S4. For each wall unit, the best viewing angle image is selected from multi-view images using evaluation indicators. Then, the image segmentation algorithm is used to identify the pixel area of ​​the installed curtain wall in the best viewing angle image, thereby obtaining the construction progress and generating a visualization report.

[0033] In step S4, the suitability of the included angle and the suitability of the distance are used as evaluation indicators.

[0034] Preferably, step S4 specifically includes the following steps: S41. Assume the normal to the building facade is in the world coordinate system as... The actual image viewpoint direction vector is (The viewpoint direction vector can be calculated from the extrinsic rotation matrix), and the included angle is calculated using the Eigen library. and the suitability of the included angle : (6); (7); Distance suitability The expression is as follows: (8); In the formula, This indicates the distance between the network PTZ camera and the building in the actual captured image; S42, Based on the suitability of the included angle and distance suitability Determine the overall score : (9); In the formula, and All represent weights; S43, Overall score of all multi-view images Sort the images from highest to lowest score, and consider the highest-scoring image as the wall unit. The optimal viewing angle is denoted as the optimal viewing angle image. ; S44. For each image with the best viewing angle The YOLOv8 algorithm was used to identify the curtain wall components, and only the wall units were retained. The distribution of curtain wall components within a wall unit is obtained by analyzing the curtain wall pixels within the area. S45, All images from the best viewing angles The pixels of the identified curtain wall components are mapped onto the front view of the building facade. All curtain wall pixels are merged to obtain the distribution of installed curtain walls on the facade, reflecting the location of uninstalled curtain wall components, as well as the curtain wall construction progress of the entire building, each building facade, and each floor. (10); (11); (12); In the formula, Indicates the inverse operation; This indicates the construction progress of the curtain wall on each building facade or each floor. Represents the pixels belonging to the curtain wall components on the front view of the building facade. , Represents the pixels belonging to each floor or each building facade on the front view of the building's exterior. ; This indicates the overall construction progress of the building's curtain wall. Indicates the first Construction progress of the curtain wall of each building facade; This indicates the total number of building facades.

[0035] A system for implementing a method for monitoring the construction progress of high-rise building curtain walls based on multi-view images includes: The hardware layout and data acquisition module is used to fix the network PTZ camera to the front end of the tower crane boom and arrange at least four non-collinear ground control points within the shooting range of the network PTZ camera. Then, the network PTZ camera and the total station are used to acquire multi-view images covering the entire facade of the building and the real coordinates of the ground control points. The 3D reconstruction module for the main building is used to generate a normalized 3D point cloud based on multi-view images and the real coordinates of ground control points using the COLMAP algorithm. Then, a real-scale point cloud is obtained through coordinate transformation. Finally, the main building is separated from the real-scale point cloud, and a Poisson surface reconstruction algorithm is used to generate the building appearance model. The wall unit division module is used to configure virtual camera parameters based on the building appearance model, generate a distortion-free curtain wall front view through ray tracing algorithm, establish an image-building mapping relationship, and divide the distortion-free curtain wall front view into layers and blocks according to the floor to obtain multiple wall units in layers and blocks. The curtain wall construction progress recognition module is used to select the best viewing angle image from multiple wall units with the help of evaluation indicators, and then use image segmentation algorithm to identify the installed curtain wall pixel area in the best viewing angle image, thereby obtaining the construction progress and generating a visualization report.

[0036] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for monitoring the construction progress of high-rise building curtain walls based on multi-view images, characterized in that: Includes the following steps: S1. Fix the network PTZ camera to the front end of the tower crane boom, and set up at least four non-collinear ground control points within the shooting range of the network PTZ camera. Then use the network PTZ camera and total station to obtain multi-view images covering the entire facade of the building and the real coordinates of the ground control points. S2. Based on multi-view images and the real coordinates of ground control points, a normalized 3D point cloud is generated using the COLMAP algorithm, and then a real-scale point cloud is obtained through coordinate transformation. Finally, the main building is separated from the real-scale point cloud, and a building appearance model is generated using the Poisson surface reconstruction algorithm. S3. Based on the building appearance model, configure the virtual camera parameters, generate a distortion-free front view of the curtain wall through the ray tracing algorithm, establish the image-building mapping relationship, and divide the distortion-free front view of the curtain wall into layers and blocks according to the floor to obtain multiple wall units of the layered blocks. S4. For each wall unit, the best viewing angle image is selected from multi-view images using evaluation indicators. Then, the image segmentation algorithm is used to identify the pixel area of ​​the installed curtain wall in the best viewing angle image, thereby obtaining the construction progress and generating a visualization report.

2. The method for monitoring the construction progress of high-rise building curtain walls based on multi-view images according to claim 1, characterized in that: Step S2 specifically includes the following steps: S21. Preprocessing: Collect multi-view images covering different seasons, weather and lighting conditions, and perform grayscale conversion and contrast enhancement processing on the multi-view images in sequence. S22. Mark the positions of ground control points in the preprocessed multi-view image and obtain the centroid coordinates of the marked area, and use them as the pixel coordinates of the ground control points. S23. Using the pixel coordinates of ground control points as constraints, generate a normalized 3D point cloud using the COLMAP algorithm. S24. Coordinate Transformation: Determine the relative coordinates of each ground control unit from the normalized 3D point cloud. And combined with the actual coordinates of ground control points Solve for the coordinate transformation matrix : （1）； In the formula, Represents the actual coordinates of ground control points homogeneous coordinates; Relative coordinates of the ground control instrument homogeneous coordinates, and , These represent the pixel coordinates of the ground control points in the a-view image and the b-view image, respectively. These represent the camera extrinsic parameters for the image from viewpoint a and viewpoint b, respectively. These represent the camera intrinsic parameters for the image from viewpoint a and viewpoint b, respectively. Represents the DLT algorithm; S25. Use formula (1) to obtain the true coordinates of each ground control point. The homogeneous coordinates are obtained to obtain the point cloud at the true scale; S26. Based on the orthogonal projection of point clouds, the main body of a single building is separated from the real-scale point cloud to obtain the point cloud of the main body of the building; S27. Configure the Poisson surface reconstruction parameters and run the Poisson surface reconstruction algorithm to convert the discrete building point cloud into a building mesh model. S28. Attach the texture information from the multi-view images to the mesh facets of the building mesh model to restore the building's appearance and obtain the building appearance model.

3. The method for monitoring the construction progress of high-rise building curtain walls based on multi-view images according to claim 2, characterized in that: Step S23 specifically includes the following steps: S231. Organize the pixel coordinates of the ground control points and the real-world coordinates into a text file that conforms to COLMAP requirements; S232. Import the multi-view image marked with the coordinates of the ground control points into COLMAP; S233. After setting the number of feature points, use the SIFT algorithm to extract features and obtain the coordinates of the feature points. S234. Use the FLANN matching algorithm to perform cross-image matching of feature points in different images and output feature matching pairs. S235. The Bundle Adjustment algorithm is used to minimize the deviation between the theoretical corresponding position of the ground control point pixel coordinates and the actual coordinates of the ground control point in order to optimize the camera's intrinsic and extrinsic parameters. S236. Based on the optimized camera intrinsic and extrinsic parameters and feature matching pairs, the three-dimensional coordinates of feature points are calculated using a triangulation algorithm to generate a sparse three-dimensional point cloud. S237. Call the built-in MVS algorithm of COLMAP to calculate the three-dimensional spatial depth corresponding to each pixel in each view image, and fuse the depth maps of multi-view images to obtain a normalized dense three-dimensional point cloud. S238. The dense 3D point cloud is sequentially denoised and smoothed to obtain a normalized 3D point cloud.

4. The method for monitoring the construction progress of high-rise building curtain walls based on multi-view images according to claim 2, characterized in that: Step S3 specifically includes the following steps: S31. Configure virtual camera parameters based on the building appearance model; S32. Divide the building facade model into triangular facets, each of which is configured with three vertex coordinates and material properties; and emit light rays from the virtual camera, using an octree structure to find the intersection points of the light rays and the triangular facets to obtain the front view of the building facade. S33. Set the starting point of the light source to the optical center of the virtual camera. The direction vector is The equation of the ray is as follows: （2）； In the formula, Indicates the intersection of the light ray and the triangular facet. The position vector; The position vector representing the optical center of the virtual camera; Indicates a scalar parameter; S34. Solve the equations of formula (2) and the plane equation of the triangular facet to find the intersection point of the light ray and the triangular facet. ; S35, Based on intersection points The material properties of the triangular facet are used to calculate the pixel color of the corresponding pixel in the front view of the building facade using an improved Phong lighting model. : （3）； In the formula, , and These represent ambient light intensity, diffuse light intensity, and specular light intensity, respectively. , and These are coefficients representing ambient light intensity, diffuse light intensity, and specular light intensity, respectively. Represents the normal vector of a surface patch; Indicates the specular reflectance index; Represents the direction vector of the light source; This represents the direction vector of the reflected ray; Represents the observation direction vector; S36. Repeat steps S33-S35 until all pixels of the front view of the building facade are traversed to obtain the complete front view; S36. Determine the image scale based on the actual dimensions of the building and the pixel dimensions of the complete front view. : （4）； In the formula, and These represent the actual height and width of the building, respectively. and These represent the height and width of the complete front view, respectively. S37, Based on image ratio Establish the following image-building mapping relationship: （5）； In the formula, Indicates the virtual camera's orientation; These represent the virtual camera in and Focal length in direction; These represent the center coordinates of the complete front view; Represents the three-dimensional coordinates of the building; Represents the two-dimensional coordinates of the image; S38. Vertical Stratification: Based on standard floor height and actual building height. and the height of the complete front view Determine the pixel height of each layer to obtain the sub-image of each layer; S39. Horizontal segmentation: Divide each sub-image layer into blocks in the horizontal direction according to the set pixel values ​​to obtain wall units.

5. The method for monitoring the construction progress of high-rise building curtain walls based on multi-view images according to claim 4, characterized in that: In step S31, configure the following virtual camera parameters: The virtual camera is a pinhole camera model; the optical center of the virtual camera is placed at a distance of 1.5 times the building height from the building facade to ensure coverage of the entire building facade; the deviation between the virtual camera's viewing direction and the building facade normal is ≤±3°; the virtual camera's extrinsic parameters are represented by quaternions for the rotation matrix, the direction of the rotation matrix is ​​consistent with the building facade normal, and the translation vector points to the position of the optical center in the world coordinate system.

6. The method for monitoring the construction progress of high-rise building curtain walls based on multi-view images according to claim 4, characterized in that: In step S4, the suitability of the included angle and the suitability of the distance are used as evaluation indicators.

7. The method for monitoring the construction progress of high-rise building curtain walls based on multi-view images according to claim 6, characterized in that: Step S4 specifically includes the following steps: S41. Assume the normal to the building facade is in the world coordinate system as... The actual image viewpoint direction vector is The included angle is calculated using the Eigen library. and the suitability of the included angle : （6）； （7）； Distance suitability The expression is as follows: （8）； In the formula, This indicates the distance between the network PTZ camera and the building in the actual captured image; S42, Based on the suitability of the included angle and distance suitability Determine the overall score : （9）； In the formula, and All represent weights; S43, Overall score of all multi-view images Sort the images from highest to lowest score, and consider the highest-scoring image as the wall unit. The optimal viewing angle is denoted as the optimal viewing angle image. ; S44. For each image with the best viewing angle The YOLOv8 algorithm was used to identify the curtain wall components, and only the wall units were retained. The distribution of curtain wall components within a wall unit is obtained by analyzing the curtain wall pixels within the area. S45, All images from the best viewing angles The pixels of the identified curtain wall components are mapped onto the front view of the building facade. All curtain wall pixels are merged to obtain the distribution of installed curtain walls on the facade, reflecting the location of uninstalled curtain wall components, as well as the curtain wall construction progress of the entire building, each building facade, and each floor. （10）； （11）； （12）； In the formula, Indicates the inverse operation; This indicates the construction progress of the curtain wall on each building facade or each floor. Represents the pixels belonging to the curtain wall components on the front view of the building facade. , Represents the pixels belonging to each floor or each building facade on the front view of the building's exterior. ; This indicates the overall construction progress of the building's curtain wall. Indicates the first Construction progress of the curtain wall of each building facade; This indicates the total number of building facades.

8. A system for implementing the method for monitoring the construction progress of high-rise building curtain walls based on multi-view images as described in any one of claims 1-7, characterized in that: include: The hardware layout and data acquisition module is used to fix the network PTZ camera to the front end of the tower crane boom and arrange at least four non-collinear ground control points within the shooting range of the network PTZ camera. Then, the network PTZ camera and the total station are used to acquire multi-view images covering the entire facade of the building and the real coordinates of the ground control points. The 3D reconstruction module for the main building is used to generate a normalized 3D point cloud based on multi-view images and the real coordinates of ground control points using the COLMAP algorithm. Then, a real-scale point cloud is obtained through coordinate transformation. Finally, the main building is separated from the real-scale point cloud, and a Poisson surface reconstruction algorithm is used to generate the building appearance model. The wall unit division module is used to configure virtual camera parameters based on the building appearance model, generate a distortion-free curtain wall front view through ray tracing algorithm, establish an image-building mapping relationship, and divide the distortion-free curtain wall front view into layers and blocks according to the floor to obtain multiple wall units in layers and blocks. The curtain wall construction progress recognition module is used to select the best viewing angle image from multiple wall units with the help of evaluation indicators, and then use image segmentation algorithm to identify the installed curtain wall pixel area in the best viewing angle image, thereby obtaining the construction progress and generating a visualization report.

Images