Curtain wall installation precision real-time intelligent detection method based on multi-view visual fusion

Abstract

The present application belongs to the technical field of image processing, and particularly relates to a curtain wall installation precision real-time intelligent detection method based on multi-view visual fusion, which comprises the following steps: extracting the two-dimensional coordinate offset direction gradient of the continuous physical joint pixel path, fusing the visual angle offset sensitivity and the gradient average intensity of the joint path to obtain the grid topology deformation energy; calculating the depth perception alignment compensation factor based on the grid topology deformation energy and the curtain wall standard grid modulus coefficient, adaptively weighting and fusing the initial visual three-dimensional space coordinates and the theoretical three-dimensional space coordinates of the preset building BIM model through the compensation factor to obtain the corrected three-dimensional space coordinates; calculating the spatial absolute deviation of the corrected three-dimensional space coordinates and the theoretical three-dimensional space coordinates, comparing the spatial absolute deviation with the preset installation tolerance, and completing the detection and qualification of the curtain wall installation precision. The present application improves the automation degree, detection efficiency and engineering robustness of the real-time detection of the curtain wall installation precision.

Description

Technical Field

[0001] This invention relates to the field of image processing technology. More specifically, this invention relates to a real-time intelligent detection method for curtain wall installation accuracy based on multi-view visual fusion. Background Technology

[0002] As a common external envelope structure for modern large-scale buildings, the installation accuracy of curtain walls directly affects the overall appearance, structural safety, and subsequent service life of the building. Real-time intelligent detection of curtain wall installation accuracy mainly refers to the use of automated detection methods to collect the positional data of curtain wall panels in real time during the building construction process, and compare it with the design drawings to automatically assess whether the installation position of the panels and the gaps between joints meet the tolerance range required by the specifications.

[0003] In actual engineering construction scenarios, due to the large span of building spaces and frequent obstructions to the line of sight at the construction site, traditional manual contact measurement or single-viewpoint measurement instruments are inefficient and cannot capture the global installation posture characteristics. Therefore, multi-view visual inspection technology has been introduced into curtain wall construction sites. By deploying a calibration camera group on site, multiple views of the curtain wall are simultaneously acquired, and the three-dimensional spatial coordinates of the curtain wall feature points are reconstructed using the principle of triangulation, thereby achieving non-contact, large-scale, real-time, and accurate inspection.

[0004] However, due to the repetitive texture features of the curtain wall panels themselves, and the fact that the camera's line of sight is at a long distance and a large angle of incidence when observing large spans on site, the inherent optical perspective magnification effect is superimposed on the repetitive glass texture. This makes it easy for traditional visual algorithms to produce illusions when matching image feature points. As a result, the three-dimensional spatial coordinates calculated by the system deviate from the actual physical position of the curtain wall panel, resulting in coordinate drift points. This can not only trigger false alarms in the system, causing on-site construction personnel to mistakenly believe that the installation is unqualified and blindly rework, but may also cover up the actual installation deviations, causing tilted panels with safety hazards to be missed and released, interfering with the normal progress of on-site construction operations. Summary of the Invention

[0005] To address the aforementioned technical problem of insufficient accuracy in curtain wall installation precision detection, this invention provides a real-time intelligent detection method for curtain wall installation precision based on multi-view visual fusion, comprising: Acquire multi-view image sources of the curtain wall installation process and extract the initial two-dimensional feature point set of each image; obtain the initial calibration matrix and camera focal length of the calibration camera group, and construct multi-view geometric constraints based on the initial calibration matrix; use the multi-view geometric constraints to perform nonlinear triangulation on the initial two-dimensional feature point set to obtain the initial visual three-dimensional spatial coordinates of each feature point; obtain the initial physical distance and spatial incident angle of each feature point; calculate the viewpoint offset sensitivity of each feature point, which is positively correlated with the ratio of the initial physical distance to the camera focal length and negatively correlated with the cosine value of the spatial incident angle; along the adjacent features in the multi-view image source... The process involves extracting continuous physical seam pixel paths from the physical seams formed by points, calculating the 2D coordinate offset gradient of all pixels along the continuous physical seam pixel paths, and using the product of the viewpoint offset sensitivity and the average gradient intensity of the continuous physical seam pixel paths as the mesh topology deformation energy. The theoretical 3D spatial coordinates of each feature point in a preset building model are then obtained. Based on the mesh topology deformation energy, the pre-acquired standard mesh modulus coefficients of the curtain wall panel, and the initial visual 3D spatial coordinates, the corrected 3D spatial coordinates are obtained. Finally, based on the distance between the corrected 3D spatial coordinates and the theoretical 3D spatial coordinates, the installation accuracy of the curtain wall is tested.

[0006] In complex application scenarios such as curtain wall construction sites with large spans, this invention completes the 3D reconstruction of feature points through multi-view epipolar geometric constraints. It combines the sensitivity of viewpoint offset and the average intensity of joint gradient to construct the mesh topological deformation energy and obtain the degree of visual matching distortion. At the same time, it introduces the 3D model of building BIM theory and achieves coordinate adaptive weighted correction through depth perception alignment compensation factors. While capturing small installation deviations on site with high sensitivity, it effectively eliminates extreme anomalies caused by large-scale perspective errors and repetitive glass textures, realizing real-time intelligent detection of curtain wall installation errors with global vision and high precision.

[0007] Preferably, the extraction of the initial two-dimensional feature point set includes: The edge detection algorithm is used to perform Gaussian filtering smoothing and gradient calculation on the multi-view image source to extract the geometric boundary contour pixels of the curtain wall panel. The corner detection operator is combined to extract the contour intersection points. All the extracted contour intersection points are used as the initial two-dimensional feature point set of the curtain wall panel.

[0008] Preferably, the step of constructing multi-view geometric constraints based on the initial calibration matrix includes: designating any two camera views in the multi-view image source as the first camera view and the second camera view, respectively; obtaining the intrinsic parameter matrices of the first camera view and the second camera view; and obtaining the relative rotation matrix from the first camera view to the second camera view. Translation vector Construct the antisymmetric matrix of the translation vector; the fundamental matrix satisfies the expression: The multi-view geometric constraints satisfy the epipolar constraint expression: ; In the formula, Represents the fundamental matrix; , The intrinsic parameter matrix represents the first camera view and the second camera view; Represents the translation vector The antisymmetric matrix; , This represents the two-dimensional homogeneous coordinate vector of the same physical feature point in the first and second camera views. This is the transpose symbol.

[0009] In the application scenario of multi-camera cross-field collaborative observation of large-scale curtain walls, this invention establishes a strict optical coplanar epipolar constraint relationship based on the camera intrinsic and extrinsic parameters to construct a basic matrix, thereby narrowing the search range of feature matching, eliminating the disordered divergence problem in the feature matching process of two-dimensional images under a wide field of view, and improving the accuracy and computational efficiency of multi-viewpoint cross-lens matching.

[0010] Preferably, the viewpoint offset sensitivity satisfies the expression: ; In the formula, Indicates the first Sensitivity to viewpoint shift of each feature point; Indicates the first The initial physical distance between each feature point; Indicates the camera focal length of the calibrated camera group; Indicates the first The spatial incident angle of each feature point; This represents the cosine function.

[0011] When performing long-distance, oblique-view inspections of large-format curtain walls in high-rise buildings with limited camera shooting angles, this invention quantifies the sensitivity of viewing angle shift by analyzing the correlation between observation distance, camera focal length, and spatial incident angle. It also calculates the risk of nonlinear amplification of perspective errors caused by increased observation distance and tilted line of sight, thereby pre-identifying potential areas of visual divergence.

[0012] Preferably, the mesh topology deformation energy satisfies the expression: ; In the formula, Indicates the first Mesh topological deformation energy at each feature point; Indicates the first Sensitivity to viewpoint shift of each feature point; Indicates the first The total number of pixels in the continuous physical seam pixel path of each feature point; This represents the gradient of the two-dimensional coordinate offset direction of the s-th pixel along the continuous physical seam pixel path; This represents the absolute value function.

[0013] When faced with scenarios where repetitive glass textures on curtain walls can easily cause computer visual illusions, this invention extracts imperceptible abnormal distortions in the image by combining the gradient average intensity on the joint path with the sensitivity of the viewing angle offset. This accurately reflects the degree of deformation of the rigid lines of the panel caused by mismatch, thus avoiding misjudging light and shadow and texture interference as actual engineering installation quality problems.

[0014] Preferably, the corrected three-dimensional spatial coordinates satisfy the expression: ; In the formula, Indicates the first The corrected three-dimensional spatial coordinates of each feature point; Indicates the first Depth-aware alignment compensation factor for each feature point; Indicates the first The initial visual three-dimensional space coordinates of each feature point; Indicates the first The theoretical three-dimensional spatial coordinates of each feature point.

[0015] When calculating the three-dimensional spatial coordinates of large-scale curtain walls, this invention achieves intelligent dynamic weight allocation between visual dynamic measurement results and the theoretical prior model of building BIM based on a negative exponential decay depth perception alignment compensation factor. This fully leverages the advantage of visual measurement in capturing real minute displacements, while also safely anchoring the coordinates within the theoretical physical framework when encountering visual environmental distortions, thus avoiding dangerous divergence and offset of the detection results.

[0016] Preferably, the acquisition of the depth-aware alignment compensation factor includes: The negative natural exponential function value of the ratio of the mesh topological deformation energy of the i-th feature point to the standard mesh modulus coefficient of the curtain wall panel is taken as the i-th feature point. Depth-aware alignment compensation factor for each feature point.

[0017] Preferably, the initial physical distance is the Euclidean distance between the optical center of the calibration camera group and the initial visual three-dimensional spatial coordinates of the corresponding feature point.

[0018] Preferably, the spatial incident angle is the angle between the optical axis of the camera that captures the corresponding feature point in the calibration camera group and the normal vector of the theoretical plane of the curtain wall.

[0019] Preferably, the step of detecting the curtain wall installation accuracy based on the distance between the corrected three-dimensional spatial coordinates and the theoretical three-dimensional spatial coordinates includes: Obtain the preset installation tolerance of the curtain wall panel; calculate the Euclidean distance between the corrected three-dimensional spatial coordinates and the theoretical three-dimensional spatial coordinates to obtain the spatial absolute deviation; determine the feature points whose spatial absolute deviation is greater than the preset installation tolerance as feature points with an out-of-tolerance alarm state; highlight the feature points with an out-of-tolerance alarm state and their associated curtain wall panel areas on the real-time monitoring interface; Record the corrected three-dimensional spatial coordinates, theoretical three-dimensional spatial coordinates, and corresponding absolute spatial deviation values ​​of feature points that have out-of-tolerance alarm states to complete the detection of curtain wall installation accuracy.

[0020] The beneficial effects of this invention are as follows: (1) This invention shifts the core detection dimension from conventional surface texture to the rigid physical joint of the curtain wall. By extracting the gradient average intensity and viewing angle shift sensitivity of the joint path, the core features are constructed, which shields the interference of glass reflection and homogeneous repetitive texture, avoids the problem of visual matching distortion, and improves the robustness and detection accuracy of data acquisition in complex construction environments. (2) The grid topology deformation energy index constructed by the present invention can map local visual observation data to the global rigid topology of the curtain wall, and accurately assess the degree of spatial distortion by combining perspective error sensitivity, thereby offsetting the geometric deformation error caused by large field of view perspective projection and improving the three-dimensional measurement accuracy of curtain wall installation in large span spatial scenarios. (3) This invention combines the building BIM theoretical model to establish a dynamic coordinate correction mechanism between visual measurement results and theoretical model. By introducing a depth perception alignment compensation factor that is adaptively adjusted according to matching quality, a dynamic balance between the real-time visual measurement sensitivity and the safety constraints of the BIM theoretical model is achieved. It can automatically identify and filter out coordinate abnormal drift points caused by environmental interference, so that the final detection results can not only keenly capture small installation errors, but also strictly comply with engineering physical constraints, thereby enhancing the system's environmental anti-interference ability and engineering practicality. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating the real-time intelligent detection method for curtain wall installation accuracy based on multi-view visual fusion in this invention; Figure 2 This is a schematic diagram showing the original image and the feature point detection results; Figure 3 It is a schematic diagram showing the comparison of the two-dimensional coordinates of each feature point; Figure 4 It is a schematic diagram showing the comparison of the three-dimensional coordinates of each feature point. Detailed Implementation

[0022] This invention discloses a real-time intelligent detection method for curtain wall installation accuracy based on multi-view visual fusion, referring to... Figure 1 This includes steps S1-S4: S1: Obtain the initial two-dimensional feature point set of the curtain wall panel based on the multi-view image source of the curtain wall installation process; obtain the initial calibration matrix and camera focal length of the calibration camera group; establish epipolar geometric relationship based on the initial calibration matrix and construct multi-view geometric constraints of the calibration camera group; use the multi-view geometric constraints to obtain the initial visual three-dimensional spatial coordinates, initial physical distance and spatial incident angle of any feature point.

[0023] It should be noted that the construction site environment for large-scale curtain walls is complex. Due to obstructed views and large spatial spans, traditional contact-based or single-viewpoint measurements cannot capture the overall installation posture characteristics. Furthermore, curtain wall installation operations have strict spatial logic, with the seams and edges between adjacent panels serving as critical evaluation benchmarks. Therefore, this invention utilizes a calibration camera group deployed at the construction site to simultaneously acquire multi-dimensional data, extracting image edge and corner features. By leveraging the camera's calibration parameters, a preliminary mapping from two-dimensional image pixels to three-dimensional physical space is achieved. This captures spatiotemporal multi-dimensional data consistent with the real engineering environment at the data source, providing a data foundation for subsequent large-scale perspective error analysis and dynamic spatial coordinate correction.

[0024] Specifically, the initial set of two-dimensional feature points of the curtain wall panel is obtained from a multi-view image source during the curtain wall installation process, including: Multi-view image sources of the curtain wall installation process are collected in real time and synchronously by a set of calibrated cameras deployed at the construction site.

[0025] It should be noted that curtain wall panels are typically composed of a rectangular frame and internal glass, and have relatively obvious straight line boundaries and corner geometric features. Therefore, this invention uses classic image processing operators to quickly and stably extract these high-frequency geometric nodes from the complex construction background and use them as basic primitives for three-dimensional reconstruction.

[0026] An edge detection algorithm is used to perform Gaussian filtering and gradient calculation on the multi-view image source to extract the geometric boundary contour pixels of the curtain wall panel. A corner detection operator is then used to extract contour intersections, and all extracted contour intersections are used as the initial set of two-dimensional feature points for the curtain wall panel. For example, the edge detection algorithm uses the Canny operator.

[0027] It should be noted that, as Figure 2 The image shows the original image and the detected feature points, illustrating the original image of the area to be detected on the curtain wall and the initial two-dimensional feature points detected.

[0028] Preferably, the initial calibration matrix and camera focal length of the calibration camera group are obtained; based on the initial calibration matrix, the epipolar geometric relationship is established, and the multi-view geometric constraints of the calibration camera group are constructed, including: Obtain the initial calibration matrix of the calibration camera group and the camera focal length of the calibration camera group that are pre-stored in the control system.

[0029] Based on the intrinsic and extrinsic parameters of the initial calibration matrix of the calibration camera group, the epipolar geometric relationship between multiple camera views is established, and multi-view geometric constraints for feature point space mapping of the calibration camera group are constructed, including: It should be noted that constructing multi-view geometric constraints based on epipolar geometry theory is a multi-view... Figure 3 The core foundational technology for 3D reconstruction is to construct a basic matrix using camera intrinsic and extrinsic parameters, establish projection constraints for the same feature point in different camera views, narrow the feature matching range, and ensure the accuracy of 3D reconstruction.

[0030] For example, any two camera views in the calibrated camera group are referred to as the first camera view and the second camera view, respectively.

[0031] Obtain the intrinsic parameter matrix of the first camera view. Intrinsic parameter matrix of the second camera view Obtain the relative rotation matrix from the first camera view to the second camera view. Translation vector .

[0032] To standardize the cross product of spatial vectors into matrix multiplication, a translation vector is constructed. An antisymmetric matrix, satisfying the expression: ; In the formula, Represents the translation vector The antisymmetric matrix; , , They represent translation vectors respectively. Along in the three-dimensional physical coordinate system axis, axis, Displacement components of the axis.

[0033] Based on the antisymmetric matrix, the relative rotation matrix, and two intrinsic parameter matrices, the fundamental matrix is ​​derived and calculated, satisfying the expression: ; In the formula, Represents the fundamental matrix; This represents the inverse of the intrinsic parameter matrix of the second camera view; Represents the translation vector The antisymmetric matrix; This represents the relative rotation matrix from the first camera view to the second camera view; This represents the inverse of the intrinsic parameter matrix of the first camera view.

[0034] Finally, multi-view geometric constraints for the calibration camera group are constructed based on the fundamental matrix, satisfying the epipolar constraint expression: ; In the formula, Represents the fundamental matrix; , This represents the two-dimensional homogeneous coordinate vector of the same physical feature point in the first and second camera views. This is the transpose symbol.

[0035] In the formula, This study comprehensively reflects the optical coplanarity constraint logic that must be satisfied when multiple camera views observe the same physical space point. Specifically, given a feature point in the first camera view, the corresponding feature point in the second camera view lies on an epipolar line determined by a fundamental matrix. It establishes the epipolar geometric relationship between multiple camera views, eliminates disordered divergence in the two-dimensional image matching process, and provides spatial theoretical constraints for subsequent nonlinear triangular space reconstruction.

[0036] Preferably, by utilizing multi-view geometric constraints, the initial visual 3D spatial coordinates, initial physical distance, and spatial incident angle of any feature point are obtained, including: The initial two-dimensional feature point set of the curtain wall panel is substituted into the multi-view geometric constraints of the calibration camera group to perform nonlinear triangulation based on ray intersection, and the three-dimensional spatial vector of each initial two-dimensional feature point in the physical coordinate system is obtained as the initial visual three-dimensional spatial coordinate of the corresponding feature point.

[0037] Based on the initial visual 3D spatial coordinates, the absolute spatial straight-line distance from the calibrated camera group to the corresponding feature point is obtained as the initial physical distance of the corresponding feature point; the angle between the vector corresponding to the camera optical axis and the theoretical normal vector of the curtain wall plane is extracted as the spatial incident angle of the corresponding feature point.

[0038] This completes the data acquisition of the multi-view images and the initial physical parameter mapping of arbitrary feature points.

[0039] S2: Based on the initial physical distance of any feature point, the camera focal length, and the spatial incident angle, obtain the viewpoint offset sensitivity of any feature point; extract the continuous physical seam pixel path from the multi-view image source, and use the edge gradient operator to obtain the two-dimensional coordinate offset direction gradient of any feature point; combine the viewpoint offset sensitivity of any feature point and the two-dimensional coordinate offset direction gradient to perform integral calculation on the continuous physical seam pixel path to obtain the mesh topology deformation energy of any feature point.

[0040] It should be noted that in large-scale detection scenarios, there is an inherent optical perspective error amplification effect when 2D image pixels are mapped to 3D space. Optical perspective laws indicate that the farther the observation distance and the more oblique the line of sight, the more easily tiny pixel errors are distorted into huge physical deviations in 3D space. Furthermore, the curtain wall panel itself possesses strict rigid mesh properties, making it difficult to identify false deformations caused by repetitive glass textures using a single perspective index. Therefore, this invention first calculates the sensitivity to viewpoint shift caused by perspective effects, and then combines the local gradient integral of the physical seams to construct a mesh topology deformation energy that can truly reflect the degree of image matching distortion.

[0041] Specifically, based on the initial physical distance of any feature point, the camera focal length, and the spatial incident angle, the viewpoint offset sensitivity of any feature point is obtained, including: It should be noted that, based on the principle of optical perspective, perspective error is positively correlated with observation distance and the angle of inclination of the line of sight. Considering that distance and incident angle can determine the severity of perspective error, when the camera's line of sight is not perpendicular to the curtain wall plane, tiny shifts in the image will be nonlinearly amplified in physical space. Therefore, this invention obtains the perspective error sensitivity of each feature point by analyzing the correlation between distance, focal length, and incident angle.

[0042] Obtain the initial physical distance of any feature point, the camera focal length of the calibrated camera group, and the spatial incident angle of any feature point.

[0043] The viewpoint offset sensitivity of any feature point satisfies the expression: ; In the formula, Indicates the first The viewpoint offset sensitivity of each feature point is dimensionless. Indicates the first The initial physical distance between each feature point; Indicates the camera focal length of the calibrated camera group; Indicates the first The spatial incident angle of each feature point; This represents the cosine function.

[0044] In the formula, The distance scale scaling factor of the camera reflects the basic error magnification ratio caused by the increase in observation depth; This represents the penalty for tilting the line of sight. The penalty is minimal when the line of sight is perfectly vertical, and the more tilted the line of sight, the greater the penalty. This comprehensively reflects that as the initial physical distance increases or the spatial incident angle increases, the sensitivity to perspective shift increases, demonstrating the potential risk of perspective error divergence.

[0045] Preferably, the continuous physical seam pixel path is extracted from the multi-view image source, and the two-dimensional coordinate offset direction gradient of any feature point is obtained using the edge gradient operator, including: Extract the curtain wall panel joint lines that are directly connected to any feature point from the multi-view image source, define them as continuous physical joint pixel paths of the feature points, and obtain the total arc length of the continuous physical joint pixel paths.

[0046] The Sobel edge gradient operator is used to calculate the directional gradient of all pixels on the continuous physical seam pixel path of any feature point in the image coordinate system, which is used as the two-dimensional coordinate offset directional gradient of the feature point.

[0047] Preferably, the grid topology deformation energy of any feature point is obtained by integrating the viewpoint offset sensitivity and the gradient of the two-dimensional coordinate offset direction on the continuous physical seam pixel path, including: It should be noted that perspective sensitivity only represents the potential probability of error occurrence; in actual multi-view matching processes, the repetitive glass texture on the curtain wall surface can easily cause nonlinear distortion and offset in pixel matching at the seam positions. Since real curtain wall seams are straight, smooth, and rigid structures, any abnormal distortions appearing in the image can be identified as matching errors. In large-scale on-site detection scenarios, perspective error and gradient distortion will jointly affect matching accuracy; this invention quantifies the degree of distortion of seam lines based on the well-known gradient statistics principle in the field of image processing, and integrates perspective sensitivity as a weight to jointly represent the error, overcoming the technical defects of traditional gradient statistics methods that ignore perspective interference and are prone to causing matching distortion in large-scale scenes.

[0048] The viewpoint offset sensitivity of any feature point, the continuous physical seam pixel path of the feature point, and the two-dimensional coordinate offset direction gradient of the feature point are obtained.

[0049] The mesh topological deformation energy at any feature point satisfies the expression: ; In the formula, Indicates the first The mesh topological deformation energy at each feature point is dimensionless. Indicates the first Sensitivity to viewpoint shift of each feature point; Indicates the first The total number of pixels in the continuous physical seam pixel path of each feature point; This represents the gradient of the two-dimensional coordinate offset direction of the s-th pixel along the continuous physical seam pixel path; This represents the absolute value function.

[0050] In the formula, The average intensity of the seam offset gradient reflects the severity of the nonlinear distortion of the panel's rigid lines in a two-dimensional image. This reflects that the mesh topology deformation energy is jointly triggered by the violent fluctuations in the seam offset gradient and the high viewpoint offset sensitivity. When the mesh topology deformation energy increases, it indicates that the multi-view matching algorithm has fallen into the illusion of repetitive textures, resulting in false topology deformation of the physical mesh.

[0051] At this point, the mesh topology deformation energy of all feature points has been obtained.

[0052] S3: Obtain the standard grid modulus coefficient of the curtain wall panel, perform exponential decay mapping based on the standard grid modulus coefficient and the grid topology deformation energy, and obtain the depth perception alignment compensation factor for any feature point; use the depth perception alignment compensation factor to adaptively weight the initial visual three-dimensional space coordinates and the theoretical three-dimensional space coordinates to obtain the corrected three-dimensional space coordinates of any feature point.

[0053] It should be noted that, in order to correct the spatial coordinates that have undergone topological deformation, this invention converts the deformation energy into dynamically adjustable weights during the 3D dynamic fusion process. If only visual triangulation results are used, the system will output drift points that deviate from the actual physical positions when encountering large-scale perspective and repetitive textures. Therefore, this invention constructs an adaptive fusion architecture that includes a priori architectural model. When feature matching is reliable, the visual dynamic reconstruction results are highly trusted. When severe perspective distortion is detected, the measurement points are constrained to the architectural priori theoretical domain, thus balancing the high sensitivity of capturing small errors with the safety of eliminating extreme outliers.

[0054] Specifically, the standard mesh modulus coefficient of the curtain wall panel is obtained, and an exponential decay mapping is performed based on the standard mesh modulus coefficient and the mesh topological deformation energy to obtain the depth-sensing alignment compensation factor for any feature point, including: It should be noted that, based on the well-known exponential decay weighting mechanism in control theory, this invention achieves adaptive adjustment by using an exponential function to reduce the weight as the error increases. Combined with the rigid reference of the curtain wall building, the weight is physically normalized. The improvement is based on the engineering principle that curtain wall panels are rigid structures, and when the deformation energy exceeds the rigid reference, the visual measurement results will have a large deviation.

[0055] Extract the pre-defined standard grid modular coefficients for the curtain wall panels from the building model data, denoted as . This serves as a benchmark for characterizing the physical rigidity of curtain wall panels in three-dimensional space. For example, Set it to 0.8. It should be noted that if... If the value is too large, such as 10, the system will become insensitive to deformation energy, resulting in an overestimation of the depth sensing alignment compensation factor. This will prevent erroneous 3D matching points from being effectively filtered out, thus contaminating the final detection accuracy. If the value is too small, such as 0.01, the depth perception alignment compensation factor will decay too quickly, causing normal small visual observation fluctuations to be assigned a weight close to zero. This will cause the system to rely too much on the theoretical model and reduce the real-time performance of machine vision detection.

[0056] Obtain the mesh topology deformation energy and the standard mesh modulus coefficient of the curtain wall panel at any feature point.

[0057] The depth-aware alignment compensation factor for any feature point satisfies the expression: ; In the formula, Indicates the first A dimensionless depth-sensing alignment compensation factor for each feature point; Indicates the first Mesh topological deformation energy at each feature point; This represents the standard grid module coefficient of the curtain wall panel; This represents the natural exponential function.

[0058] In the formula, This represents the normalized relative deviation of the mesh topology deformation energy from the physical rigidity reference of the curtain wall; This indicates a decay weighting mechanism constructed using a negative exponential function. When the deformation energy is large, the depth perception alignment compensation factor approaches zero, suppressing visually contaminated matching points; when the deformation energy is small, the depth perception alignment compensation factor approaches... The weight allocation of normal matching points is preserved in subsequent coordinate calculations.

[0059] Preferably, the initial visual 3D spatial coordinates and theoretical 3D spatial coordinates are adaptively weighted using a depth-sensing alignment compensation factor to obtain the corrected 3D spatial coordinates of any feature point, including: It should be noted that dynamic correction of 3D spatial coordinates is essentially a dynamic centroid allocation process of multi-source spatial measurement information. Visual measurement can capture minute displacement deviations during actual installation, while the building BIM theoretical model provides a safe physical baseline. Considering the well-known technology of multi-source information fusion, dynamically adjusting the weighting of reliable information to have a high weight and unreliable information to a low weight through weighted fusion is a common method for coordinate correction in industrial measurement. Therefore, this invention uses a depth-sensing alignment compensation factor as a weight to assign an adaptive ratio to the visual 3D coordinates and the architectural theoretical constraint coordinates, so that the final output 3D coordinates can maintain the sensitivity of on-site detection without producing dangerous divergence offsets.

[0060] Obtain the theoretical three-dimensional spatial coordinates of any feature point pre-extracted from the building BIM model.

[0061] The corrected 3D spatial coordinates of any feature point satisfy the expression: ; In the formula, Indicates the first The corrected three-dimensional spatial coordinates of each feature point; Indicates the first Depth-aware alignment compensation factor for each feature point; Indicates the first The initial visual three-dimensional space coordinates of each feature point; Indicates the first The theoretical three-dimensional spatial coordinates of each feature point.

[0062] In the formula, This indicates the proportion of the purely visual dynamic 3D measurement components in the final correction coordinates; This represents the proportion of complementary support in the final correction coordinates for the theoretically rigid prior constraint components. This comprehensively reflects the dynamic correction mechanism implemented based on the current reliability of visual measurement. When the depth perception alignment compensation factor approaches 1, the system fully amplifies the initial visual 3D spatial coordinates to capture the actual installation deviation. When the depth perception alignment compensation factor approaches 0, the system intercepts and suppresses the divergence error of visual measurement, anchoring the final coordinates within the rigid theoretical space.

[0063] At this point, the corrected three-dimensional spatial coordinates of any feature point have been obtained.

[0064] S4: Obtain the preset installation tolerance of the curtain wall panel; calculate the distance between the corrected three-dimensional spatial coordinates and the theoretical three-dimensional spatial coordinates to obtain the spatial absolute deviation of any feature point; generate an installation accuracy test report of the curtain wall panel based on the comparison result between the spatial absolute deviation and the preset installation tolerance, and complete the intelligent detection of the installation accuracy of the curtain wall.

[0065] It should be noted that national or construction industry standards usually strictly stipulate the maximum allowable deviation for curtain wall installation. Transforming this standard into the judgment threshold for the system execution enables direct connection between the machine vision bottom-level inspection results and the upper-level engineering acceptance standards, thereby automatically and efficiently determining whether the current installation work is qualified.

[0066] Specifically, obtain the preset installation tolerances of the curtain wall panels; calculate the distance between the corrected 3D spatial coordinates and the theoretical 3D spatial coordinates to obtain the absolute spatial deviation of any feature point, including: Set the preset installation tolerance for the curtain wall panels, denoted as . This is used to define the maximum permissible deviation in three-dimensional space during the standard installation process of curtain wall panels. For example, Set to 5mm. It should be noted that if... If the value is too large, such as 20mm, the system's judgment standard will be too lenient, causing tilted panels with serious safety hazards to be missed and allowed to pass; if If the value is too small, such as 0.1mm, it will exceed the mechanical manufacturing and processing limits of large building components, causing the system to output false alarms and making it impossible for construction site operations to proceed normally.

[0067] Obtain the corrected 3D spatial coordinates of any feature point and the theoretical 3D spatial coordinates of any feature point.

[0068] Obtain the Euclidean distance between the corrected 3D spatial coordinates of any feature point and the theoretical 3D spatial coordinates of the feature point, and denote it as the absolute spatial deviation of the feature point.

[0069] Preferably, an installation accuracy test report for the curtain wall panels is generated based on the comparison between the absolute spatial deviation and the preset installation tolerance, thus completing the intelligent detection of the curtain wall installation accuracy, including: The spatial absolute deviation of each feature point is extracted one by one and compared with the preset installation tolerance of the curtain wall panel. If the spatial absolute deviation of any feature point is greater than the preset installation tolerance of the curtain wall panel, it is determined and electronically recorded that the curtain wall panel area associated with the feature point has an out-of-tolerance alarm state; if the spatial absolute deviation of any feature point is less than or equal to the preset installation tolerance of the curtain wall panel, it is determined that the curtain wall panel area associated with the feature point is qualified.

[0070] The system summarizes the status comparison results of all feature points and the specific deviation values, generates and outputs the installation accuracy test report of the curtain wall panel in real time according to the predetermined layout format, and completes the automated and intelligent inspection of the overall acceptance process.

[0071] It should be noted that, as Figure 3The image shows a comparison of the two-dimensional coordinates of each feature point, illustrating the original visual coordinates, the corrected coordinates, and the theoretical coordinates of each feature point in the two-dimensional coordinate plane. Figure 4 The diagram shows the comparison of the three-dimensional coordinates of each feature point, illustrating the spatial relationship between the original visual coordinates, the corrected coordinates, and the theoretical coordinates of each feature point in the three-dimensional coordinate plane.

[0072] This completes the intelligent detection process for curtain wall installation accuracy based on multi-view visual fusion.

[0073] While various embodiments of the invention have been shown and described in this specification, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and essence of the invention.

Claims

1. A real-time intelligent detection method for curtain wall installation accuracy based on multi-view visual fusion, characterized in that, include: Acquire multi-view image sources of the curtain wall installation process and extract the initial two-dimensional feature point set of each image; obtain the initial calibration matrix and camera focal length of the calibration camera group, and construct multi-view geometric constraints based on the initial calibration matrix; use the multi-view geometric constraints to perform nonlinear triangulation on the initial two-dimensional feature point set to obtain the initial visual three-dimensional spatial coordinates of each feature point; Obtain the initial physical distance and spatial incident angle of each feature point; calculate the viewpoint offset sensitivity of each feature point, which is positively correlated with the ratio of the initial physical distance to the camera focal length and negatively correlated with the cosine value of the spatial incident angle; extract continuous physical seam pixel paths along the physical seams formed by adjacent feature points in the multi-view image source, and calculate the two-dimensional coordinate offset direction gradient of all pixels on the continuous physical seam pixel path; use the product of the viewpoint offset sensitivity and the average gradient intensity of the continuous physical seam pixel path as the mesh topology deformation energy. Obtain the theoretical three-dimensional spatial coordinates of each feature point in the preset building model; based on the mesh topological deformation energy, the pre-acquired standard mesh modulus coefficient of the curtain wall panel, and the initial visual three-dimensional spatial coordinates, obtain the corrected three-dimensional spatial coordinates; based on the distance between the corrected three-dimensional spatial coordinates and the theoretical three-dimensional spatial coordinates, complete the detection of the curtain wall installation accuracy.

2. The real-time intelligent detection method for curtain wall installation accuracy based on multi-view visual fusion according to claim 1, characterized in that, The extraction of the initial two-dimensional feature point set includes: The edge detection algorithm is used to perform Gaussian filtering smoothing and gradient calculation on the multi-view image source to extract the geometric boundary contour pixels of the curtain wall panel. The corner detection operator is combined to extract the contour intersection points. All the extracted contour intersection points are used as the initial two-dimensional feature point set of the curtain wall panel.

3. The real-time intelligent detection method for curtain wall installation accuracy based on multi-view visual fusion according to claim 1, characterized in that, The step of constructing multi-view geometric constraints based on the initial calibration matrix includes: designating any two camera views in the multi-view image source as the first camera view and the second camera view, respectively; obtaining the intrinsic parameter matrices of the first camera view and the second camera view; and obtaining the relative rotation matrix from the first camera view to the second camera view. Translation vector Construct the antisymmetric matrix of the translation vector; the fundamental matrix satisfies the expression: The multi-view geometric constraints satisfy the epipolar constraint expression: ; In the formula, Represents the fundamental matrix; , The intrinsic parameter matrix represents the first camera view and the second camera view; Represents the translation vector The antisymmetric matrix; , This represents the two-dimensional homogeneous coordinate vector of the same physical feature point in the first and second camera views. This is the transpose symbol.

4. The real-time intelligent detection method for curtain wall installation accuracy based on multi-view visual fusion according to claim 1, characterized in that, The viewpoint offset sensitivity satisfies the expression: ； In the formula, Indicates the first Sensitivity to viewpoint shift of each feature point; Indicates the first The initial physical distance between each feature point; Indicates the camera focal length of the calibrated camera group; Indicates the first The spatial incident angle of each feature point; This represents the cosine function.

5. The real-time intelligent detection method for curtain wall installation accuracy based on multi-view visual fusion according to claim 1, characterized in that, The mesh topological deformation energy satisfies the expression: ； In the formula, Indicates the first Mesh topological deformation energy at each feature point; Indicates the first Sensitivity to viewpoint shift of each feature point; Indicates the first The total number of pixels in the continuous physical seam pixel path of each feature point; This represents the gradient of the two-dimensional coordinate offset direction of the s-th pixel along the continuous physical seam pixel path; This represents the absolute value function.

6. The real-time intelligent detection method for curtain wall installation accuracy based on multi-view visual fusion according to claim 1, characterized in that, The corrected three-dimensional spatial coordinates satisfy the expression: ； In the formula, Indicates the first The corrected three-dimensional spatial coordinates of each feature point; Indicates the first Depth-aware alignment compensation factor for each feature point; Indicates the first The initial visual three-dimensional space coordinates of each feature point; Indicates the first The theoretical three-dimensional spatial coordinates of each feature point.

7. The real-time intelligent detection method for curtain wall installation accuracy based on multi-view visual fusion according to claim 6, characterized in that, The acquisition of the depth-aware alignment compensation factor includes: The negative natural exponential function value of the ratio of the mesh topological deformation energy of the i-th feature point to the standard mesh modulus coefficient of the curtain wall panel is taken as the i-th feature point. Depth-aware alignment compensation factor for each feature point.

8. The real-time intelligent detection method for curtain wall installation accuracy based on multi-view visual fusion according to claim 1, characterized in that, The initial physical distance is the Euclidean distance between the optical center of the calibration camera group and the initial visual three-dimensional spatial coordinates of the corresponding feature point.

9. The real-time intelligent detection method for curtain wall installation accuracy based on multi-view visual fusion according to claim 1, characterized in that, The spatial incident angle is the angle between the optical axis of the camera that captures the corresponding feature point in the calibration camera group and the normal vector of the theoretical plane of the curtain wall.

10. The real-time intelligent detection method for curtain wall installation accuracy based on multi-view visual fusion according to claim 1, characterized in that, The process of detecting the installation accuracy of the curtain wall based on the distance between the corrected three-dimensional spatial coordinates and the theoretical three-dimensional spatial coordinates includes: Obtain the preset installation tolerance of the curtain wall panel; calculate the Euclidean distance between the corrected three-dimensional spatial coordinates and the theoretical three-dimensional spatial coordinates to obtain the spatial absolute deviation; determine the feature points whose spatial absolute deviation is greater than the preset installation tolerance as feature points with an out-of-tolerance alarm state; highlight the feature points with an out-of-tolerance alarm state and their associated curtain wall panel areas on the real-time monitoring interface; Record the corrected three-dimensional spatial coordinates, theoretical three-dimensional spatial coordinates, and corresponding absolute spatial deviation values ​​of feature points that have out-of-tolerance alarm states to complete the detection of curtain wall installation accuracy.

Images