Curtain wall surface defect intelligent detection method based on deep learning image recognition

Abstract

This invention belongs to the field of image processing and defect detection technology, specifically relating to an intelligent detection method for curtain wall surface defects based on deep learning image recognition. The method includes: acquiring a high-resolution image of the curtain wall and the relative distance for inspection; calculating the global illumination reference, local high-frequency contrast, and initial distortion thickness; calculating optical distortion scores; calculating stress tearing degree; calculating panel fragility scores; calculating enhanced semantic features; generating defect topological patches; obtaining a three-dimensional defect mapping map; and generating an inspection diagnosis report and maintenance work order. This invention solves the technical problems of low accuracy and severe light and shadow artifact interference in existing technologies under strong reflective environments on curtain walls, making it impossible to accurately assess the degree of structural stress tearing.

Description

Technical Field

[0001] This invention relates to the fields of image processing and defect detection technology. More specifically, this invention relates to an intelligent detection method for surface defects in curtain walls based on deep learning image recognition. Background Technology

[0002] With the acceleration of modern urbanization, building curtain walls, such as glass curtain walls, aluminum panel curtain walls, and stone curtain walls, are being used in the external envelope of high-rise buildings. However, curtain walls are subjected to wind loads, solar temperature differences, and structural deformation over long periods of time. The sealing strips on their surfaces are prone to aging and cracking, and the glass or metal panels may also develop hidden micro-cracks or stress deformation. If these surface defects are not detected in time, they can not only lead to serious water and air leakage problems, but may even cause catastrophic safety accidents such as panel detachment.

[0003] Currently, the industry has begun to introduce drones equipped with cameras for curtain wall surface inspection. However, existing image recognition methods based on conventional deep learning, such as traditional CNN or YOLO algorithms, face significant challenges in curtain wall scenarios. Due to the strong specular and diffuse reflection characteristics of glass and metal materials, outdoor solar glare, cloud reflections, and the reflections of surrounding buildings easily create high-frequency optical artifacts in the images.

[0004] Furthermore, traditional deep learning networks are black boxes, lacking explicit constraints on physical optical properties. They are prone to misjudging high-gradient edges caused by reflections as real cracks, or misjudging shadows as adhesive strips falling off, resulting in a high false alarm rate. At the same time, simple two-dimensional image recognition lacks the perception of spatial deformation depth, making it difficult to assess the severity of defects. Summary of the Invention

[0005] To address the technical problems of low accuracy and severe light and shadow artifact interference in deep learning recognition under strong reflective conditions of curtain walls, which prevent accurate assessment of structural stress and tearing, this invention provides an intelligent detection method for curtain wall surface defects based on deep learning image recognition, including: By controlling an inspection drone to perform close-range flight scanning and simultaneously activating visual and laser ranging modules, high-definition images of the curtain wall and inspection relative distances are obtained. Based on the brightness channel and gradient extreme value features extracted from the deep learning backbone network, combined with the inspection relative distance, the global illumination benchmark, local high-frequency contrast, and initial distortion thickness are calculated. Based on the nonlinear suppression relationship between global illumination and local contrast, combined with the initial distortion thickness, the optical distortion score is calculated. Based on the exponential amplification mechanism of neighborhood distortion in microscopic deformation, combined with the optical distortion score, the stress tearing degree is calculated. Based on the nonlinear contrast mapping mechanism of local displacement stress relative to the global background, combined with the stress tearing degree, the panel fragility score is calculated. Based on the complementary mechanism of physical anti-shadow features and AI semantic recognition, combined with the panel fragility score, enhanced semantic features are calculated. Based on the enhanced semantic features, a region growing algorithm is used to generate defect topological patches. The defect topological patches are mapped and pseudo-color rendered to obtain a three-dimensional defect mapping map. Based on the physical features of the defect topological patches in the three-dimensional defect mapping map, an inspection diagnosis report and maintenance work order are generated.

[0006] This invention effectively reduces the impact of environmental factors on inspection results and minimizes the possibility of misjudgment by progressively separating environmental interference from actual changes, amplifying actual damage characteristics, and combining overall background risk assessment. Simultaneously, the solution maps inspection results to specific locations on actual buildings, forming a complete process from data acquisition to processing recommendations, improving the reliability and practicality of the inspection results and reducing reliance on human experience. The seamless integration of each stage of the solution allows it to adapt to various outdoor working environments, providing a more stable and valuable processing method for building facade inspection and offering a referable approach for similar inspection scenarios.

[0007] Preferably, acquiring high-resolution images of the curtain wall and the relative distance for inspection includes: The inspection drone is controlled to perform a close-up flight scan of the target building's curtain wall. Using an onboard high-definition vision sensor and laser ranging module, a high-definition image stream of the curtain wall surface is acquired in real time and recorded as a high-definition image of the curtain wall. At the same time, the vertical spatial distance from the center of the drone's lens to the curtain wall surface when the high-definition image of the curtain wall is captured is recorded and recorded as the inspection relative distance.

[0008] Preferably, the calculation of the global illumination reference, local high-frequency contrast, and initial distortion thickness includes: A high-resolution image of the curtain wall is input into the feature extraction network of a pre-trained deep learning backbone network. The average feature value of the brightness channel of the high-resolution image of the curtain wall is obtained and normalized, and recorded as the global illumination baseline. The local pixel gradient extrema in the shallow feature map of the network are extracted and normalized to the maximum and minimum values. The result is recorded as the local high-frequency contrast. At the same time, the camera focal length parameters of the airborne high-resolution vision sensor are obtained from the database. Based on the pinhole imaging geometry principle of the camera, combined with the camera focal length parameters and the relative distance of the inspection, the pixel displacement of the two-dimensional image is proportionally converted into the actual absolute length in physical space and then normalized, and recorded as the initial distortion thickness.

[0009] Preferably, the optical distortion component satisfies the following expression: ; In the formula, Indicates optical distortion. Indicates local high-frequency contrast; Indicates the global illumination reference, which is greater than 0; Indicates the initial distortion thickness; It is the natural logarithm function.

[0010] This invention processes the underlying feature data through a specific calculation method, effectively suppressing false image features caused by curtain wall reflections and shadows, and restoring the deformation features of the curtain wall caused by actual physical damage.

[0011] Preferably, the stress tear strength satisfies the following expression: ; In the formula, Indicates the degree of stress tearing; The optical distortion component is greater than 0; The distortion range; It is a natural exponential function.

[0012] This invention combines distortion feature data of curtain walls with in-depth calculations to effectively distinguish between real cracks on the curtain wall surface and irrelevant traces such as water stains and dirt, allowing for clear identification of actual structural damage points. By magnifying the feature differences in crack areas, even previously hidden micro-cracks can be accurately captured, avoiding situations where these minor damage features are overlooked due to their inconspicuousness.

[0013] Preferably, calculating panel vulnerability scores includes: The stress tear strength calculated for all pixels is summed and divided by the total number of pixels to obtain the global average tear strength, which characterizes the overall stress benchmark level of the curtain wall; the panel fragility score satisfies the following expression: ; In the formula, This indicates the panel's fragility. This refers to the stress tear strength; The mean of the tearing effect across the entire domain is greater than 0; It is the hyperbolic tangent function.

[0014] This invention combines global curtain wall inspection data to analyze local damage characteristics, automatically filtering out risk-free background interference and focusing on curtain wall damage areas with structural collapse risks. Through non-linear calculations, it highlights high-risk fracture areas, effectively distinguishing the failure risks at different locations within the curtain wall and more clearly determining the degree of danger in each part.

[0015] Preferably, calculating enhanced semantic features includes: The original semantic feature matrix of the deep output of the deep learning backbone network is extracted. After the panel fragile submatrix is ​​scaled and normalized, the Hadamard product operation is performed on the original semantic feature matrix element by element using the tensor multiplication rule to complete the spatial fusion of physical features and semantic features, and output the enhanced semantic features enhanced by physical features.

[0016] This invention combines physically calculated curtain wall risk data with deep learning-based image semantic analysis, guiding the intelligent analysis system to focus on the actual defect areas of the curtain wall, thus avoiding the influence of false light and shadow interference information.

[0017] Preferably, generating defective topological patches includes: Perform on enhanced semantic features The classification activation operation is performed, and the corresponding pixel value is mapped to the defect probability in the range [0,1]. The probability is compared with the confidence threshold preset by the system, and seed pixels with defect probabilities greater than the confidence threshold are extracted and recorded as defect pixels. The eight-neighbor morphological region growing algorithm is used to merge spatially adjacent defect pixels to generate defect topological patches with independent contour boundaries.

[0018] Preferably, obtaining a three-dimensional defect mapping map includes: Retrieve the pre-calibration intrinsic and extrinsic parameter matrix of the vision sensor from the database, denoted as the camera intrinsic and extrinsic parameter matrix; retrieve the curtain wall data of the target building. The document, based on the spatial three-dimensional coordinates obtained by the inspection drone and the camera's intrinsic and extrinsic parameter matrices, uses affine transformation of the coordinate system to inversely project the defect topological patches on the two-dimensional image to... The model's three-dimensional surface is rendered using pseudo-color based on the severity of the defects, generating a globally intuitive three-dimensional defect mapping map.

[0019] Preferably, an inspection and diagnostic report and a maintenance work order are generated, including: The system retrieves pre-set industry safety standard thresholds from the system database as alarm tolerance levels, and traverses the 3D defect mapping map to calculate the actual physical area and maximum depth of each defect topological patch. These values ​​are then logically compared with the alarm tolerance levels. If the actual physical area and maximum depth are greater than the alarm tolerance levels, the system automatically outputs a diagnostic report indicating the exact location and damage type of the defect. Simultaneously, it generates an electronic maintenance order containing suggestions for high-altitude operation routes and the required material specifications, which is then sent to the maintenance unit's terminal.

[0020] The beneficial effects of this invention are as follows: Through an automated processing flow, this invention reduces the workload of manual high-altitude inspections, lowers the safety risks of manual high-altitude operations, and simultaneously improves the coverage and efficiency of inspection work. The detection results and processing suggestions output by the solution can help building management promptly identify potential safety hazards on the facade, take corresponding maintenance measures in advance, reduce the risk of safety accidents caused by facade damage, and ensure the safety of the building during its use. At the same time, the solution can retain complete detection data, providing data support for the full lifecycle maintenance of the building facade, contributing to the standardization and refinement of building operation and maintenance management, and adapting to the operation and maintenance management needs of modern high-rise buildings. Attached Figure Description

[0021] Figure 1 The flowchart of the intelligent detection method for curtain wall surface defects based on deep learning image recognition in this invention is illustrated schematically. Figure 2 This schematically illustrates a spatial clustering diagram of defect topological patches in this invention; Figure 3 This schematic diagram illustrates the curtain wall in the present invention. 3D defect mapping diagram of the model. Detailed Implementation

[0022] This invention discloses an intelligent detection method for curtain wall surface defects based on deep learning image recognition, referring to... Figure 1 This includes steps S1-S4: S1: By controlling the inspection drone to perform close-range flight scanning and simultaneously activating the visual and laser ranging modules, high-definition images of the curtain wall and the relative distance of the inspection are obtained.

[0023] It should be noted that acquiring high-quality images with spatial reference coordinates is the starting point for intelligent curtain wall detection. When drones fly in outdoor gusts of wind, fluctuations in the shooting distance will change the physical scale of the image. Therefore, simultaneously acquiring the image and its corresponding spatial distance is a fundamental condition for subsequent physical evaluation of image features.

[0024] Specifically, by controlling the inspection drone to perform close-range flight scanning and simultaneously activating the visual and laser ranging modules, high-definition images of the curtain wall and relative distances for inspection are obtained, including: The inspection drone is controlled to perform a close-up flight scan of the target building's curtain wall. Using an onboard high-definition vision sensor and laser ranging module, a high-definition image stream of the curtain wall surface is acquired in real time and recorded as a high-definition image of the curtain wall. At the same time, the vertical spatial distance from the center of the drone's lens to the curtain wall surface when the high-definition image of the curtain wall is captured is recorded and recorded as the inspection relative distance.

[0025] Thus, a high-resolution image of the curtain wall representing its visual state and the relative distance of the inspection representing its spatial scale were obtained.

[0026] S2: Based on the brightness channel and gradient extreme value features extracted by the deep learning backbone network, combined with the inspection relative distance, calculate the global illumination benchmark, local high-frequency contrast and initial distortion thickness; based on the nonlinear suppression relationship of global illumination on local contrast, combined with the initial distortion thickness, calculate the optical distortion score; based on the exponential amplification mechanism of neighborhood distortion in micro-deformation, combined with the optical distortion score, calculate the stress tearing degree.

[0027] It should be noted that directly feeding the original image into a deep network is easily affected by global illumination, such as backlighting and strong glare. This invention introduces a deep learning backbone network, such as the pre-convolutional layer of the ResNet network, to extract the texture contrast of the bottom layer and calculate the distortion thickness by combining it with physical distance. The aim is to decouple the pure optical reference from the physical deformation and provide data raw materials for subsequent filtering of reflection artifacts.

[0028] Specifically, based on the brightness channel and gradient extremum features extracted from the deep learning backbone network, and combined with the inspection relative distance, the global illumination benchmark, local high-frequency contrast, and initial distortion thickness are calculated, including: A high-resolution image of the curtain wall is input into the feature extraction network of a pre-trained deep learning backbone network. The average feature value of the brightness channel of the high-resolution image of the curtain wall is obtained and normalized, and recorded as the global illumination baseline. The local pixel gradient extrema in the shallow feature map of the network are extracted and normalized to the maximum and minimum values. The result is recorded as the local high-frequency contrast. At the same time, the camera focal length parameters of the airborne high-resolution vision sensor are obtained from the database. Based on the pinhole imaging geometry principle of the camera, combined with the camera focal length parameters and the relative distance of the inspection, the pixel displacement of the two-dimensional image is proportionally converted into the actual absolute length in physical space and then normalized, and recorded as the initial distortion thickness.

[0029] Thus, a global illumination baseline and local high-frequency contrast characterizing ambient light and underlying texture were obtained, as well as an initial distortion thickness characterizing the degree of microscopic physical deformation of the curtain wall surface.

[0030] It should be noted that cloud reflections and strong glare on glass curtain walls can produce false high-frequency contrast. This invention aims to strongly strip away the false gradient caused by pure optical reflection by standardizing the local contrast under a global illumination reference and introducing an initial distortion thickness as a logarithmic smoothing penalty term, and accurately restore the distortion caused by real physical unevenness, such as broken rubber strips or cracked glass.

[0031] Preferably, based on the nonlinear suppression relationship between global illumination and local contrast, and combined with the initial distortion thickness, the optical distortion component is calculated, including: Optical distortion satisfies the following expression: ; In the formula, Indicates optical distortion. Indicates local high-frequency contrast; Indicates the global illumination reference, which is greater than 0; Indicates the initial distortion thickness; It is the natural logarithm function.

[0032] In the formula, Normalizing local high-frequency contrast using a global illumination reference, when in a highly reflective area, When the contrast is large, false high-frequency contrasts will be suppressed; The distortion thickness of the physical space is used as a nonlinear adjustment weight, and a logarithmic function is used to smooth it to ensure that the gain is only generated for defects with real physical thickness changes. It achieves accurate extraction of real physical distortion against a highly reflective curtain wall background.

[0033] For example, if =45、 =120、 =0.15, calculated as follows This indicates that the invention effectively suppresses high-gradient artifacts caused by strong reflections, as shown in the calculation results. Keep three decimal places.

[0034] It should be noted that optical distortion analysis is based on the classic contrast normalization model and optical imaging distortion correction theory in the field of machine vision defect detection. It is a method for removing ambient light interference and extracting true physical deformation in the detection of highly reflective surface defects. For highly reflective glass and metal curtain wall surfaces, fluctuations in global illumination intensity directly change the presentation effect of local texture contrast. Normalizing local high-frequency contrast with a global illumination benchmark can adapt to complex outdoor lighting scenarios such as backlighting and strong glare, eliminating the interference of ambient light changes on defect feature extraction. The reflections on the curtain wall surface only have changes in grayscale gradient at the optical level, without actual physical spatial concavity and convexity deformation. This invention introduces the initial distortion thickness, which characterizes the degree of physical deformation, into a logarithmic weighting term, realizing the physical constraint that the higher the initial distortion thickness, the stronger the gain effect on the real defect. This fits the characteristic difference law between real deformation and false artifacts in optical imaging, effectively removing the false gradient caused by pure optical reflection. Curtain wall inspection scenarios naturally involve random fluctuations in lighting conditions. These environmental fluctuations are fundamentally different from actual structural defects. Optical distortion analysis, through the dual constraints of global normalization and physical deformation weighting, confines the numerical value of environmental interference to an extremely low range, amplifying only the effective distortion caused by real physical damage, thus adapting to the actual inspection scenarios of drone outdoor curtain wall inspection.

[0035] Thus, an optical distortion score for evaluating the unevenness of a real physical surface was obtained.

[0036] It should be noted that real damage to curtain wall materials, such as aluminum plate tearing or glass shattering, exhibits a sharp deformation gradient at the microscopic level. This invention introduces an exponential amplification mechanism, which combines the distortion range in the neighborhood with the optical distortion score. The aim is to widen the score gap between real cracks and smooth water stains and dirt through the exponential mapping effect, thereby achieving precise enhancement of stress concentration areas.

[0037] Preferably, based on the exponential amplification mechanism of neighborhood distortion in microscopic deformation, and combined with optical distortion analysis, the stress tearing degree is calculated, including: Slide a preset window of length X within the computational domain to obtain the difference between the maximum and minimum values ​​of the optical distortion score at the current point in the neighborhood, which is denoted as the distortion score range.

[0038] The stress-tear strength satisfies the following expression: ; In the formula, Indicates the degree of stress tearing; The optical distortion component is greater than 0; The distortion range; It is a natural exponential function.

[0039] In the formula, The severity of the distortion gradient within the neighborhood was assessed. When the local range is large while the underlying distortion is relatively stable, it indicates the presence of a sharp geometric fault. The function exponentially amplifies this local fault; finally, it multiplies it with the intrinsic optical distortion component, precisely pinpointing the hidden tear point in the high-stress structure.

[0040] For example, if =0.08、 =0.24, calculated as follows This indicates that, compared to smooth areas such as water stains, the stress-tear intensity of this invention increases by tens of times, according to the calculation results. Round to two decimal places.

[0041] It should be noted that the stress tear strength is constructed based on the classic gradient mutation enhancement model in the field of structural non-destructive testing and the stress concentration theory in materials mechanics. It is an improved method for distinguishing between gentle surface interference and sharp structural damage in the identification of microcracks in brittle materials. For glass-aluminum panel curtain wall materials, real structural damage exhibits sharp deformation gradients at the microscopic level, while surface interference such as water stains and dirt only shows gentle gray-scale gradations. The ratio of the neighborhood distortion range to the basic distortion score characterizes the intensity of local deformation, which can accurately identify the stress concentration characteristics at microcracks and eliminate the interference of gentle surface interference on damage identification. Hidden microcracks on the curtain wall surface only have localized small deformations, and direct detection is easily drowned out by background noise. This invention uses a natural exponential function to nonlinearly amplify the local deformation gradient, realizing a mapping relationship that the more intense the deformation gradient, the stronger the enhancement effect on damage characteristics. This aligns with the deformation distribution law at stress concentration points in materials mechanics and effectively amplifies the characteristic differences of hidden microcracks. In curtain wall inspection scenarios, the differences between microcracks and surface disturbances are minimal. These differences are fundamentally different from the safety risks of structural damage. The stress tearing degree, through an exponential amplification mechanism, constrains the value of risk-free surface disturbances to an extremely low range, amplifying only the strong stress characteristics caused by structural tearing, thus meeting the risk identification needs of curtain wall safety inspection.

[0042] Thus, the stress tear strength, which characterizes the physical damage strength of the curtain wall panel, was obtained.

[0043] It should be noted that the length of the sliding setting window is The pixel block is dynamically matched with the camera resolution based on the lower limit of the physical scale of the defect in the curtain wall to be detected, so as to ensure that the window can just cover the local high-frequency distortion range of a microcrack.

[0044] S3: Based on the nonlinear contrast mapping mechanism of local displacement stress relative to the global background, combined with stress tearing degree, calculate panel vulnerability score; based on the complementary mechanism of physical anti-light and shadow characteristics and AI semantic recognition, combined with panel vulnerability score, calculate enhanced semantic features.

[0045] It should be noted that the risk level of the curtain wall panel needs to be judged in conjunction with the global background. This invention utilizes the nonlinear activation characteristics of the hyperbolic tangent function to polarize and compare the local tearing degree against the background of the global mean. This mechanism aims to automatically filter out background noise below the mean and saturate and map out the outlier targets that truly have the risk of structural detachment.

[0046] Specifically, based on the nonlinear contrast mapping mechanism of local displacement stress relative to the global background, and combined with stress tearing degree, the panel vulnerability component is calculated, including: The stress tearing degree calculated for all pixels was summed and divided by the total number of pixels to obtain the global average tearing value, which represents the overall stress benchmark level of the curtain wall.

[0047] Panel fragility satisfies the following expression: ; In the formula, This indicates the panel's fragility. This refers to the stress tear strength; The mean of the tearing effect across the entire domain is greater than 0; It is the hyperbolic tangent function.

[0048] In the formula, By using the global mean as the base background, the local stress tearing is magnified quadratically, highlighting the more dangerous fracture areas. By introducing a nonlinear saturation feature, a positive gain is given when the local tearing degree is higher than the mean, and when it is lower than the mean, it is quickly suppressed to tend to be negative or zero, thus realizing nonlinear screening of true and false defects.

[0049] For example, if =1.15, =0.15, calculated as follows This indicates that the panel fragility of the present invention effectively establishes a serious structural defect at that location. Round to two decimal places.

[0050] It should be noted that the panel vulnerability assessment is based on the classic outlier significance evaluation model and structural failure probability analysis theory in the field of industrial safety risk assessment. It is a method for assessing local failure risk and filtering background interference in building facade safety assessment. For building curtain wall panels, the risk level of local structural damage needs to be judged in conjunction with the overall safety benchmark. The local stress tearing degree is relative using the global tearing average as a benchmark, which is adaptable to curtain wall panels of different materials and specifications, eliminating the interference of overall environmental fluctuations on risk assessment. The stress level in the risk-free background area of ​​the curtain wall panel is basically consistent with the global benchmark, while the stress level in the high-risk damage area is significantly higher than the global benchmark. This invention amplifies the degree of local stress anomaly through the square term and achieves nonlinear screening through the hyperbolic tangent function. This realizes the assessment logic that the higher the local stress is compared to the global benchmark, the stronger the gain effect on failure risk. This aligns with the correlation between local anomalies and overall failure in structural engineering, effectively filtering out interference from risk-free background noise. Curtain wall safety assessment requires accurate differentiation between normal background fluctuations and high-risk damage. These fluctuations are fundamentally different from structural failure risks. Panel fragility assessment uses a combination of square amplification and nonlinear screening to constrain the values ​​of normal background fluctuations to an extremely low range, amplifying only the high-risk characteristics caused by structural damage, thus meeting the risk classification requirements of curtain wall maintenance scenarios.

[0051] Thus, a panel fragility score was obtained for assessing the risk of localized failure of curtain wall panels.

[0052] It should be noted that while physical operators can accurately locate deformations, they cannot understand the category of defects like deep learning networks, such as distinguishing between glass breakage and sealant aging. This invention uses the panel fragility score calculated based on physical logic as a hard attention mask, which is then back-injected into the deep semantic feature map of the deep learning network, aiming to guide the process with knowledge of physics. The network eliminates the need to focus on reflection artifacts, completely resolving the attention distraction problem of deep learning in curtain wall scenarios.

[0053] Preferably, based on the complementary mechanism of physical anti-light and shadow features and AI semantic recognition, and combined with panel vulnerability scores, enhanced semantic features are calculated, including: The original semantic feature matrix of the deep output of the deep learning backbone network is extracted. After the panel fragile submatrix is ​​scaled and normalized, the Hadamard product operation is performed on the original semantic feature matrix element by element using the tensor multiplication rule to complete the spatial fusion of physical features and semantic features, and output the enhanced semantic features enhanced by physical features.

[0054] Thus, enhanced semantic features were obtained to address the attention distraction problem in curtain wall scenarios.

[0055] S4: Based on enhanced semantic features, a region growing algorithm is used to generate defect topology patches; the defect topology patches are mapped and pseudo-color rendered to obtain a three-dimensional defect mapping map; based on the physical features of the defect topology patches in the three-dimensional defect mapping map, an inspection diagnosis report and maintenance work order are generated.

[0056] It should be noted that the high-dimensional abstract signals on the deep learning feature map need to be re-parsed into geometric entities that can be understood by humans or subsequent algorithms. This invention uses a region growing algorithm to perform connected component clustering on the enhanced feature map, aiming to combine scattered defect pixels into planar defect entities with actual physical boundaries, thereby assessing the actual damage scale of the defect.

[0057] Specifically, based on enhanced semantic features, a region growing algorithm is used to generate defective topological patches, including: Perform on enhanced semantic features The classification activation operation is performed, and the corresponding pixel value is mapped to the defect probability in the range [0,1]. The probability is compared with the confidence threshold preset by the system, and seed pixels with defect probabilities greater than the confidence threshold are extracted and recorded as defect pixels. The eight-neighbor morphological region growing algorithm is used to merge spatially adjacent defect pixels to generate defect topological patches with independent contour boundaries.

[0058] It should be noted that, compared to four-neighborhood morphological algorithms, eight-neighborhood algorithms can effectively identify subtle connectivity extending along the diagonal direction, preventing feature breaks in the spatial clustering of minute glass cracks and ensuring the integrity of defect topological patches. The confidence threshold is typically set to... to A positive floating-point number between these values, for example, can be taken as a value in scenarios where the curtain wall is highly reflective and a low false alarm rate is required. .

[0059] It should be noted that, Figure 2 This is a spatial clustering diagram of defect topology patches, illustrating the spatial distribution and patch morphology of three types of defects: glass cracks, panel deformation, and sealant aging. Glass cracks appear as discontinuous patches distributed along the diagonal, panel deformation as large, gradually changing patches in the central region, and sealant aging as multiple discrete small patches in the lower right region. The boundaries and distribution of different patches demonstrate that this solution can integrate dispersed defect signals into entities with actual physical boundaries, facilitating intuitive judgment of the defect's location, extent, and distribution characteristics, and providing a clear visual basis for subsequent assessment of defect impact.

[0060] Thus, defect topological patches with clear geometric contours and category attributes were obtained.

[0061] It should be noted that defects in two-dimensional images must be mapped to specific floors, facades, and panel numbers to guide high-altitude maintenance operations. This invention utilizes computer graphics reverse projection technology to map two-dimensional defects back into the three-dimensional digital model of the building, aiming to completely eliminate the spatial barrier between inspection data and the actual building.

[0062] Preferably, the defect topological patches are mapped and rendered in pseudo-color to obtain a three-dimensional defect mapping map, including: Retrieve the pre-calibration intrinsic and extrinsic parameter matrix of the vision sensor from the database, denoted as the camera intrinsic and extrinsic parameter matrix; retrieve the curtain wall data of the target building. The document, based on the spatial three-dimensional coordinates obtained by the inspection drone and the camera's intrinsic and extrinsic parameter matrices, uses affine transformation of the coordinate system to inversely project the defect topological patches on the two-dimensional image to... The model's three-dimensional surface is rendered using pseudo-color based on the severity of the defects, generating a globally intuitive three-dimensional defect mapping map.

[0063] It should be noted that, Figure 3 For curtain wall The model's 3D defect mapping diagram illustrates the location and depth information of glass cracks, panel deformation, and sealant aging in the building's 3D space. Glass cracks appear as banded structures penetrating multiple panels, panel deformation as bulges or depressions in the central area, and sealant aging as blocky structures in the lower area. The 3D morphology and location of different defects demonstrate that this solution can accurately project 2D detection data into the building's physical space, eliminating spatial discrepancies between the image and the actual structure. This facilitates maintenance personnel in locating the specific floor, facade, and depth of defects, providing precise location guidance for subsequent maintenance work.

[0064] Thus, a three-dimensional defect mapping map accurately located on the actual physical surface of the curtain wall was obtained.

[0065] It should be noted that curtain wall maintenance, such as high-altitude resin replacement and glass replacement, is costly and dangerous, and must be scheduled based on accurate risk assessment. This invention is the final decision-making closed loop of the detection system, which directly transforms the algorithm's computing power into engineering productivity by comparing safety thresholds.

[0066] Preferably, based on the physical characteristics of defect topological patches in the 3D defect mapping map, an inspection diagnosis report and maintenance work order are generated, including: The system retrieves pre-set industry safety standard thresholds from the system database as alarm tolerance levels, and traverses the 3D defect mapping map to calculate the actual physical area and maximum depth of each defect topological patch. These values ​​are then logically compared with the alarm tolerance levels. If the actual physical area and maximum depth are greater than the alarm tolerance levels, the system automatically outputs a diagnostic report indicating the exact location and damage type of the defect. Simultaneously, it generates an electronic maintenance order containing suggestions for high-altitude operation routes and the required material specifications, which is then sent to the maintenance unit's terminal.

[0067] Thus, intelligent detection of surface defects in curtain walls based on deep learning image recognition has been completed.

[0068] While various embodiments of the invention have been shown and described in this specification, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and essence of the invention.

Claims

1. A method for intelligent detection of surface defects in curtain walls based on deep learning image recognition, characterized in that, include: By controlling the inspection drone to perform close-fly scanning and simultaneously activating the visual and laser ranging modules, high-definition images of the curtain wall and the relative distance of the inspection are obtained. Based on the brightness channel and gradient extreme value features extracted by the deep learning backbone network, combined with the inspection relative distance, the global illumination benchmark, local high-frequency contrast and initial distortion thickness are calculated; based on the nonlinear suppression relationship between global illumination and local contrast, combined with the initial distortion thickness, the optical distortion score is calculated. Based on the exponential amplification mechanism of neighborhood distortion in microscopic deformation, combined with optical distortion, the stress tearing degree is calculated. Based on the nonlinear contrast mapping mechanism of local displacement stress relative to the global background, and combined with stress tearing degree, the panel vulnerability score is calculated; based on the complementary mechanism of physical anti-light and shadow characteristics and AI semantic recognition, and combined with the panel vulnerability score, the enhanced semantic features are calculated. Based on enhanced semantic features, a region growing algorithm is used to generate defective topological patches; The defect topological patches are mapped and rendered in pseudo-color to obtain a three-dimensional defect mapping map. Based on the physical characteristics of defect topological patches in the 3D defect mapping map, inspection and diagnosis reports and maintenance work orders are generated.

2. The intelligent detection method for curtain wall surface defects based on deep learning image recognition according to claim 1, characterized in that, The acquisition of high-definition images of the curtain wall and the relative distance for inspection include: The inspection drone is controlled to perform a close-up flight scan of the target building's curtain wall. Using an onboard high-definition vision sensor and laser ranging module, a high-definition image stream of the curtain wall surface is acquired in real time and recorded as a high-definition image of the curtain wall. At the same time, the vertical spatial distance from the center of the drone's lens to the curtain wall surface when the high-definition image of the curtain wall is captured is recorded and recorded as the inspection relative distance.

3. The intelligent detection method for curtain wall surface defects based on deep learning image recognition according to claim 1, characterized in that, The calculation of the global illumination reference, local high-frequency contrast, and initial distortion thickness includes: A high-resolution image of the curtain wall is input into the feature extraction network of a pre-trained deep learning backbone network. The average feature value of the brightness channel of the high-resolution image of the curtain wall is obtained and normalized, and recorded as the global illumination baseline. The local pixel gradient extrema in the shallow feature map of the network are extracted and normalized to the maximum and minimum values. The result is recorded as the local high-frequency contrast. At the same time, the camera focal length parameters of the airborne high-resolution vision sensor are obtained from the database. Based on the pinhole imaging geometry principle of the camera, combined with the camera focal length parameters and the relative distance of the inspection, the pixel displacement of the two-dimensional image is proportionally converted into the actual absolute length in physical space and then normalized, and recorded as the initial distortion thickness.

4. The intelligent detection method for curtain wall surface defects based on deep learning image recognition according to claim 1, characterized in that, The optical distortion component satisfies the following expression: ； In the formula, Indicates optical distortion. Indicates local high-frequency contrast; Indicates the global illumination reference, which is greater than 0; Indicates the initial distortion thickness; It is the natural logarithm function.

5. The intelligent detection method for curtain wall surface defects based on deep learning image recognition according to claim 1, characterized in that, The stress-tear strength satisfies the following expression: ； In the formula, Indicates the degree of stress tearing; The optical distortion component is greater than 0; The distortion range; It is a natural exponential function.

6. The intelligent detection method for curtain wall surface defects based on deep learning image recognition according to claim 1, characterized in that, The calculation panel vulnerability segment includes: The stress tear strength calculated for all pixels is summed and divided by the total number of pixels to obtain the global average tear strength, which characterizes the overall stress benchmark level of the curtain wall; the panel fragility score satisfies the following expression: ； In the formula, This indicates the panel's fragility. This refers to the stress tear strength; The mean of the tearing effect across the entire domain is greater than 0; It is the hyperbolic tangent function.

7. The intelligent detection method for curtain wall surface defects based on deep learning image recognition according to claim 1, characterized in that, The computational enhancement of semantic features includes: The original semantic feature matrix of the deep output of the deep learning backbone network is extracted. After the panel fragile submatrix is ​​scaled and normalized, the Hadamard product operation is performed on the original semantic feature matrix element by element using the tensor multiplication rule to complete the spatial fusion of physical features and semantic features, and output the enhanced semantic features enhanced by physical features.

8. The intelligent detection method for curtain wall surface defects based on deep learning image recognition according to claim 1, characterized in that, The generation of defective topological patches includes: Perform on enhanced semantic features The classification activation operation is performed, and the corresponding pixel value is mapped to the defect probability in the range [0,1]. The probability is compared with the confidence threshold preset by the system, and seed pixels with defect probabilities greater than the confidence threshold are extracted and recorded as defect pixels. The eight-neighbor morphological region growing algorithm is used to merge spatially adjacent defect pixels to generate defect topological patches with independent contour boundaries.

9. The intelligent detection method for curtain wall surface defects based on deep learning image recognition according to claim 1, characterized in that, The process of obtaining the three-dimensional defect mapping map includes: Retrieve the pre-calibration intrinsic and extrinsic parameter matrix of the vision sensor from the database, denoted as the camera intrinsic and extrinsic parameter matrix; retrieve the curtain wall data of the target building. The document, based on the spatial three-dimensional coordinates obtained by the inspection drone and the camera's intrinsic and extrinsic parameter matrices, uses affine transformation of the coordinate system to inversely project the defect topological patches on the two-dimensional image to... The model's three-dimensional surface is rendered using pseudo-color based on the severity of the defects, generating a globally intuitive three-dimensional defect mapping map.

10. The intelligent detection method for curtain wall surface defects based on deep learning image recognition according to claim 1, characterized in that, The generation of inspection and diagnostic reports and maintenance work orders includes: The system retrieves pre-set industry safety standard thresholds from the system database as alarm tolerance levels, and traverses the 3D defect mapping map to calculate the actual physical area and maximum depth of each defect topological patch. These values ​​are then logically compared with the alarm tolerance levels. If the actual physical area and maximum depth are greater than the alarm tolerance levels, the system automatically outputs a diagnostic report indicating the exact location and damage type of the defect. Simultaneously, it generates an electronic maintenance order containing suggestions for high-altitude operation routes and the required material specifications, which is then sent to the maintenance unit's terminal.

Images