Curtain Wall Defect Detection System Based on Computer Vision

Abstract

The present invention relates to the technical field of defect detection, specifically a curtain wall defect detection system based on computer vision. The system includes an image acquisition module, a texture analysis module, a dynamic defect tracking module, a defect location module, and a defect recognition module. In the present invention, multi-angle curtain wall datasets are collected through the arranged fixed cameras and adjustable-angle cameras, increasing the coverage of the overall and details of the building curtain wall. The precise adjustment of the synchronous shooting parameters and the in-depth analysis of each frame of image make the data collection more complete, ensuring high-quality image data. It allows the system to improve the automation degree of defect detection through local texture feature analysis and dynamic trajectory tracking, reducing the need for manual intervention, thereby improving the detection efficiency and accuracy. The abnormal motion areas are automatically marked and analyzed, enabling the system to timely discover potential structural safety problems, providing accurate data support for maintenance and repair, reducing maintenance costs and building safety risks.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection, and in particular to a curtain wall defect detection system based on computer vision. Background Art

[0002] Defect detection technology mainly focuses on using various technical means to identify defects, cracks or abnormalities on the surface of objects or materials, covering from traditional visual inspection methods to automatic detection systems using computer vision and machine learning technologies, which can quickly and accurately locate and identify defects by analyzing images or sensing data, and is of great significance for improving production efficiency and product quality. It usually includes image acquisition devices, image processing software, and algorithms for performing classification and recognition tasks.

[0003] Among them, a curtain wall defect detection system based on computer vision is a system that uses computer vision technology to automatically identify defects such as cracks, spalling or other structural problems in building curtain walls. The system analyzes the pictures taken of the curtain wall surface and uses image processing algorithms to detect existing problems, which can greatly improve the detection efficiency and accuracy, reduce the need for manual inspection, lower maintenance costs, and timely discover potential structural safety problems, thus ensuring the long-term use safety of the building.

[0004] The prior art relies on traditional visual inspection and basic computer vision technology, lacking the ability of automatic response to rapid changes and in-depth analysis functions. This limitation easily leads to a slow detection process in actual operation, inaccurate identification of small or complex defects. Especially in the case of lacking dynamic tracking ability, early small cracks or corrosion areas are easily ignored, and these initial defects develop into serious structural problems over time. In addition, traditional technologies are less efficient in processing a large amount of data and lack a data processing process, resulting in limited efficiency and performance in application scenarios that require rapid response. Summary of the Invention

[0005] The purpose of the present invention is to solve the disadvantages existing in the prior art, and to propose a curtain wall defect detection system based on computer vision.

[0006] To achieve the above purpose, the present invention adopts the following technical solution: A curtain wall defect detection system based on computer vision, the system includes:

[0007] An image acquisition module, based on the arranged fixed cameras and adjustable-angle cameras, acquires the image data of the curtain wall and the curtain wall connection member area, analyzes the overlapping area of adjacent image frames, and obtains a multi-angle curtain wall data set;

[0008] Based on the multi - angle curtain wall dataset, the texture analysis module divides the curtain wall panels and the aluminum alloy frames of the curtain wall into pixel blocks, analyzes the gray - scale distribution of each pixel block, identifies the regions with abnormal gray - scale changes, and obtains the local texture feature information;

[0009] Based on the local texture feature information, the dynamic defect tracking module analyzes the movement trajectories of pixel points in the image frames, filters out the pixel points with offset exceeding the normal range, marks them as abnormal movement regions, and obtains the dynamic trajectory data of curtain wall defects;

[0010] Based on the dynamic trajectory data of curtain wall defects, the defect location module analyzes the depth information of the crack region, aligns the three - dimensional coordinates of the crack region with the curtain wall structure coordinate system, identifies the spatial distribution trend of the crack region, and obtains the defect spatial distribution parameters;

[0011] Based on the defect spatial distribution parameters, the defect identification module analyzes the texture changes in the curtain wall cracks and metal corrosion regions, identifies the key defect categories with high matching degree, and obtains the curtain wall defect category identifiers.

[0012] The improvements of the present invention are as follows: the multi - angle curtain wall dataset includes a timestamp set, a shooting angle set, and a position coordinate set; the local texture feature information includes reflection texture description, corrosion texture description, and abnormal gray - scale regions; the dynamic trajectory data of curtain wall defects includes crack trajectories, sealant trajectories, and abnormal marks; the defect spatial distribution parameters include crack depth, crack coordinates, and stress regions; the curtain wall defect category identifiers include shape parameters, texture change types, and matching defect categories.

[0013] The improvements of the present invention are as follows: the image acquisition module includes:

[0014] Based on the fixed cameras and adjustable - angle cameras arranged, the synchronization parameter setting sub - module obtains the shooting angles, timestamps, and position coordinate information of the cameras, sets the synchronous shooting parameters between the cameras, calculates the time synchronization error, viewing angle difference, and spatial offset value between each pair of cameras, and filters out the cameras with synchronization error and viewing angle difference within the normal range to obtain a set of synchronous camera parameters;

[0015] Based on the set of synchronous camera parameters, the image integrity evaluation sub - module analyzes the timestamp, shooting angle, and position coordinate information of each frame of image, and uses the formula:

[0016] ;

[0017] Calculate the integrity index of the image data , and obtain the complete image data, where and respectively represent the timestamps of the th frame and the th frame of the image, is the maximum allowable difference of the time stamp, and respectively represent the shooting angles of the th frame and the th frame of the image,

[0018] The image matching and screening sub-module analyzes the overlapping regions of adjacent image frames based on the complete image data, determines the matching degree between the image frames, clears the image frames with the edge matching degree lower than the normal range, and obtains a multi-angle curtain wall data set.

[0019] The improvement of the present invention is that the texture analysis module includes:

[0020] The pixel block division sub-module collects the image data of the curtain wall panel and the curtain wall aluminum alloy frame based on the multi-angle curtain wall data set, performs pixel-level segmentation on the image, divides the curtain wall image into multiple pixel blocks by using a uniform grid division method, extracts the gray-scale distribution data of each pixel block, then calculates the gray-scale mean value and variance of the pixel block, and obtains the pixel block gray-scale statistical data;

[0021] The gray-scale gradient calculation sub-module calculates the gray-scale gradient between adjacent pixel blocks based on the pixel block gray-scale statistical data, using the formula:

[0022] ;

[0023] to obtain the pixel block gray-scale gradient data, where represents the gray-scale gradient between pixel block and pixel block , and respectively represent the gray-scale mean values of pixel block and pixel block , and are the gray-scale variances of the corresponding pixel blocks, is the smoothing factor;

[0024] The abnormal area recognition sub-module calls the pixel block gray-scale gradient data, performs global statistics on the gray-scale gradient, optimizes the global gray-scale histogram, screens the pixel block areas with abnormal gray-scale gradient changes, and combines the surface reflection texture of the curtain wall and the corrosion texture characteristics of the metal components to mark the areas with abnormal gray-scale changes, and obtains the local texture feature information.

[0025] The improvement of the present invention is that the dynamic defect tracking module includes:

[0026] Based on the local texture feature information, the pixel motion analysis sub-module obtains time-series image frames, extracts the coordinates of the pixels in each frame, establishes a time-series coordinate sequence of the pixels, and calculates the displacement of the pixels between adjacent frames to obtain the pixel motion trajectory;

[0027] The feature point matching sub-module calls the pixel motion trajectory, matches the pixels at the edge of the curtain wall crack and the pixels in the area where the curtain wall sealant has fallen off in each frame, and uses the formula:

[0028] ;

[0029] Calculate the motion vector of each feature point between consecutive frames to obtain the feature point motion vector data, where represents the feature point 's motion vector between two frames, and are the horizontal and vertical coordinates of the feature point in the th frame respectively, and are the corresponding coordinates in the th frame;

[0030] Based on the feature point motion vector data, the abnormal area marking sub-module analyzes the distribution of the feature point motion vectors, screens out the pixels whose offset exceeds the normal range, marks them as abnormal motion areas, and obtains the dynamic trajectory data of the curtain wall defects.

[0031] The improvement of the present invention is that the defect location module includes:

[0032] Based on the dynamic trajectory data of the curtain wall defects, the crack area coordinate extraction sub-module analyzes the pixel coordinates of the curtain wall crack area, extracts the pixels at the crack edge, constructs the two-dimensional coordinates of the crack area, and matches the crack boundary features in adjacent frames to obtain the two-dimensional crack coordinate data;

[0033] Based on the two-dimensional crack coordinate data and combined with the depth information of the image frame, the crack depth analysis sub-module uses the formula:

[0034] ;

[0035] Calculate the depth offset of the crack area relative to the curtain wall structure reference point to obtain the crack depth data, where represents the depth value of the crack pixel , , and are the three-dimensional coordinates of the crack pixel respectively, , and are the three-dimensional coordinates of the curtain wall structure reference point;

[0036] Based on the crack depth data, the defect spatial distribution analysis sub-module aligns the three-dimensional coordinates of the crack area with the curtain wall structure coordinate system, identifies the expansion trend of the crack area in three-dimensional space, screens the curtain wall areas with abnormal stress distribution, and obtains the defect spatial distribution parameters.

[0037] The improvement of the present invention is that the defect identification module includes:

[0038] Based on the defect spatial distribution parameters, the texture change analysis sub-module extracts the gray distribution, gradient direction, and edge contour information, calculates the local gray change rate and direction gradient distribution, and analyzes the texture changes in the curtain wall crack and metal corrosion areas to obtain the defect area texture parameters;

[0039] Based on the defect area texture parameters, the defect morphology comparison sub-module extracts the defect boundary contour, determines the area, perimeter, compactness, and shape ratio of the defect shape parameters, and uses the formula:

[0040] ;

[0041] Calculate the normalized difference value of the defect morphology , and obtain the defect area morphology parameters, where is the perimeter of the defect area, is the area of the defect area, is the gray value of the th pixel point, is the average gray value of the defect area, is the number of pixel points within the defect area;

[0042] Based on the defect area morphology parameters, the defect category matching sub-module matches with the defect types in the known defect data, analyzes the matching degree of the differential category defects, and screens the key defect categories with high matching degree to obtain the curtain wall defect category identifier.

[0043] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0044] In the present invention, by deploying fixed cameras and adjustable-angle cameras to collect multi-angle curtain wall data sets, the coverage of the overall and details of the building curtain wall is increased. The precise adjustment of the synchronous shooting parameters and the in-depth analysis of each frame of image make the data collection more complete, ensuring high-quality image data. The system is allowed to improve the automation degree of defect detection through local texture feature analysis and dynamic trajectory tracking, reducing the need for manual intervention, thereby improving the detection efficiency and accuracy. Automatically marking and analyzing abnormal motion areas enables the system to timely discover potential structural safety problems, providing accurate data support for maintenance and repair, reducing maintenance costs and building safety risks. Description of the Drawings

[0045] Figure 1 is the system flow chart of the present invention;

[0046] Figure 2 is the flow chart of the image acquisition module in the present invention;

[0047] Figure 3 is the flow chart of the texture analysis module in the present invention;

[0048] Figure 4 is the flow chart of the dynamic defect tracking module in the present invention;

[0049] Figure 5 is the flow chart of the defect location module in the present invention;

[0050] Figure 6 is the flow chart of the defect identification module in the present invention. Detailed Description of the Invention

[0051] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0052] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. In addition, in the description of the present invention, "a plurality of" means two or more unless otherwise specifically defined.

[0053] Embodiment: Please refer to Figure 1 , the present invention provides a technical solution: A curtain wall defect detection system based on computer vision includes:

[0054] The image acquisition module is based on the arranged fixed camera and adjustable-angle camera, sets the synchronous shooting parameters between the cameras, acquires the image data of the curtain wall and the curtain wall connection member area, analyzes each frame of image obtained by the camera, evaluates the integrity of the timestamp, shooting angle and position coordinates of the image data, analyzes the overlapping area of adjacent image frames, determines the matching degree between the image frames, and clears the image frames with the edge matching degree lower than the normal range to obtain a multi-angle curtain wall data set;

[0055] Based on the multi-angle curtain wall dataset, the texture analysis module divides the curtain wall panels and the aluminum alloy frames of the curtain wall into pixel blocks, analyzes the gray-scale distribution of each pixel block, compares the gray-scale gradient changes of adjacent pixels, and extracts the characteristic data of the reflection texture on the curtain wall surface and the corrosion texture of the metal components, optimizes the global gray-scale histogram, identifies the areas with abnormal gray-scale changes, and obtains the local texture feature information;

[0056] Based on the local texture feature information, the dynamic defect tracking module analyzes the movement trajectories of pixel points in the time-series image frames, matches the pixel points at the edges of the curtain wall cracks and the pixel points in the areas where the curtain wall sealant has fallen off in each frame, calculates the motion vectors of the feature points, screens out the pixel points whose offset exceeds the normal range, and marks them as abnormal motion areas to obtain the dynamic trajectory data of the curtain wall defects;

[0057] Based on the dynamic trajectory data of the curtain wall defects, the defect location module analyzes the pixel coordinates of the crack areas of the curtain wall, analyzes the depth information of the crack areas, compares the displacements of the pixel points at the edges of the cracks in each image frame, aligns the three-dimensional coordinates of the crack areas with the curtain wall structure coordinate system, identifies the spatial distribution trend of the crack areas, screens out the curtain wall areas that bear greater stress, and obtains the defect spatial distribution parameters;

[0058] Based on the defect spatial distribution parameters, the defect identification module analyzes the texture changes in the crack and metal corrosion areas of the curtain wall, determines the shape parameters of the defect areas, compares the differences in the types of defect morphologies and the known defect data, identifies the defect categories with key matching degrees, and obtains the curtain wall defect category identifiers.

[0059] The multi-angle curtain wall dataset includes a timestamp set, a shooting angle set, and a position coordinate set. The local texture feature information includes reflection texture descriptions, corrosion texture descriptions, and abnormal gray-scale areas. The dynamic trajectory data of the curtain wall defects includes crack trajectories, sealant trajectories, and abnormal markings. The defect spatial distribution parameters include crack depths, crack coordinates, and stress areas. The curtain wall defect category identifiers include shape parameters, texture change types, and matching defect categories.

[0060] Please refer to Figure 2 , the image acquisition module includes:

[0061] Based on the fixed cameras and adjustable-angle cameras arranged, the synchronization parameter setting sub-module obtains the shooting angles, timestamps, and position coordinate information of the cameras, sets the synchronous shooting parameters between the cameras, calculates the time synchronization error, viewing angle difference, and spatial offset value between each pair of cameras, screens out the cameras with the synchronization error and viewing angle difference within the normal range, and obtains the synchronous camera parameter set;

[0062] It is necessary to obtain the basic parameters of each camera, including the shooting angle, timestamp, and position coordinates. The shooting angle can be measured by the built-in gyroscope of the camera, the timestamp is recorded by the built-in clock of the camera, and the position coordinates can be obtained from the GPS data of the installation point or ranging equipment. For example, after deploying 10 cameras, their timestamps are recorded as 12:00:00.001, 12:00:00.005, 12:00:00.010, etc. respectively, and the position coordinates are set in the relative coordinate system, such as (0, 0), (1, 2), (2, 3), etc. To ensure that all cameras shoot synchronously, it is necessary to calculate the time synchronization error of each camera. The method is to select a master camera as the time reference point and calculate the time difference between it and other cameras. For example, if a certain camera is 5 ms later than the reference camera, the synchronization error is 5 ms. The calculation method of the viewing angle difference is to compare the pitch angle and horizontal rotation angle of the cameras. If a certain camera is offset by 3° compared to the reference camera, the viewing angle difference is 3°. The calculation method of the spatial offset value is to calculate the Euclidean distance between the two cameras through coordinates. For example, if the two cameras are located at (0, 0) and (3, 4) respectively, the spatial offset value is , and then set the normal ranges of the synchronization error and viewing angle offset. For example, the maximum allowable time error is 10 ms, and the maximum allowable viewing angle offset is 5°. Filter out the cameras with a synchronization error less than 10 ms and a viewing angle offset less than 5°, and eliminate the cameras that do not meet the conditions to obtain the synchronized camera parameter set.

[0063] Based on the synchronized camera parameter set, the image integrity evaluation sub-module analyzes the timestamp, shooting angle, and position coordinate information of each frame of the image, and uses the formula:

[0064] ;

[0065] Calculate the integrity index of the image data , to obtain the complete image data, where, and respectively represent the th and th frame timestamps of the image, is the maximum allowable difference of the timestamps, and respectively represent the th and th frame shooting angles of the image, is the maximum allowable difference of the shooting angles, represents the total number of image frames;

[0066] If the total number of image frames , the timestamps and shooting angles of the image frames are as follows:

[0067] Image frame data table:

[0068] ;

[0069] Based on the content of the above table, set the maximum allowable difference in timestamps , and the maximum allowable difference in shooting angles , and calculate the normalized value of the time error between adjacent video frames:

[0070] ;

[0071] ;

[0072] Calculate the normalized value of the angle error between adjacent video frames:

[0073] ;

[0074] ;

[0075] Substitute into the formula for calculation:

[0076] ;

[0077] ;

[0078] ;

[0079] The calculation result shows the image integrity index . If the integrity threshold is 0.65, since the result is greater than 0.65, the image data will be screened out. Therefore, this index is used to measure the time synchronization and perspective consistency of image data and determine which images can be used for subsequent analysis.

[0080] The image matching and screening sub-module analyzes the overlapping areas of adjacent video frames based on the complete image data, determines the matching degree between video frames, and clears the video frames with edge matching degrees below the normal range to obtain a multi-angle curtain wall data set;

[0081] Calculate the overlapping area of the images through the coordinate information of two adjacent video frames. For example, if the coverage range of the first video frame is from (0, 0) to (10, 10) and the coverage range of the second video frame is from (5, 5) to (15, 15), then the area of their overlapping region is , the total area is 100, so the overlapping ratio is . Secondly, calculate the number of matching feature points. For example, 200 feature points are detected in the first video frame, 180 feature points are detected in the second video frame, and the number of matching feature points is 150, then the matching rate is . Finally, calculate the matching error. For example, the average offset of the feature points in the two video frames is 1.2 pixels, and the maximum allowable value of the matching error is set to 2 pixels, then the normalized value of the matching error is Calculate the image matching degree comprehensively, and according to the set matching degree threshold, for example, set the lower limit of the matching degree to 0.5, screen the image frames with a matching degree lower than 0.5, clear the image frames with a lower matching degree, and obtain a multi-angle curtain wall data set.

[0082] Please refer to Figure 3 , the texture analysis module includes:

[0083] Based on the multi-angle curtain wall data set, the pixel block division sub-module collects the image data of the curtain wall panel and the curtain wall aluminum alloy frame, performs pixel-level segmentation on the image, uses a uniform grid division method to divide the curtain wall image into multiple pixel blocks, extracts the gray distribution data of each pixel block, and then calculates the gray mean and variance of the pixel block to obtain the pixel block gray statistical data;

[0084] For the curtain wall panel and aluminum alloy frame areas, use a camera to collect data to ensure that the pixel accuracy of the image can meet the requirements of subsequent analysis. For example, for the curtain wall glass area, an image resolution of 4000×3000 pixels can be used to ensure that the fine texture information on the glass surface is completely recorded. For the aluminum alloy frame, by comparing images from different angles, the influence of light is removed, the contrast of the collected data is optimized, and then the obtained image is subjected to pixel-level segmentation. Using a uniform grid division method, that is, the entire image is gridified according to a fixed window size of 20×20 pixels, so that the pixel data within each grid maintains statistical consistency. Then, for each pixel block, extract the gray distribution data, calculate the gray values of all pixel points inside the pixel block, and use the mean calculation formula: , where, represents the gray mean of the pixel block , is the number of pixel points within the pixel block, is the gray value of the pixel point within the pixel block, and then calculate the variance to measure the gray change within the pixel block. Use the variance calculation formula: , where, represents the gray variance of the pixel block . After calculation, store the mean and variance data of each pixel block in the pixel block gray statistical table to obtain the pixel block gray statistical data.

[0085] Based on the pixel block gray statistical data, the gray gradient calculation sub-module calculates the gray gradient between adjacent pixel blocks, using the formula:

[0086] ;

[0087] Obtain the pixel block gray gradient data, where, represents the pixel block and the pixel block the gray-scale gradient between and respectively represent the average gray-scale values of pixel blocks and pixel blocks ; and are the gray-scale variances of the corresponding pixel blocks, and

[0088] is the smoothing factor; and Calculate the gray-scale gradient between adjacent pixel blocks. For any two adjacent pixel blocks and First, calculate the difference in their average gray-scale values, that is , then calculate the cumulative value of their gray-scale variances, that is Set the smoothing factor

[0089] ;

[0090] The calculated value represents the difference in gray-scale gradient between two pixel blocks. If this value exceeds the threshold of 3.0, it is determined that there is an obvious gray-scale mutation in this area, and the gray-scale gradient data of the pixel blocks is obtained.

[0091] The abnormal area recognition sub-module calls the gray-scale gradient data of the pixel blocks, performs global statistics on the gray-scale gradient, optimizes the global gray-scale histogram, filters out the pixel block areas with abnormal gray-scale gradient changes, and combines the reflection texture of the curtain wall surface and the corrosion texture characteristics of the metal components to mark the areas with abnormal gray-scale changes, obtaining the local texture feature information;

[0092] Perform global statistics on the gray-scale gradient and optimize the global gray-scale histogram. Statistically analyze all the calculated gray-scale gradient data, display the distribution frequency of different gradient ranges in the form of a histogram, and set the abnormal gray-scale gradient threshold as the 90th percentile of the gray-scale gradient data, that is, calculate the 90th percentile value of all data. If the 90th percentile value is 3.5 among all the gray-scale gradient data of the pixel blocks, set the abnormal threshold as 3.5, filter out the pixel blocks corresponding to all gray-scale gradient data higher than 3.5, and combine the reflection texture of the curtain wall surface and the corrosion texture characteristics of the metal components to analyze whether the selected abnormal pixel blocks meet the characteristic standards of the curtain wall materials. For example, if a high gray-scale gradient is detected in the curtain wall glass area, it is necessary to determine whether it is a stain, reflection interference or scratch. If a high gray-scale gradient appears in the metal components, it corresponds to the corrosion spot or boundary damage area. Finally, mark the areas with abnormal gray-scale, obtaining the local texture feature information.

[0093] Please refer to Figure 4 , the dynamic defect tracking module includes:

[0094] Based on the local texture feature information, the pixel motion analysis sub-module obtains the time-series image frames, extracts the coordinates of the pixels in each frame, establishes the time-series coordinate sequence of the pixels, and calculates the displacement of the pixels between adjacent frames to obtain the pixel motion trajectory;

[0095] Obtain the time-series image frames, extract the coordinates of the pixels in the image frames. The position information of each pixel is represented by the abscissa and ordinate. For example, in a certain frame of the image, a pixel on the curtain wall surface is located at (250, 340), and in the next frame, this pixel is located at (252, 342). The position information is used to establish a complete time-series coordinate sequence. Next, compare the pixels between adjacent frames, calculate the displacement of each pixel to describe its motion trajectory, record the motion trajectory of each pixel, and store it as time-series data. For example, if the displacement of a pixel within 10 frames is 2.0, 2.1, 2.2, 5.8, 2.3, 2.0, 2.2, 2.1, 2.3, and 2.4 respectively, it can be observed that the displacement in the 4th frame is significantly larger, indicating that local structural changes or external interference effects occurred at this time point. After processing the motion trajectory data of all pixels, the complete motion trajectory information is obtained through sorting. This information can be used for subsequent feature point matching analysis to obtain the pixel motion trajectory.

[0096] The feature point matching sub-module calls the pixel motion trajectory, matches the pixel points at the edge of the curtain wall crack and the pixel points in the area where the curtain wall sealant has fallen off in each frame, and uses the formula:

[0097] ;

[0098] Calculate the motion vector of each feature point between consecutive frames to obtain the feature point motion vector data, where represents the feature point The motion vector between two frames, and are the abscissa and ordinate of the feature point in the frame respectively, and are the corresponding coordinates of the frame;

[0099] First, screen the pixel points at the edge of the curtain wall crack and the pixel points in the area where the curtain wall sealant has fallen off, extract the position information of the feature points in the time series for frame-by-frame matching. For each feature point, calculate its motion vector in adjacent frames. For example, the coordinates of a crack edge pixel point in frame 10 are (500, 620), and the coordinates in frame 11 become (503, 622). Substitute into the calculation:

[0100] ;

[0101] The result shows that the motion vector of this feature point between frame 10 and frame 11 is 3.6. If the normal range of the motion vector is from 0 to 4.5, the motion state of this feature point is within the normal range, which means that the displacement of the pixel points at the edge of the curtain wall crack during this frame interval has not exceeded the abnormal motion threshold. Therefore, it is not marked as an abnormal area for the time being. If the motion vector of this feature point continues to increase in subsequent frames and exceeds the threshold of 4.5, it is necessary to further analyze whether there is a crack propagation phenomenon in this area. Then, by combining the motion vector trends of multiple feature points, abnormal motion areas are screened to obtain the motion vector data of feature points for subsequent abnormal area marking analysis.

[0102] The abnormal area marking sub-module analyzes the motion vector distribution of feature points based on the motion vector data of feature points, screens out the pixel points whose offset exceeds the normal range, and marks them as abnormal motion areas to obtain the dynamic trajectory data of curtain wall defects;

[0103] According to the motion vector data of feature points, calculate the motion vector distribution of all feature points, analyze its overall change trend, and set an abnormal motion vector threshold to screen out the pixel points whose offset exceeds the normal range. This threshold is usually set as the 95th percentile of all motion vectors. For example, if the 95th percentile value of all calculated motion vector data is 4.5, then set the abnormal motion vector threshold to 4.5. Mark the pixel points with all motion vector values greater than 4.5, and at the same time analyze the distribution of the areas where the pixel points are located. If the motion vectors of multiple adjacent pixel points all exceed the threshold, it can be determined that there are curtain wall defects in this area. Finally, mark the abnormal motion area to obtain the dynamic trajectory data of curtain wall defects.

[0104] Please refer to Figure 5 , the defect location module includes:

[0105] The crack area coordinate extraction sub-module analyzes the pixel coordinates of the curtain wall crack area based on the dynamic trajectory data of curtain wall defects, extracts the pixel points at the crack edge, constructs the two-dimensional coordinates of the crack area, and matches the crack boundary features in adjacent frames to obtain the two-dimensional coordinate data of the crack;

[0106] In the image frame, the coordinates of each pixel point in the crack area are represented by the abscissa and ordinate. For example, in a certain frame of the image, a crack edge pixel point is located at (120, 300), and in the next frame, this pixel point is located at (122, 305). The position information is stored, and by comparing the changes between adjacent frames, the boundary change trend of the crack is analyzed. On this basis, the pixel points on the crack edge are screened to remove isolated noise points, and the pixel gradient information is used to optimize the boundary extraction of the crack area. For example, if the pixel gradient change in a certain area is abnormal and this point deviates greatly from the overall trend of the crack edge, then this point can be determined as an interfering pixel and removed. After obtaining the pixel coordinates of the crack area, a two-dimensional coordinate data of the crack area is constructed, and the method of inter-frame matching is used to compare the crack boundary features in adjacent frames to ensure the continuity of the boundary information of the crack area.

[0107] The crack depth analysis sub-module, based on the crack two-dimensional coordinate data and combined with the depth information of the image frame, uses the formula:

[0108] ;

[0109] to calculate the depth offset of the crack area relative to the reference point of the curtain wall structure, and obtain the crack depth data. Among them, represents the depth value of the crack pixel point , , and are the three-dimensional coordinates of the crack pixel point respectively, , and are the three-dimensional coordinates of the reference point of the curtain wall structure;

[0110] Call the depth information of the image frame, assign depth values to each pixel point in the crack area, and calculate the depth offset of the crack area relative to the reference point of the curtain wall structure. If the coordinates of a certain crack pixel point are (500, 620, 12.5) and the coordinates of the curtain wall reference point are (480, 600, 10.0), substitute them into the calculation:

[0111] ;

[0112] ;

[0113] The result shows that the depth offset of this crack pixel point is 0.088. If the normal range of the crack depth offset is 0.02 to 0.15, then the depth change of this crack point is within the normal range, indicating that the depth of the crack at this position has not changed abnormally. Therefore, this pixel point can be regarded as part of the stable crack area and does not need to be marked as abnormal.

[0114] Based on the crack depth data, the defect spatial distribution analysis sub-module aligns the three-dimensional coordinates of the crack area with the curtain wall structure coordinate system, identifies the expansion trend of the crack area in three-dimensional space, screens the curtain wall areas with abnormal stress distribution, and obtains the defect spatial distribution parameters;

[0115] Call the three-dimensional coordinate data of the crack area, align it with the curtain wall structure coordinate system, construct the spatial distribution model of the curtain wall crack, analyze the expansion trend of the crack area in three-dimensional space, calculate the depth gradient change for all crack areas. If the crack depth gradient change in a certain area exceeds the set abnormal threshold, mark this area as the stress concentration area. This abnormal threshold can be set based on the statistical analysis of historical crack data. For example, if the normal crack depth gradient change range is from 0.02 to 0.15, when the depth gradient change in a certain area exceeds 0.15, it can be determined as an abnormal area. By screening all abnormal areas, the curtain wall areas with abnormal stress distribution are obtained, and the defect spatial distribution parameters are obtained.

[0116] Please refer to Figure 6 , the defect identification module includes:

[0117] Based on the defect spatial distribution parameters, the texture change analysis sub-module extracts the gray-scale distribution, gradient direction, and edge contour information, calculates the local gray-scale change rate and the direction gradient distribution, and analyzes the texture changes of the curtain wall cracks and metal corrosion areas to obtain the defect area texture parameters;

[0118] Separate the gray-scale distribution, gradient direction, and edge contour information of each area from the acquired image data. During the image processing, the gray value of each pixel point is represented by 0 to 255, where 0 represents the darkest and 255 represents the brightest. By comparing the gray value changes between the crack area and the surrounding normal areas, calculate the local gray-scale change rate. For example, if the pixel points and adjacent pixel points in a certain crack area are 180 and 200 respectively, then the gray-scale change rate of this area is calculated as . If this value is greater than the preset threshold of 0.10, it can be determined that there is an abnormal gray-scale change in this area. In addition, to further analyze the texture features, calculate the direction gradient distribution. Calculate the gradient values in the x and y directions through the Sobel operator. For example, if the x-direction gradient of a certain area is 50 and the y-direction gradient is 80, then the gradient amplitude can be calculated as , and the gradient direction angle is . By calculating the gradient value of each pixel point and statistically analyzing the gradient direction, analyze the texture change situation of the cracks and metal corrosion areas to obtain the defect area texture parameters.

[0119] The defect morphology comparison sub-module extracts the defect boundary contour based on the texture parameters of the defect area, determines the area, perimeter, compactness, and form ratio of the defect shape parameters, and uses the formula:

[0120] ;

[0121] Calculate the normalized difference value of the defect morphology , and obtain the morphology parameters of the defect area. Among them, is the perimeter of the defect area, is the area of the defect area, is the gray value of the th pixel point, is the average gray value of the defect area,

[0122] is the number of pixel points in the defect area; 2 By edge detection, obtain the closed contour of the defect area, calculate the area, perimeter, compactness, and form ratio of the defect. The area can be calculated by the pixel counting method. For example, if a defect area contains 500 pixel points, and each pixel point corresponds to an actual area of 0.1 mm , then the perimeter is obtained by calculating the total number of edge pixels. If the boundary of this area contains 100 pixel points, the perimeter is: , where, if the gray value of the pixel points in this defect area is , then the mean value is:

[0123] ;

[0124] Calculate the gray variance:

[0125] ;

[0126] Calculate the standard deviation:

[0127] ;

[0128] Calculate the compactness:

[0129] ;

[0130] Calculate the normalized gray difference:

[0131] ;

[0132] Calculate to get:

[0133] ;

[0134] This result shows that the normalized difference value of the morphology parameters of the defect area , if the normal range of the morphological normalization difference is from 0.5 to 1.5, then the defect morphological parameter is within this range, indicating that the morphological characteristics of the defect are within the known defect type range and can be used for further matching of defect categories. If it exceeds 1.5, it indicates that the defect belongs to an abnormal category or an unrecorded category, and it is necessary to further analyze whether there is a new defect type or measurement error.

[0135] The defect category matching sub-module matches the defect type in the known defect data based on the defect area morphological parameter, analyzes the matching degree of the differential category defect, and screens the key defect categories with high matching degree to obtain the curtain wall defect category identifier;

[0136] Based on the defect area morphological parameter, call the defect type data in the known defect data to perform defect matching calculation. For each category of known defect data, calculate the matching degree. If the known morphological parameter of a certain category of defect , then calculate the difference value from the current defect . If the difference value is less than the preset matching threshold of 0.02, it is determined that the defect matches this category; otherwise, continue to match other defect categories. Finally, screen the key defect categories with high matching degree to obtain the curtain wall defect category identifier.

[0137] The above is only a preferred embodiment of the present invention, and it does not limit the present invention in other forms. Any person skilled in the art may use the disclosed technical content to make changes or modifications into equivalent embodiments with equivalent changes and apply them to other fields. However, as long as it does not depart from the technical solution content of the present invention, any simple modification, equivalent change, and modification made to the above embodiments based on the technical essence of the present invention still fall within the protection scope of the technical solution of the present invention.

Claims

1. A curtain wall defect detection system based on computer vision, characterized in that The system includes: The image acquisition module acquires the image data of the curtain wall and the curtain wall connection member area based on the arranged fixed camera and adjustable-angle camera, analyzes the overlapping area of adjacent image frames, and obtains a multi-angle curtain wall data set. The texture analysis module divides the curtain wall panel and the curtain wall aluminum alloy frame into pixel blocks based on the multi-angle curtain wall data set, analyzes the gray distribution of each pixel block, identifies the area with abnormal gray change, and obtains the local texture feature information. The dynamic defect tracking module analyzes the motion trajectories of the pixel points in the image frames according to the local texture feature information, filters out the pixel points with offset exceeding the normal range, marks them as abnormal motion areas, and obtains the dynamic trajectory data of the curtain wall defects. The dynamic defect tracking module includes: The pixel point motion analysis sub-module obtains the time-series image frames according to the local texture feature information, extracts the coordinates of the pixel points in each frame, establishes the time-series coordinate sequence of the pixel points, and calculates the displacement of the pixel points between adjacent frames to obtain the pixel point motion trajectory. The feature point matching sub-module calls the pixel point motion trajectory to match the pixel points at the crack edge of the curtain wall and the pixel points in the area where the curtain wall sealant falls off in each frame, and uses the formula: ； Calculate the motion vectors of each feature point between consecutive frames to obtain the feature point motion vector data, where, represents the feature point the motion vector between two frames, and are respectively the horizontal and vertical coordinates of the feature point in the frame, and are the corresponding coordinates in the frame; The abnormal area marking sub-module analyzes the motion vector distribution of the feature points based on the feature point motion vector data, filters out the pixel points with offset exceeding the normal range, and marks them as abnormal motion areas to obtain the dynamic trajectory data of the curtain wall defects. The defect location module analyzes the depth information of the crack area based on the dynamic trajectory data of the curtain wall defects, aligns the three-dimensional coordinates of the crack area with the curtain wall structure coordinate system, identifies the spatial distribution trend of the crack area, and obtains the defect spatial distribution parameters. The defect location module includes: The crack area coordinate extraction sub-module analyzes the pixel coordinates of the curtain wall crack area based on the dynamic trajectory data of the curtain wall defects, extracts the pixel points at the crack edge, constructs the two-dimensional coordinates of the crack area, and matches the crack boundary features in adjacent frames to obtain the crack two-dimensional coordinate data. The crack depth analysis sub-module combines the depth information of the image frame based on the crack two-dimensional coordinate data and uses the formula: ； Calculate the depth offset of the crack area relative to the reference point of the curtain wall structure to obtain crack depth data, where represents the depth value of the crack pixel point , , and are the three-dimensional coordinates of the crack pixel point respectively, , and are the three-dimensional coordinates of the reference point of the curtain wall structure; The defect spatial distribution analysis sub-module aligns the three-dimensional coordinates of the crack area with the curtain wall structure coordinate system based on the crack depth data, identifies the expansion trend of the crack area in three-dimensional space, and filters out the curtain wall areas with abnormal stress distribution to obtain the defect spatial distribution parameters. The defect identification module analyzes the texture changes of the curtain wall cracks and the metal corrosion areas based on the defect spatial distribution parameters, identifies the defect categories with key matching degrees, and obtains the curtain wall defect category identifiers.

2. The curtain wall defect detection system based on computer vision according to claim 1, characterized in that, The multi-angle curtain wall data set includes a time stamp set, a shooting angle set, and a position coordinate set. The local texture feature information includes reflection texture description, corrosion texture description, and abnormal gray area. The dynamic trajectory data of the curtain wall defects includes crack trajectory, sealant trajectory, and abnormal mark. The defect spatial distribution parameters include crack depth, crack coordinates, and stress area. The curtain wall defect category identifiers include shape parameters, texture change types, and matching defect categories.

3. The curtain wall defect detection system based on computer vision according to claim 1, characterized in that The image acquisition module includes: The synchronization parameter setting sub-module obtains the shooting angles, timestamps, and position coordinate information of the cameras based on the deployed fixed cameras and adjustable-angle cameras, sets the synchronous shooting parameters between the cameras, calculates the time synchronization error, viewing angle difference, and spatial offset value between each pair of cameras, and filters out the cameras with the time synchronization error and viewing angle difference within the normal range to obtain a set of synchronous camera parameters; The image integrity evaluation sub-module analyzes the timestamp, shooting angle, and position coordinate information of each frame of image based on the set of synchronous camera parameters, using the formula: ； Calculate the integrity index of the image data , to obtain the complete image data, where and represent the timestamps of the th and th frames of the image respectively, is the maximum allowable difference of the timestamps, and represent the shooting angles of the th and th frames of the image respectively, is the maximum allowable difference of the shooting angles, represents the total number of image frames; The image matching and screening sub-module analyzes the overlapping regions of adjacent image frames based on the complete image data, determines the matching degree between the image frames, clears the image frames with an edge matching degree lower than the normal range, and obtains a multi-angle curtain wall data set.

4. The curtain wall defect detection system based on computer vision according to claim 1, characterized in that, The texture analysis module includes: The pixel block division sub-module collects the image data of the curtain wall panel and the curtain wall aluminum alloy frame based on the multi-angle curtain wall data set, performs pixel-level segmentation on the image, divides the curtain wall image into multiple pixel blocks using a uniform grid division method, extracts the gray-scale distribution data of each pixel block, and then calculates the gray-scale mean and variance of the pixel block to obtain the pixel block gray-scale statistical data; The gray-scale gradient calculation sub-module calculates the gray-scale gradient between adjacent pixel blocks based on the pixel block gray-scale statistical data, using the formula: ； Obtain the gray-scale gradient data of the pixel block, where, represents the pixel block and the pixel block The gray-scale gradient between them, and respectively represent the pixel block and the pixel block The gray-scale mean value of, and Is the gray-scale variance of the corresponding pixel block, Is the smoothing factor; The abnormal area recognition sub-module calls the pixel block gray-scale gradient data, performs global statistics on the gray-scale gradient, optimizes the global gray-scale histogram, filters out the pixel block areas with abnormal gray-scale gradient changes, and combines the surface reflection texture of the curtain wall and the corrosion texture features of the metal components to mark the areas with abnormal gray-scale changes, obtaining local texture feature information.

5. The curtain wall defect detection system based on computer vision according to claim 1, characterized in that, The defect recognition module includes: The texture change analysis sub-module extracts the gray-scale distribution, gradient direction, and edge contour information based on the defect spatial distribution parameters, calculates the local gray-scale change rate and direction gradient distribution, and analyzes the texture changes in the curtain wall crack and metal corrosion areas to obtain the defect area texture parameters; The defect shape comparison sub-module extracts the defect boundary contour based on the defect area texture parameters, determines the area, perimeter, compactness, and aspect ratio of the defect shape parameters, using the formula: ； Calculate the normalized difference value of the defect morphology , and obtain the morphological parameters of the defect area, where is the perimeter of the defect area, is the area of the defect area, is the gray value of the th pixel point, is the average gray value of the defect area, is the number of pixel points in the defect area; The defect category matching sub-module matches the defect area shape parameters with the defect types in the known defect data, analyzes the matching degree of different category defects, and filters out the key defect categories with high matching degree to obtain the curtain wall defect category identifier.

Images