Curtain wall defect detection system based on computer vision

Abstract

The invention relates to the technical field of defect detection, in particular to a curtain wall defect detection system based on computer vision, which comprises an image acquisition module, a texture analysis module, a dynamic defect tracking module, a defect positioning module and a defect identification module. According to the invention, a multi-angle curtain wall data set is collected through the arranged fixed camera and the angle-adjustable camera, the coverage range of the whole and details of the building curtain wall is increased, the accurate adjustment of synchronous shooting parameters and the deep analysis of each frame of image enable the data collection to be more complete, and the high-quality image data is ensured. The system is allowed to perform local texture feature analysis and dynamic trajectory tracking, so that the automation degree of defect detection is improved, the manual intervention requirement is reduced, the detection efficiency and accuracy are improved, an abnormal motion area is automatically marked and analyzed, the system can find potential structural safety problems in time, and the safety of the system is improved. Accurate data support is provided for maintenance and repair, and maintenance cost and building safety risks are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection, and in particular to a curtain wall defect detection system based on computer vision. Background Art

[0002] Defect detection technology mainly focuses on using various technical means to identify defects, cracks or anomalies on the surface of objects or materials. It covers everything from traditional visual inspection methods to automatic inspection systems using computer vision and machine learning technologies. It can quickly and accurately locate and identify defects by analyzing images or sensor data, which is of great significance for improving production efficiency and product quality. It usually includes image acquisition equipment, image processing software, and algorithms for performing classification and recognition tasks.

[0003] Among them, the computer vision curtain wall defect detection system is a system that uses computer vision technology to automatically identify defects in building curtain walls such as cracks, peeling or other structural problems. The system analyzes pictures taken of the curtain wall surface and uses image processing algorithms to detect existing problems. It can greatly improve the efficiency and accuracy of detection, reduce the need for manual inspections, reduce maintenance costs, and promptly detect potential structural safety problems, thereby ensuring the long-term safety of the building.

[0004] Existing technologies rely on traditional visual inspection and basic computer vision technology, lacking the ability to automatically respond to rapid changes and in-depth analysis capabilities. This limitation can easily lead to a slow detection process in actual operations and inaccurate identification of small or complex defects, especially in the absence of dynamic tracking capabilities. Early tiny cracks or corrosion areas are easily overlooked, and these initial defects develop into serious structural problems over time. In addition, traditional technologies are inefficient when processing large amounts of data and lack a data processing process, resulting in limited efficiency and performance in application scenarios that require a quick response. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings in the prior art and to propose a curtain wall defect detection system based on computer vision.

[0006] In order to achieve the above object, the present invention adopts the following technical solution: a curtain wall defect detection system based on computer vision, the system comprising: The image acquisition module collects image data of the curtain wall and the curtain wall connection component area based on the deployed fixed cameras and adjustable angle cameras, analyzes the overlapping area of ​​adjacent image frames, and obtains a multi-angle curtain wall data set; The texture analysis module divides the curtain wall panel and the curtain wall aluminum alloy frame into pixel blocks based on the multi-angle curtain wall data set, analyzes the grayscale distribution of each pixel block, identifies the area with abnormal grayscale changes, and obtains local texture feature information; The dynamic defect tracking module analyzes the motion trajectory of the pixels in the image frame according to the local texture feature information, selects the pixels whose offset exceeds the normal range, marks them as abnormal motion areas, and obtains the dynamic trajectory data of the curtain wall defects; The defect location module analyzes the depth information of the crack area based on the dynamic trajectory data of the curtain wall defect, aligns the three-dimensional coordinates of the crack area with the curtain wall structure coordinate system, identifies the spatial distribution trend of the crack area, and obtains the defect spatial distribution parameters; The defect recognition module analyzes the texture changes of the curtain wall cracks and the metal corrosion area based on the defect spatial distribution parameters, identifies the defect category with a critical matching degree, and obtains the curtain wall defect category identification.

[0007] The improvements of the present invention are that the multi-angle curtain wall data set includes a timestamp set, a shooting angle set, and a position coordinate set; the local texture feature information includes a reflection texture description, a corrosion texture description, and an abnormal grayscale area; the curtain wall defect dynamic trajectory data includes a crack trajectory, a sealant trajectory, and an abnormal mark; the defect spatial distribution parameters include crack depth, crack coordinates, and stress area; the curtain wall defect category identifier includes shape parameters, texture change type, and matching defect category.

[0008] The present invention is improved in that the image acquisition module comprises: The synchronization parameter setting submodule obtains the shooting angle, timestamp and position coordinate information of the camera based on the deployed fixed cameras and adjustable angle cameras, sets the synchronization shooting parameters between the cameras, calculates the time synchronization error, viewing angle difference and spatial offset value between each camera, selects the cameras whose synchronization error and viewing angle offset are within the normal range, and obtains the synchronization camera parameter set; The image integrity assessment submodule parses the timestamp, shooting angle and position coordinate information of each frame of the image based on the synchronized camera parameter set, using the formula: ; Calculate the integrity index of image data , get the complete image data, where and Respectively represent Frame and The timestamp of the frame image, is the maximum allowed difference in timestamps, and Respectively represent Frame and The shooting angle of the frame image, is the maximum permissible difference in shooting angles, Represents the total number of image frames; The image matching and screening submodule analyzes the overlapping areas of adjacent image frames based on the complete image data, determines the matching degree between the image frames, removes the image frames whose edge matching degree is lower than the normal range, and obtains a multi-angle curtain wall data set.

[0009] The present invention is improved in that the texture analysis module comprises: The pixel block division submodule collects image data of the curtain wall panel and the curtain wall aluminum alloy frame based on the multi-angle curtain wall data set, performs pixel-level segmentation on the image, divides the curtain wall image into multiple pixel blocks using a uniform grid division method, extracts the grayscale distribution data of each pixel block, and then calculates the grayscale mean and variance of the pixel block to obtain the grayscale statistical data of the pixel block; The grayscale gradient calculation submodule calculates the grayscale gradient between adjacent pixel blocks based on the grayscale statistics of the pixel blocks, using the formula: ; Get the pixel block grayscale gradient data, where Represents a pixel block With pixel blocks The grayscale gradient between and Represents pixel blocks With pixel blocks The grayscale mean, and is the grayscale variance of the corresponding pixel block, is the smoothing factor; The abnormal area identification submodule calls the grayscale gradient data of the pixel block, performs global statistics on the grayscale gradient, optimizes the global grayscale histogram, screens the pixel block areas with abnormal grayscale gradient changes, and combines the curtain wall surface reflection texture and metal component corrosion texture characteristics to mark the grayscale abnormal areas and obtain local texture feature information.

[0010] The present invention is improved in that the dynamic defect tracking module includes: The pixel point motion analysis submodule obtains time series image frames according to the local texture feature information, extracts the coordinates of the pixels in each frame, establishes a time series coordinate sequence of the pixels, and calculates the displacement of the pixels between adjacent frames to obtain the pixel point motion trajectory; The feature point matching submodule calls the pixel point motion trajectory to match the curtain wall crack edge pixel points with the curtain wall sealant peeling area pixel points in each frame, using the formula: ; Calculate the motion vector of each feature point between consecutive frames to obtain the feature point motion vector data, where: Representative feature points The motion vector between two frames, and The feature points are The horizontal and vertical coordinates of the frame, and For the The corresponding coordinates of the frame; The abnormal area marking submodule analyzes the motion vector distribution of the feature points based on the feature point motion vector data, selects pixels whose offsets exceed the normal range, marks them as abnormal motion areas, and obtains dynamic trajectory data of curtain wall defects.

[0011] The present invention is improved in that the defect location module comprises: The crack region coordinate extraction submodule analyzes the pixel coordinates of the curtain wall crack region based on the curtain wall defect dynamic trajectory data, extracts the crack edge pixel points, constructs the two-dimensional coordinates of the crack region, and matches the crack boundary features in adjacent frames to obtain the crack two-dimensional coordinate data; The crack depth analysis submodule is based on the two-dimensional coordinate data of the crack and the depth information of the image frame, using the formula: ; Calculate the depth offset of the crack area relative to the reference point of the curtain wall structure to obtain the crack depth data, where: Represents crack pixels The depth value of , and are the three-dimensional coordinates of the crack pixels, , and The three-dimensional coordinates of the reference point of the curtain wall structure; The defect spatial distribution analysis submodule aligns the three-dimensional coordinates of the crack area with the curtain wall structure coordinate system based on the crack depth data, identifies the expansion trend of the crack area in the three-dimensional space, and screens the curtain wall area with abnormal stress distribution to obtain the defect spatial distribution parameters.

[0012] The present invention is improved in that the defect recognition module comprises: The texture change analysis submodule extracts the grayscale distribution, gradient direction and edge profile information based on the defect spatial distribution parameters, calculates the local grayscale change rate and directional gradient distribution, and analyzes the texture changes of the curtain wall cracks and metal corrosion areas to obtain the texture parameters of the defect area; The defect morphology comparison submodule extracts the defect boundary contour based on the texture parameters of the defect area, determines the area, perimeter, compactness and morphology ratio of the defect shape parameters, and adopts the formula: ; Calculate the normalized difference value of the defect shape , and obtain the defect area morphological parameters, where is the perimeter of the defect area, is the area of ​​the defect region, For the The gray value of a pixel, is the average gray value of the defect area, is the number of pixels in the defect area; The defect category matching submodule matches the defect type in the known defect data according to the defect area morphological parameters, analyzes the matching degree of the differentiated category defects, and screens the defect categories with the key matching degree to obtain the curtain wall defect category identification.

[0013] Compared with the prior art, the advantages and positive effects of the present invention are: In the present invention, multi-angle curtain wall data sets are collected by deploying fixed cameras and adjustable-angle cameras, thereby increasing the coverage of the entire building curtain wall and its details. The precise adjustment of synchronous shooting parameters and the in-depth analysis of each frame of the image make the data collection more complete, ensuring high-quality image data. The system is allowed to improve the automation of defect detection through local texture feature analysis and dynamic trajectory tracking, reducing the need for manual intervention, thereby improving detection efficiency and accuracy, automatically marking abnormal movement areas and analyzing them, so that the system can promptly discover potential structural safety problems, provide accurate data support for maintenance and repair, and reduce maintenance costs and building safety risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a system flow chart of the present invention; Figure 2 It is a flow chart of the image acquisition module in the present invention; Figure 3 is a flow chart of the texture analysis module in the present invention; Figure 4 is a flow chart of the dynamic defect tracking module in the present invention; Figure 5 This is a flow chart of the defect location module in the present invention; Figure 6 It is a flow chart of the defect identification module in the present invention. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0016] In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating positions or positional relationships, are based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, in the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0017] Example: See Figure 1 The present invention provides a technical solution: a curtain wall defect detection system based on computer vision comprises: The image acquisition module sets the synchronous shooting parameters between the cameras based on the fixed cameras and the adjustable angle cameras, collects the image data of the curtain wall and the curtain wall connection component area, analyzes each frame of the image acquired by the camera, evaluates the integrity of the timestamp, shooting angle and position coordinates of the image data, parses the overlapping area of ​​adjacent image frames, determines the matching degree between image frames, and removes the image frames with edge matching degrees lower than the normal range to obtain a multi-angle curtain wall data set; The texture analysis module divides the curtain wall panels and curtain wall aluminum alloy frames into pixel blocks based on the multi-angle curtain wall data set, analyzes the grayscale distribution of each pixel block, compares the grayscale gradient changes of adjacent pixels, and extracts the feature data of the curtain wall surface reflection texture and the metal component corrosion texture, optimizes the global grayscale histogram, identifies the areas with abnormal grayscale changes, and obtains local texture feature information; The dynamic defect tracking module analyzes the motion trajectory of pixels in the time series image frames based on local texture feature information, matches the pixels at the edge of the curtain wall cracks with the pixels in the curtain wall sealant detachment area in each frame, calculates the motion vector of the feature points, filters out pixels whose offsets exceed the normal range, and marks them as abnormal motion areas, thereby obtaining dynamic trajectory data of curtain wall defects; The defect location module analyzes the pixel coordinates of the curtain wall crack area based on the dynamic trajectory data of curtain wall defects, resolves the depth information of the crack area, compares the displacement of the crack edge pixel points in each image frame, aligns the three-dimensional coordinates of the crack area with the curtain wall structure coordinate system, identifies the spatial distribution trend of the crack area, screens the curtain wall area with greater stress, and obtains the defect spatial distribution parameters; The defect recognition module analyzes the texture changes of curtain wall cracks and metal corrosion areas based on the defect spatial distribution parameters, determines the shape parameters of the defect area, compares the defect morphology with the type differences of known defect data, identifies the defect category with the key matching degree, and obtains the curtain wall defect category identification.

[0018] The multi-angle curtain wall dataset includes a timestamp set, a shooting angle set, and a position coordinate set. The local texture feature information includes reflection texture description, corrosion texture description, and abnormal grayscale area. The dynamic trajectory data of curtain wall defects includes crack trajectory, sealant trajectory, and abnormal mark. The defect spatial distribution parameters include crack depth, crack coordinates, and stress area. The curtain wall defect category identification includes shape parameters, texture change type, and matching defect category.

[0019] See also Figure 2 , the image acquisition module includes: The synchronization parameter setting submodule obtains the shooting angle, timestamp and position coordinate information of the camera based on the deployed fixed cameras and adjustable angle cameras, sets the synchronization shooting parameters between the cameras, calculates the time synchronization error, viewing angle difference and spatial offset value between each camera, selects the cameras whose synchronization error and viewing angle offset are within the normal range, and obtains the synchronization camera parameter set; It is necessary to obtain the basic parameters of each camera, including shooting angle, timestamp and location coordinates. The shooting angle can be measured by the built-in gyroscope of the camera, the timestamp is recorded by the built-in clock of the camera, and the location coordinates can be obtained by the GPS data or ranging equipment of the installation point. For example, after 10 cameras are deployed, their timestamps are recorded as 12:00:00.001, 12:00:00.005, 12:00:00.010, etc. The location coordinates are set in the relative coordinate system, such as (0, 0), (1, 2), (2, 3), etc., to ensure the synchronization of all cameras. To shoot, it is necessary to calculate the time synchronization error of each camera. The method is to select a main camera as the time reference point and calculate the time difference between it and other cameras. For example, if a camera is 5ms behind the reference camera, the synchronization error is 5ms. The method of calculating the viewing angle difference is to compare the pitch angle and horizontal rotation angle of the camera. If a camera is offset by 3° compared with the reference camera, the viewing angle difference is 3°. The method of calculating the spatial offset value is to calculate the Euclidean distance between the two cameras through the coordinates. For example, if the two cameras are at (0, 0) and (3, 4) respectively, the spatial offset value is , and then set the normal range of synchronization error and viewing angle offset, for example, the maximum allowed time error is 10ms, and the maximum allowed viewing angle offset is 5°. Filter out cameras with synchronization error less than 10ms and viewing angle offset less than 5°, and eliminate cameras that do not meet the conditions to obtain a synchronized camera parameter set.

[0020] The image integrity assessment submodule parses the timestamp, shooting angle and position coordinate information of each frame based on the synchronized camera parameter set, using the formula: ; Calculate the integrity index of image data , get the complete image data, where and Respectively represent Frame and The timestamp of the frame image, is the maximum allowed difference in timestamps, and Respectively represent Frame and The shooting angle of the frame image, is the maximum permissible difference in shooting angles, Represents the total number of image frames; If the total number of image frames , the timestamp and shooting angle of the image frame are as follows: Image frame data table: ; According to the above table, set the maximum allowable difference of timestamps , the maximum allowable difference in shooting angle , calculate the normalized value of the time error between adjacent image frames: ; ; Calculate the normalized value of the angular error between adjacent image frames: ; ; Substitute into the formula to calculate: ; ; ; The calculation results show that the image integrity index ,If the integrity threshold is 0.65, the image data will be screened out because the result is greater than 0.65. Therefore, this index is used to measure the temporal synchronization and perspective consistency of the image data to decide which images can be used for subsequent analysis.

[0021] The image matching and screening submodule analyzes the overlapping areas of adjacent image frames based on the complete image data, determines the matching degree between image frames, removes image frames with edge matching degrees lower than the normal range, and obtains a multi-angle curtain wall data set; The overlapping area of ​​the images is calculated using the coordinate information of two adjacent frames. For example, if the coverage range of the first frame is (0, 0) to (10, 10), and the coverage range of the second frame is (5, 5) to (15, 15), then the area of ​​the overlapping area is , the total area is 100, then the overlap ratio is , Secondly, calculate the number of feature point matches. For example, if 200 feature points are detected in the first frame and 180 feature points are detected in the second frame, the number of matched feature points is 150, then the matching rate is Finally, the matching error is calculated. For example, if the average offset of the feature points in the two frames is 1.2 pixels and the maximum allowable value of the matching error is set to 2 pixels, the normalized value of the matching error is , comprehensively calculate the image matching degree, and according to the set matching degree threshold, for example, set the matching degree lower limit to 0.5, filter the image frames with matching degrees lower than 0.5, remove the image frames with low matching degrees, and obtain the multi-angle curtain wall data set.

[0022] See also Figure 3 , the texture analysis module includes: The pixel block segmentation submodule collects image data of curtain wall panels and curtain wall aluminum alloy frames based on multi-angle curtain wall data sets, performs pixel-level segmentation on the images, divides the curtain wall images into multiple pixel blocks using a uniform grid division method, extracts the grayscale distribution data of each pixel block, and then calculates the grayscale mean and variance of the pixel block to obtain the grayscale statistics of the pixel block; For the curtain wall panels and aluminum alloy frame areas, cameras are used for data collection to ensure that the pixel accuracy of the image can meet the requirements of subsequent analysis. For example, for the curtain wall glass area, an image resolution of 4000×3000 pixels can be used to ensure that the subtle texture information on the glass surface is fully recorded. For the aluminum alloy frame, images at different angles are compared to remove the influence of lighting and optimize the contrast of the collected data. Subsequently, the acquired image is segmented at the pixel level using a uniform grid division method, that is, the entire image is gridded with a fixed window size of 20×20 pixels to keep the pixel data in each grid statistically consistent. Next, the grayscale distribution data is extracted for each pixel block, and the grayscale values ​​of all pixels inside the pixel block are calculated using the mean calculation formula: ,in, Represents a pixel block The grayscale mean, is the number of pixels in the pixel block, is the gray value of the pixel in the pixel block, and the variance is further calculated to measure the gray change inside the pixel block. The variance calculation formula is: ,in, Represents a pixel block After the grayscale variance is calculated, the mean and variance data of each pixel block are stored in the pixel block grayscale statistics table to obtain the pixel block grayscale statistics data.

[0023] The grayscale gradient calculation submodule calculates the grayscale gradient between adjacent pixel blocks based on the grayscale statistics of the pixel blocks, using the formula: ; Get the pixel block grayscale gradient data, where Represents a pixel block With pixel blocks The grayscale gradient between and Represents pixel blocks With pixel blocks The grayscale mean, and is the grayscale variance of the corresponding pixel block, is the smoothing factor; Calculate the grayscale gradient between adjacent pixel blocks. and , first calculate the grayscale mean difference between the two, that is, , and then calculate the cumulative value of the grayscale variance of the two, that is, , combined with the smoothing factor Calculate the grayscale gradient, The smoothing factor is set to 1.5 to ensure the stability of the calculation. If two adjacent pixel blocks are selected, their grayscale means are 120 and 135, and their variances are 25 and 30, respectively, then the calculation is: ; Calculated value Represents the grayscale gradient difference between two pixel blocks. If the value exceeds the threshold of 3.0, it is determined that there is an obvious grayscale mutation in the area, and the grayscale gradient data of the pixel block is obtained.

[0024] The abnormal area identification submodule calls the grayscale gradient data of the pixel block, performs global statistics on the grayscale gradient, optimizes the global grayscale histogram, screens the pixel block areas with abnormal grayscale gradient changes, and combines the curtain wall surface reflection texture and metal component corrosion texture characteristics to mark the grayscale abnormal areas and obtain local texture feature information; Perform global statistics on grayscale gradients, optimize the global grayscale histogram, perform statistical analysis on all calculated grayscale gradient data, display the distribution frequency of different gradient ranges in the form of a histogram, set the abnormal grayscale gradient threshold to the 90% percentile of the grayscale gradient data, that is, calculate all The 90% percentile value of the data is 3.5. If the 90% percentile value of the grayscale gradient data of all pixel blocks is 3.5, the abnormal threshold is set to 3.5, and the pixel blocks corresponding to all grayscale gradient data higher than 3.5 are screened. Combined with the curtain wall surface reflection texture and metal component corrosion texture characteristics, analyze whether the screened abnormal pixel blocks meet the characteristic standards of the curtain wall material. For example, if a high grayscale gradient is detected in the curtain wall glass area, it is necessary to determine whether it is a stain, reflection interference or scratch. If a high grayscale gradient appears in the metal component, it corresponds to a corrosion spot or a boundary damaged area. Finally, the grayscale abnormal area is marked to obtain local texture feature information.

[0025] See also Figure 4 ,The dynamic defect tracking module includes: The pixel motion analysis submodule obtains time series image frames based on local texture feature information, extracts the coordinates of the pixels in each frame, establishes a time series coordinate sequence of the pixels, and calculates the displacement of the pixels between adjacent frames to obtain the pixel motion trajectory; The time series image frames are obtained, and the coordinates of the pixels in the image frames are extracted. The position information of each pixel is represented by the horizontal coordinate and the vertical coordinate. For example, in a certain frame of the image, a pixel on the surface of the curtain wall is located at (250, 340), and in the next frame, the pixel is located at (252, 342). The position information is used to establish a complete time series coordinate sequence. Next, the pixels between adjacent frames are compared, and the displacement of each pixel is calculated to describe its motion trajectory. The motion trajectory of each pixel is recorded and stored as time series data. For example, if the displacement of a pixel in 10 frames is 2.0, 2.1, 2.2, 5.8, 2.3, 2.0, 2.2, 2.1, 2.3 and 2.4 respectively, it can be observed that the displacement of the 4th frame is significantly larger, indicating that a local structural change or external interference has occurred at this time point. After the motion trajectory data of all pixels are processed, the complete motion trajectory information is obtained by sorting. This information can be used for subsequent feature point matching analysis to obtain the pixel motion trajectory.

[0026] The feature point matching submodule calls the pixel point motion trajectory to match the curtain wall crack edge pixel points with the curtain wall sealant peeling area pixel points in each frame using the formula: ; Calculate the motion vector of each feature point between consecutive frames to obtain the feature point motion vector data, where: Representative feature points The motion vector between two frames, and The feature points are The horizontal and vertical coordinates of the frame, and For the The corresponding coordinates of the frame; First, the pixel points at the edge of the curtain wall crack and the pixel points in the curtain wall sealant peeling area are selected, and the position information of the feature points in the time series is extracted for frame-by-frame matching. For each feature point, its motion vector is calculated in the adjacent frames. For example, the coordinates of a crack edge pixel in frame 10 are (500, 620), and the coordinates in frame 11 become (503, 622). Substitute it into the calculation: ; The result shows that the motion vector of the feature point between frame 10 and frame 11 is 3.6. If the normal range of the motion vector is 0 to 4.5, the motion state of the feature point is within the normal range, which means that the displacement of the pixel point at the edge of the curtain wall crack between the frames has not exceeded the abnormal motion threshold. Therefore, it is not marked as an abnormal area for the time being. If the motion vector of the feature point in subsequent frames continues to increase and exceeds the threshold of 4.5, it is necessary to further analyze whether there is crack extension in the area, and then combine the motion vector trends of multiple feature points to screen the abnormal motion area and obtain the feature point motion vector data for subsequent abnormal area marking analysis.

[0027] The abnormal area marking submodule analyzes the motion vector distribution of feature points based on the feature point motion vector data, selects pixels whose offset exceeds the normal range, marks them as abnormal motion areas, and obtains the dynamic trajectory data of curtain wall defects; According to the motion vector data of the feature points, the motion vector distribution of all feature points is calculated, the overall change trend is analyzed, and the motion vector abnormal threshold is set to filter out the pixels whose offset exceeds the normal range. The threshold is usually set to the 95% quantile of all motion vectors. For example, if the 95% quantile value of all motion vector data calculated is 4.5, the abnormal motion vector threshold is set to 4.5, and all pixels with motion vector values ​​greater than 4.5 are marked. At the same time, the distribution of the pixel area is analyzed. If the motion vectors of multiple adjacent pixels exceed the threshold, it can be determined that there are curtain wall defects in the area. Finally, the abnormal motion area is marked to obtain the dynamic trajectory data of the curtain wall defects.

[0028] See also Figure 5 , the defect location module includes: The crack area coordinate extraction submodule analyzes the pixel coordinates of the curtain wall crack area based on the dynamic trajectory data of curtain wall defects, extracts the crack edge pixel points, constructs the two-dimensional coordinates of the crack area, and matches the crack boundary features in adjacent frames to obtain the two-dimensional coordinate data of the crack; In the image frame, the coordinates of each pixel point in the crack area are represented by the horizontal coordinate and the vertical coordinate. For example, in a certain frame of the image, a crack edge pixel point is located at (120, 300), and in the next frame, the pixel point is located at (122, 305). The position information is stored, and the boundary change trend of the crack is analyzed by comparing the changes between adjacent frames. On this basis, the pixel points at the crack edge are screened, isolated noise points are removed, and the pixel gradient information is used to optimize the boundary extraction of the crack area. For example, if the pixel gradient in a certain area changes abnormally, and the point deviates greatly from the overall trend of the crack edge, the point can be determined as an interference pixel and is eliminated. After obtaining the pixel coordinates of the crack area, the two-dimensional coordinate data of the crack area is constructed, and the crack boundary features in adjacent frames are compared using the inter-frame matching method to ensure the continuity of the boundary information of the crack area.

[0029] The crack depth analysis submodule is based on the two-dimensional crack coordinate data and the depth information of the image frame, using the formula: ; Calculate the depth offset of the crack area relative to the reference point of the curtain wall structure to obtain the crack depth data, where: Represents crack pixels The depth value of , and are the three-dimensional coordinates of the crack pixels, , and The three-dimensional coordinates of the reference point of the curtain wall structure; Call the depth information of the image frame, assign depth values ​​to each pixel in the crack area, and calculate the depth offset of the crack area relative to the reference point of the curtain wall structure. If the coordinates of a crack pixel are (500, 620, 12.5) and the coordinates of the curtain wall reference point are (480, 600, 10.0), substitute them into the calculation: ; The result shows that the depth offset of the crack pixel is 0.088. If the normal range of crack depth offset is 0.02 to 0.15, the depth change of the crack point is within the normal range, indicating that the depth of the crack at this position has not changed abnormally. Therefore, this pixel can be regarded as part of the stable crack area and does not need to be marked as abnormal.

[0030] The defect spatial distribution analysis submodule aligns the three-dimensional coordinates of the crack area with the curtain wall structure coordinate system based on the crack depth data, identifies the expansion trend of the crack area in three-dimensional space, and screens the curtain wall area with abnormal stress distribution to obtain the defect spatial distribution parameters; The three-dimensional coordinate data of the crack area is called and aligned with the curtain wall structure coordinate system to construct a spatial distribution model of the curtain wall crack. The expansion trend of the crack area in three-dimensional space is analyzed, and the depth gradient change of all crack areas is calculated. If the crack depth gradient change in a certain area exceeds the set abnormal threshold, the area is marked as a stress concentration area. The abnormal threshold can be set based on the statistical analysis of historical crack data. For example, if the normal crack depth gradient change range is 0.02 to 0.15, then when the depth gradient change in a certain area exceeds 0.15, it can be determined as an abnormal area. By screening all abnormal areas, the curtain wall area with abnormal stress distribution is obtained, and the defect spatial distribution parameters are obtained.

[0031] See also Figure 6 , the defect identification module includes: The texture change analysis submodule extracts the grayscale distribution, gradient direction and edge contour information based on the defect spatial distribution parameters, calculates the local grayscale change rate and directional gradient distribution, and analyzes the texture changes of the curtain wall cracks and metal corrosion areas to obtain the texture parameters of the defect area; The grayscale distribution, gradient direction and edge contour information of each area are separated from the acquired image data. In the image processing process, the grayscale value of each pixel is represented by 0 to 255, where 0 represents the darkest and 255 represents the brightest. By comparing the grayscale value changes of the crack area and the surrounding normal area, the local grayscale change rate is calculated. For example, at a pixel in a crack area, and adjacent pixels are 180 and 200 respectively, then the grayscale change rate of the area is calculated as , if the value is greater than the preset threshold of 0.10, it can be determined that there is an abnormal grayscale change in the area. In addition, in order to further analyze the texture features and calculate the directional gradient distribution, the gradient values ​​in the x and y directions are calculated by the Sobel operator. For example, if the x-direction gradient of a region is 50 and the y-direction gradient is 80, the gradient amplitude can be calculated as , the gradient direction angle is ,By calculating the gradient value of each pixel point and performing statistics on the gradient direction, ,the texture changes of the crack and metal corrosion areas are analyzed, and the texture parameters of the defect area ,are obtained.

[0032] The defect morphology comparison submodule extracts the defect boundary contour based on the defect area texture parameters and determines the area, perimeter, compactness and morphology ratio of the defect shape parameters using the formula: ; Calculate the normalized difference value of the defect shape , and obtain the defect area morphological parameters, where is the perimeter of the defect area, is the area of ​​the defect region, For the The gray value of a pixel, is the average gray value of the defect area, is the number of pixels in the defect area; The closed contour of the defect area is obtained through edge detection, and the area, perimeter, compactness and morphological ratio of the defect are calculated. The area can be calculated by pixel counting method. For example, if a defect area contains 500 pixels, each pixel corresponds to 0.1mm 2 The actual area of ​​the defect is: , the perimeter is obtained by calculating the total number of edge pixels. If the boundary of the area contains 100 pixels, the perimeter is: , where if the gray value of the pixel point in the defect area is , then the mean is: ; Calculate the grayscale variance: ; Calculate the standard deviation: ; Calculate compactness: ; Calculate the normalized grayscale difference: ; Calculated: ; This result shows that the normalized difference value of the morphological parameters of the defect area If the normal range of the morphological normalization difference is 0.5 to 1.5, the defect morphological parameter is within this range, indicating that the morphological characteristics of the defect are within the range of known defect types and can be used to further match the defect category. If it exceeds 1.5, it indicates that the defect belongs to the abnormal category or the unrecorded category, and further analysis is needed to determine whether there is a new defect type or measurement error.

[0033] The defect category matching submodule matches the defect type in the known defect data based on the defect area morphological parameters, analyzes the matching degree of the differentiated category defects, and selects the defect categories with the key matching degree to obtain the curtain wall defect category identification; According to the defect area morphological parameters, the defect type data in the known defect data is called to perform defect matching calculation. For each category of known defect data, the matching degree is calculated. If the known morphological parameters of a certain category of defects are , then calculate the difference value with the current defect If the difference value is less than the preset matching threshold value of 0.02, the defect is determined to match the category, otherwise other defect categories are matched, and finally the defect category with the key matching degree is screened to obtain the curtain wall defect category identification.

[0034] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. The curtain wall defect detection system based on computer vision is characterized by: The system comprises: The image acquisition module collects image data of the curtain wall and the curtain wall connection component area based on the deployed fixed cameras and adjustable angle cameras, analyzes the overlapping area of ​​adjacent image frames, and obtains a multi-angle curtain wall data set; The texture analysis module divides the curtain wall panel and the curtain wall aluminum alloy frame into pixel blocks based on the multi-angle curtain wall data set, analyzes the grayscale distribution of each pixel block, identifies the area with abnormal grayscale changes, and obtains local texture feature information; The dynamic defect tracking module analyzes the motion trajectory of the pixels in the image frame according to the local texture feature information, selects the pixels whose offset exceeds the normal range, marks them as abnormal motion areas, and obtains the dynamic trajectory data of the curtain wall defects; The defect location module analyzes the depth information of the crack area based on the dynamic trajectory data of the curtain wall defect, aligns the three-dimensional coordinates of the crack area with the curtain wall structure coordinate system, identifies the spatial distribution trend of the crack area, and obtains the defect spatial distribution parameters; The defect recognition module analyzes the texture changes of the curtain wall cracks and the metal corrosion area based on the defect spatial distribution parameters, identifies the defect category with a critical matching degree, and obtains the curtain wall defect category identification.

2. The curtain wall defect detection system based on computer vision according to claim 1, characterized in that: The multi-angle curtain wall data set includes a timestamp set, a shooting angle set, and a position coordinate set. The local texture feature information includes a reflection texture description, a corrosion texture description, and an abnormal grayscale area. The curtain wall defect dynamic trajectory data includes a crack trajectory, a sealant trajectory, and an abnormal mark. The defect spatial distribution parameters include crack depth, crack coordinates, and stress area. The curtain wall defect category identifier includes shape parameters, texture change type, and matching defect category.

3. The curtain wall defect detection system based on computer vision according to claim 1, characterized in that: The image acquisition module comprises: The synchronization parameter setting submodule obtains the shooting angle, timestamp and position coordinate information of the camera based on the deployed fixed cameras and adjustable angle cameras, sets the synchronization shooting parameters between the cameras, calculates the time synchronization error, viewing angle difference and spatial offset value between each camera, selects the cameras whose synchronization error and viewing angle offset are within the normal range, and obtains the synchronization camera parameter set; The image integrity assessment submodule parses the timestamp, shooting angle and position coordinate information of each frame of the image based on the synchronized camera parameter set, using the formula: ； Calculate the integrity index of image data , get the complete image data, where and Respectively represent Frame and The timestamp of the frame image, is the maximum allowed difference in timestamps, and Respectively represent Frame and The shooting angle of the frame image, is the maximum permissible difference in shooting angles, Represents the total number of image frames; The image matching and screening submodule analyzes the overlapping areas of adjacent image frames based on the complete image data, determines the matching degree between the image frames, removes the image frames whose edge matching degree is lower than the normal range, and obtains a multi-angle curtain wall data set.

4. The curtain wall defect detection system based on computer vision according to claim 1, characterized in that: The texture analysis module comprises: The pixel block division submodule collects image data of the curtain wall panel and the curtain wall aluminum alloy frame based on the multi-angle curtain wall data set, performs pixel-level segmentation on the image, divides the curtain wall image into multiple pixel blocks using a uniform grid division method, extracts the grayscale distribution data of each pixel block, and then calculates the grayscale mean and variance of the pixel block to obtain the grayscale statistical data of the pixel block; The grayscale gradient calculation submodule calculates the grayscale gradient between adjacent pixel blocks based on the grayscale statistics of the pixel blocks, using the formula: ； Get the pixel block grayscale gradient data, where Represents a pixel block With pixel blocks The grayscale gradient between and Represents pixel blocks With pixel blocks The grayscale mean, and is the grayscale variance of the corresponding pixel block, is the smoothing factor; The abnormal area identification submodule calls the grayscale gradient data of the pixel block, performs global statistics on the grayscale gradient, optimizes the global grayscale histogram, screens the pixel block areas with abnormal grayscale gradient changes, and combines the curtain wall surface reflection texture and metal component corrosion texture characteristics to mark the grayscale abnormal areas and obtain local texture feature information.

5. The curtain wall defect detection system based on computer vision according to claim 1, characterized in that: The dynamic defect tracking module includes: The pixel point motion analysis submodule obtains time series image frames according to the local texture feature information, extracts the coordinates of the pixels in each frame, establishes a time series coordinate sequence of the pixels, and calculates the displacement of the pixels between adjacent frames to obtain the pixel point motion trajectory; The feature point matching submodule calls the pixel point motion trajectory to match the curtain wall crack edge pixel points with the curtain wall sealant peeling area pixel points in each frame, using the formula: ； Calculate the motion vector of each feature point between consecutive frames to obtain the feature point motion vector data, where: Representative feature points The motion vector between two frames, and The feature points are The horizontal and vertical coordinates of the frame, and For the The corresponding coordinates of the frame; The abnormal area marking submodule analyzes the motion vector distribution of the feature points based on the feature point motion vector data, selects pixels whose offsets exceed the normal range, marks them as abnormal motion areas, and obtains dynamic trajectory data of curtain wall defects.

6. The curtain wall defect detection system based on computer vision according to claim 1, characterized in that: The defect location module includes: The crack region coordinate extraction submodule analyzes the pixel coordinates of the curtain wall crack region based on the curtain wall defect dynamic trajectory data, extracts the crack edge pixel points, constructs the two-dimensional coordinates of the crack region, and matches the crack boundary features in adjacent frames to obtain the crack two-dimensional coordinate data; The crack depth analysis submodule is based on the two-dimensional coordinate data of the crack and the depth information of the image frame, using the formula: ； Calculate the depth offset of the crack area relative to the reference point of the curtain wall structure to obtain the crack depth data, where: Represents crack pixels The depth value of , and are the three-dimensional coordinates of the crack pixels, , and The three-dimensional coordinates of the reference point of the curtain wall structure; The defect spatial distribution analysis submodule aligns the three-dimensional coordinates of the crack area with the curtain wall structure coordinate system based on the crack depth data, identifies the expansion trend of the crack area in the three-dimensional space, and screens the curtain wall area with abnormal stress distribution to obtain the defect spatial distribution parameters.

7. The curtain wall defect detection system based on computer vision according to claim 1, characterized in that: The defect recognition module comprises: The texture change analysis submodule extracts the grayscale distribution, gradient direction and edge profile information based on the defect spatial distribution parameters, calculates the local grayscale change rate and directional gradient distribution, and analyzes the texture changes of the curtain wall cracks and metal corrosion areas to obtain the texture parameters of the defect area; The defect morphology comparison submodule extracts the defect boundary contour based on the texture parameters of the defect area, determines the area, perimeter, compactness and morphology ratio of the defect shape parameters, and adopts the formula: ； Calculate the normalized difference value of the defect shape , and obtain the defect area morphological parameters, where is the perimeter of the defect area, is the area of ​​the defect region, For the The gray value of a pixel, is the average gray value of the defect area, is the number of pixels in the defect area; The defect category matching submodule matches the defect type in the known defect data according to the defect area morphological parameters, analyzes the matching degree of the differentiated category defects, and screens the defect categories with the key matching degree to obtain the curtain wall defect category identification.

Images