Curtain wall defect identification method based on image segmentation and picture similarity

Abstract

The invention discloses a curtain wall defect identification method based on image segmentation and picture similarity, and the method specifically comprises the following steps: 1, carrying out the image segmentation based on an OfficientSAM model, 2, carrying out the vector feature extraction, similarity calculation and clustering analysis of segmented pictures, 3, carrying out outlier confirmation and alarm, 4, carrying out the secondary defect detection and positioning of classified pictures, and 5, carrying out the recognition of the defects of the classified pictures. After segmentation and classification are completed, specific defect types in glass, stone, metal and mixed material panels are identified by using a YOLOv10 algorithm; and step 5, combining and confirming defects and abnormities, combining the step 3 and the step 4 to confirm the defects and the abnormities, accurately positioning the positions of the defects, generating a defect marking graph, and outputting a detection report. According to the method, the powerful segmentation capability of the OfficientSAM model and the accurate classification capability of similarity measurement are fully utilized, and the abnormal picture detection step is added, so that the automatic detection of the curtain wall defects is realized, the detection efficiency and accuracy are improved, the interference of human factors is reduced, and the method has a wide application prospect.

Description

Technical Field

[0001] The invention relates to a recognition method, in particular to a curtain wall defect recognition method based on image segmentation and picture similarity, and belongs to the technical field of image recognition. Background Art

[0002] With the continuous development of modern construction technology, curtain walls, as an important form of building exterior walls, are widely used in high-rise buildings. Curtain walls not only have the advantages of beauty, lightness, and heat insulation, but also can improve the overall performance of buildings. However, due to long-term exposure to the natural environment, curtain walls often face various damages and defects, such as glass cracks, stone damage, sealant cracking, metal parts rust, and foreign matter adhesion. These defects not only affect the appearance of the building, but also may pose a potential threat to structural safety. Therefore, it is particularly important to detect and identify curtain wall defects in a timely and effective manner.

[0003] Traditional curtain wall defect detection methods mainly rely on manual inspection, where inspectors identify defects through visual observation and manual recording. This method has the following main problems:

[0004] 1. Inefficiency: Manual inspection requires a lot of manpower and time. Especially for high-rise buildings, the inspection process is more cumbersome and complicated.

[0005] 2. Low accuracy: Manual testing is limited by the experience and subjective judgment of the testers, and is prone to missed detections and false detections, making it difficult to ensure the accuracy of the test results.

[0006] 3. Safety issues: Manual inspection often requires working at high altitudes, which poses a great safety risk and increases the difficulty of construction and maintenance.

[0007] 4. Poor repeatability: Different testers may have different judgment criteria, resulting in a lack of consistency and repeatability in the test results.

[0008] With the rapid development of computer vision and deep learning technology, it has become possible to use image recognition technology to automatically detect curtain wall defects. Through image segmentation and similarity calculation, the curtain wall surface can be finely analyzed, and various defects can be automatically identified and classified, which greatly improves the detection efficiency and accuracy. However, the existing automated detection methods still have some shortcomings, including:

[0009] 1. Unable to detect unlabeled defects: Many existing automated detection methods rely on pre-labeled defect samples for training, resulting in the inability to effectively identify new unlabeled defects.

[0010] 2. The picture taken is too large, and simple segmentation leads to low recognition rate: Although high-resolution pictures can provide more details, if there is no appropriate segmentation strategy, the algorithm may be inefficient when processing large pictures, resulting in a decrease in recognition rate.

[0011] 3. Lack of comprehensive analysis capabilities: Existing methods often focus on the detection of a single type of defect, lack the ability to comprehensively analyze and identify multiple types of defects, and are unable to comprehensively evaluate the overall condition of the curtain wall. Summary of the invention

[0012] In view of the above problems, the present invention proposes a curtain wall defect recognition method based on image segmentation and picture similarity. By introducing the EfficientSAM model for image segmentation and combining advanced similarity measurement and deep learning algorithm, it aims to achieve efficient and accurate curtain wall defect detection and improve the automation and reliability of the detection process.

[0013] In order to achieve the above object, the technical solution of the present invention is as follows: a curtain wall defect recognition method based on image segmentation and picture similarity, the method comprising the following steps:

[0014] Step 1: Perform image segmentation based on the EfficientSAM model, and use the open source Meta EfficientSAM (Segment Anything Model) model to perform fine-grained image segmentation on the preprocessed photos to obtain the segmentation results of each structural part of the curtain wall surface. This model has powerful segmentation capabilities and can process high-resolution images and generate accurate segmentation results.

[0015] Step 2: Vector extraction and cluster analysis of segmented images, similarity calculation and cluster analysis of segmented images. For the segmented images, use Baidu's open source PPLCNetV2_base as a feature extraction network to obtain feature values, and finally perform cluster analysis (obtain image similarity as distance through cosine similarity) to classify the segmented images. The specific instructions are as follows:

[0016] (1) Image feature extraction:

[0017] Use Baidu's open source PPLCNetV2_base as the feature extraction network to extract features from each segmented image. The specific steps are as follows: Load the PPLCNetV2_base model and pre-train it. Input the segmented image into the model to obtain its feature vector.

[0018] (2) Clustering algorithm:

[0019] After obtaining the similarity of all segmented images, we use clustering algorithms to classify the segmented images in order to better identify and locate curtain wall defects.

[0020] Hierarchical clustering algorithm description:

[0021] 1) Initialization: Extract features from each image and initialize it as a separate cluster.

[0022] 2) Merge: Calculate the cosine similarity between all clusters, find the two clusters with the smallest distance and merge them.

[0023] 3) Update: Recalculate the distance between the new cluster and other clusters.

[0024] 4) Iteration: Repeat the merging and updating steps until all images are merged into a single cluster or a predetermined number of levels is reached.

[0025] (3) Cosine similarity calculation:

[0026] 1) Use the cosine similarity algorithm to calculate the similarity of the feature vectors of two images. The specific formula is as follows:

[0027]

[0028] Among them, a and b are the feature vectors of the two pictures respectively, . represents the dot product operation of the vectors, and ∥a∥ and ∥b∥ are the L2 norms of the vectors respectively.

[0029] 2) Calculate the similarity percentage and convert the result into percentage form:

[0030]

[0031] 3) The specific implementation of Python code is as follows:

[0032] def diff_two(self,image_path1,image_path2):

[0033] import numpy as np

[0034] a=self._get_pic_var(image_path1)

[0035] b=self._get_pic_var(image_path2)

[0036] cos=np.dot(a,b) / (np.linalg.norm(a)*np.linalg.norm(b))

[0037] percent=100*(cos+1) / 2

[0038] print(percent)# Output similarity percentage, such as 97.97958971132711

[0039] return percent

[0040] Through the above steps, we can effectively classify and cluster the segmented images and provide basic data for subsequent defect detection and positioning.

[0041] Step 3: Outlier confirmation and alarm: Perform similarity analysis on all segmented images. If a segmented image is found to have low similarity with all other images, it will be confirmed as an abnormal image. An alarm will be issued for abnormal images, and further confirmation will be performed manually to ensure the accuracy of the detection results.

[0042] Step 4: Secondary defect detection and location of classified images: After segmentation and classification, the classified images are subjected to secondary defect detection using the YOLOv10 algorithm to identify specific defect types in glass, stone, metal, and mixed material panels, including:

[0043] Hidden dangers of glass curtain walls: scratches, cracks, fragments, glass falling off, worn edges and corners, and fogging, mildew and condensation of insulating glass.

[0044] Hidden dangers of stone curtain walls: stone breakage, cracks, corner chipping and panel falling and damage.

[0045] Sealant hidden dangers: hardening, bubbling, debonding and cracking problems.

[0046] Hidden dangers of metal components: severe and minor rust.

[0047] Hazards of foreign matter attachment: Bird droppings, plant growth, dust, dirt and other debris.

[0048] Step 5: Combine and confirm defects and anomalies: Combine the results of steps 3 and 4 to accurately locate the defect, generate a defect annotation map, and output a test report.

[0049] Compared with the prior art, the present invention has the following advantages:

[0050] 1. High-precision image segmentation: The open source Meta EfficientSAM (Segment Anything Model) model is used for image segmentation, which can process high-resolution images and generate accurate segmentation results, ensuring the fine-grained segmentation of each structural part of the curtain wall and improving the accuracy of defect identification.

[0051] 2. Advanced similarity calculation: Using Baidu's open source PPLCNetV2_base model as the feature extraction network, by extracting image feature vectors and calculating cosine similarity, it can effectively measure the similarity between segmented images. This method not only improves the accuracy of similarity calculation, but also effectively identifies abnormal images with low similarity.

[0052] 3. The inability of existing methods to discover unknown defects: Perform similarity analysis on the segmented images, identify abnormal images by detecting outliers with low similarity, and perform alarm processing. This step can effectively discover new unlabeled defects, making up for the inability of existing methods to discover unknown defects.

[0053] 4. Comprehensive detection of multiple types of defects: After segmentation and classification, the YOLOv10 algorithm is used for secondary defect detection, which can identify multiple types of defects such as glass, stone, metal, and mixed material panels. Specific identification includes: glass curtain wall hazards: such as scratches, cracks, fragments, glass pieces falling off, corner wear, and fogging, mildew, and condensation of insulating glass.

[0054] Potential problems with stone curtain walls: such as stone breakage, cracks, corner loss, and panel falling and defect. Potential problems with sealants: such as hardening, blistering, debonding, and cracking. Potential problems with metal components: such as severe and minor rust. Potential problems with foreign matter attachment: such as bird droppings, plant growth, dust, dirt, and other debris.

[0055] 5. Defect location and report generation: This method can accurately locate the defect position, generate defect annotation map, and output detailed inspection report. This function greatly improves the intuitiveness and practicality of the inspection results, and facilitates subsequent maintenance and repair work.

[0056] 6. Automation and reliability: By introducing deep learning algorithms and automated similarity calculation, this method can greatly improve detection efficiency and accuracy, reduce interference from human factors, and has a high degree of automation and reliability.

[0057] In summary, the present invention has significant advantages in image segmentation, similarity calculation, cluster analysis, defect detection, etc. It makes full use of the powerful segmentation ability of the EfficientSAM model and the accurate classification ability of the similarity metric, and combines the deep learning algorithm to achieve efficient and accurate detection of curtain wall defects. By introducing the abnormal image detection step, it can effectively identify new unlabeled defects, overcoming the shortcomings of the existing methods. This method greatly improves the detection efficiency and accuracy, reduces the interference of human factors, and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a schematic diagram of the overall steps of Example 2 of the present invention; DETAILED DESCRIPTION

[0059] In order to deepen the understanding of the present invention, the present embodiment is described in detail below with reference to the accompanying drawings.

[0060] Embodiment 1: A curtain wall defect recognition method based on image segmentation and picture similarity, the method comprising the following steps:

[0061] Step 1: Perform image segmentation based on the EfficientSAM model, and use the open source Meta EfficientSAM (Segment Anything Model) model to perform fine-grained image segmentation on the preprocessed photos to obtain the segmentation results of each structural part of the curtain wall surface. This model has powerful segmentation capabilities and can process high-resolution images and generate accurate segmentation results.

[0062] Step 2: Vector extraction and cluster analysis of segmented images, similarity calculation and cluster analysis of segmented images. For the segmented images, use Baidu's open source PPLCNetV2_base as a feature extraction network to obtain feature values, and finally perform cluster analysis (obtain image similarity as distance through cosine similarity) to classify the segmented images. The specific instructions are as follows:

[0063] (1) Image feature extraction:

[0064] Use Baidu's open source PPLCNetV2_base as the feature extraction network to extract features from each segmented image. The specific steps are as follows: Load the PPLCNetV2_base model and pre-train it. Input the segmented image into the model to obtain its feature vector.

[0065] (2) Clustering algorithm:

[0066] After obtaining the similarity of all segmented images, we use clustering algorithms to classify the segmented images in order to better identify and locate curtain wall defects.

[0067] Hierarchical clustering algorithm description:

[0068] 1) Initialization: Extract features from each image and initialize it as a separate cluster.

[0069] 2) Merge: Calculate the cosine similarity between all clusters, find the two clusters with the smallest distance and merge them.

[0070] 3) Update: Recalculate the distance between the new cluster and other clusters.

[0071] 4) Iteration: Repeat the merging and updating steps until all images are merged into a single cluster or a predetermined number of levels is reached.

[0072] (3) Cosine similarity calculation:

[0073] 1) Use the cosine similarity algorithm to calculate the similarity of the feature vectors of two images. The specific formula is as follows:

[0074]

[0075] Among them, a and b are the feature vectors of the two pictures respectively, . represents the dot product operation of the vectors, and ∥a∥ and ∥b∥ are the L2 norms of the vectors respectively.

[0076] 2) Calculate the similarity percentage and convert the result into percentage form:

[0077]

[0078] 3) The specific implementation of Python code is as follows:

[0079] def diff_two(self,image_path1,image_path2):

[0080] import numpy as np

[0081] a=self._get_pic_var(image_path1)

[0082] b=self._get_pic_var(image_path2)

[0083] cos=np.dot(a,b) / (np.linalg.norm(a)*np.linalg.norm(b))

[0084] percent=100*(cos+1) / 2

[0085] print(percent)# Output similarity percentage, such as 97.97958971132711

[0086] return percent

[0087] Through the above steps, we can effectively classify and cluster the segmented images and provide basic data for subsequent defect detection and positioning.

[0088] Step 3: Outlier confirmation and alarm: Perform similarity analysis on all segmented images. If a segmented image is found to have low similarity with all other images, it will be confirmed as an abnormal image. An alarm will be issued for abnormal images, and further confirmation will be performed manually to ensure the accuracy of the detection results.

[0089] Step 4: Secondary defect detection and location of classified images: After segmentation and classification, the classified images are subjected to secondary defect detection using the YOLOv10 algorithm to identify specific defect types in glass, stone, metal, and mixed material panels, including:

[0090] Hidden dangers of glass curtain walls: scratches, cracks, fragments, glass falling off, worn edges and corners, and fogging, mildew and condensation of insulating glass.

[0091] Hidden dangers of stone curtain walls: stone breakage, cracks, corner chipping and panel falling and damage.

[0092] Sealant hidden dangers: hardening, bubbling, debonding and cracking problems.

[0093] Hidden dangers of metal components: severe and minor rust.

[0094] Hazards of foreign matter attachment: Bird droppings, plant growth, dust, dirt and other debris.

[0095] Step 5: Combine and confirm defects and anomalies: Combine the results of steps 3 and 4 to accurately locate the defect, generate a defect annotation map, and output a test report.

[0096] Example 2: See Figure 1 As shown, a curtain wall defect recognition method based on image segmentation and picture similarity includes the following steps:

[0097] Step 1: For the local image S1 of the curtain wall, image segmentation is performed using the open source Meta EfficientSAM (Segment Anything Model) model to obtain 10 images, namely, image S01, image S02, image S03, image S04, image S05, image S06, image S07, image S08, image S09, and image S10.

[0098] Step 2, by using Baidu's open source PPLCNetV2_base as a feature extraction network for the segmented image, obtain the eigenvalues, and then cluster them through hierarchical clustering to obtain three clusters J1 [Picture S01, Picture S06], J2 [Picture S02, Picture S03, Picture S05, Picture S07, Picture S08, Picture S09, Picture S10], and J3 [Picture S04];

[0099] Step 3, cluster J3 [Picture S04] is a picture with only one image, which is confirmed to be an abnormality;

[0100] Step 4: For 10 pictures, picture S01, picture S02, picture S03, picture S04, picture S05, picture S06, picture S07, picture S08, picture S09, and picture S10, the classified pictures are subjected to secondary defect detection by using the YOLOv10 algorithm to obtain abnormal pictures S01 (stone corner drop) and S7 (glass scratches);

[0101] Step 5, merge and confirm defects and anomalies: Combine the results of step 3 and step 4, abnormal picture S01 (stone corner falling), abnormal picture S04 (abnormal, manually confirmed as glass falling), picture S7 (glass scratches), and locate and mark the position of the picture in the original picture, generate a defect annotation map, and output the inspection report.

[0102] It should be noted that the above embodiments are not intended to limit the protection scope of the present invention, and equivalent changes or substitutions made on the basis of the above technical solutions all fall within the protection scope of the claims of the present invention.

Claims

1. A curtain wall defect recognition method based on image segmentation and picture similarity, characterized in that: The method comprises the following steps: Step 1: Image segmentation based on the EfficientSAM model. Step 2: Vector feature extraction, similarity calculation and cluster analysis of segmented images, Step 3: Outlier confirmation and alarm, Step 4: Secondary defect detection and location of classified images. After segmentation and classification, the YOLOv10 algorithm is used to identify specific defect types in glass, stone, metal, and mixed material panels. Step 5: Combine and confirm defects and anomalies. Combine steps 3 and 4 to confirm defects and anomalies, accurately locate the defect positions, generate defect annotation maps, and output inspection reports.

2. The curtain wall defect recognition method based on image segmentation and picture similarity according to claim 1 is characterized in that: Step 1: Perform image segmentation based on the EfficientSAM model, pre-process the collected local photos of the curtain wall, and use the open source Meta EfficientSAM (Segment Anything Model) model to perform fine-grained image segmentation on the photos to obtain the segmentation results of each structural part of the curtain wall surface.

3. The curtain wall defect recognition method based on image segmentation and picture similarity according to claim 1 is characterized in that: Step 2: Vector extraction and cluster analysis of segmented images, similarity calculation and cluster analysis of segmented images. For the segmented images, Baidu's open source PPLCNetV2_base is used as a feature extraction network to obtain feature values. Finally, cluster analysis is performed (the image similarity is obtained as the distance through cosine similarity) to classify the segmented images. The specific instructions are as follows: (1) Image feature extraction: Use Baidu's open source PPLCNetV2_base as the feature extraction network to extract features from each segmented image. The specific steps are as follows: Load the PPLCNetV2_base model and pre-train it. Input the segmented image into the model to obtain its feature vector. (2) Clustering algorithm: After obtaining the similarity of all segmented images, clustering algorithms are used to classify the segmented images in order to better identify and locate curtain wall defects. Hierarchical clustering algorithm description: 1) Initialization: Extract features from each image and initialize it as a separate cluster. 2) Merge: Calculate the cosine similarity between all clusters, find the two clusters with the smallest distance and merge them. 3) Update: recalculate the distance between the new cluster and other clusters, 4) Iteration: Repeat the merging and updating steps until all images are merged into a single cluster or the predetermined number of levels is reached. (3) Cosine similarity calculation: 1) Use the cosine similarity algorithm to calculate the similarity of the feature vectors of two images. The specific formula is as follows: Where a and b are the feature vectors of the two images, . represents the dot product operation of the vectors, ∥a∥ and ∥b∥ are the L2 norms of the vectors, 2) Calculate the similarity percentage and convert the result into percentage form: 3) The specific implementation of Python code is as follows: def diff_two(self,image_path1,image_path2): import numpy as np a=self._get_pic_var(image_path1) b=self._get_pic_var(image_path2) cos=np.dot(a,b) / (np.linalg.norm(a)*np.linalg.norm(b)) percent=100*(cos+1) / 2 print(percent)# Output similarity percentage, such as 97.97958971132711 return percent Through the above steps, the segmented images can be effectively classified and clustered, providing basic data for subsequent defect detection and positioning.

4. The curtain wall defect recognition method based on image segmentation and picture similarity according to claim 1 is characterized in that: Step 3: Outlier confirmation and alarm: Perform similarity analysis on all segmented images. If it is found that the similarity between a segmented image and all other images is very low, it will be confirmed as an abnormal image, an alarm will be issued for the abnormal image, and further confirmation will be performed manually to ensure the accuracy of the detection results.

5. The curtain wall defect recognition method based on image segmentation and picture similarity according to claim 1 is characterized in that: Step 4: Secondary defect detection and location of classified images: After segmentation and classification, the classified images are subjected to secondary defect detection using the YOLOv10 algorithm to identify specific defect types in glass, stone, metal, and mixed material panels, including: Glass curtain wall hazards: scratches, cracks, fragmentation, glass sheet detachment, edge wear, and fogging, mildew, and condensation of insulating glass; Hidden dangers of stone curtain wall: stone breakage, cracks, corner drop and panel falling and defect. Sealant hidden dangers: hardening, bubbling, debonding and cracking problems. Hidden dangers of metal components: severe and minor rust, Hazards of foreign matter attachment: Bird droppings, plant growth, dust, dirt and other debris.

Images