Building exterior wall defect detection method based on mixed feature enhancement and attention optimization

Abstract

The invention discloses a building outer wall defect detection method based on mixed feature enhancement and attention optimization, and relates to the technical field of image data processing, and the method comprises the steps: collecting an original image of a building outer wall, carrying out the slicing processing, obtaining a plurality of slice sub-images, and carrying out the screening, and obtaining a mixed data set; constructing a feature pyramid sampling model of the mixed data set image, and performing feature extraction, feature enhancement and information aggregation on the mixed data set image at multiple scales; generating a small target defect feature map of the building outer wall based on the extended feature pyramid sampling model; and positioning defect positions in the slice sub-images based on a defect feature map, and mapping the defect positions in the image to obtain defect positions and types in the building outer wall image. According to the invention, the technical problem of low accuracy caused by too small target of building outer wall defect detection in the prior art is solved, and the accuracy of irregular small target defect detection and classification is improved.

Description

Technical Field

[0001] The present invention relates to the field of target detection, and in particular to a building exterior wall defect detection method based on hybrid feature enhancement and attention optimization. Background Art

[0002] As a vital vehicle for human life and production, the quality of buildings is directly linked to their quality of life and safety. With the development of modernization, the scale of buildings continues to expand, and the complexity and diversity of building structures are increasing. However, due to environmental erosion, material aging, and other factors, building exterior walls often develop defects such as cracks, leaks, peeling, and corrosion. These defects not only affect the building's aesthetics but can also lead to structural safety issues, posing a threat to people's lives and property. Therefore, timely and accurate identification of building exterior wall defects has important engineering value and social significance for ensuring building safety and extending its service life.

[0003] Building defect detection is a long, repetitive process, and its efficiency and accuracy are directly linked to the development and implementation of building maintenance strategies. Traditional detection methods primarily rely on manual visual inspection, which is costly, inefficient, and highly dependent on the inspector's experience. Furthermore, due to subjective judgment, identification results vary between inspectors. Furthermore, for large buildings, especially high-rise buildings, manual inspections are labor-intensive and involve the risk of working at height, further limiting the practicality of traditional methods. In recent years, with the rapid development of computer vision and drone technology, automated inspection techniques based on image processing and machine learning have emerged as a promising approach to building exterior wall defect detection, opening up new technical paths for automating, streamlining, and refining the inspection process. These technologies offer new approaches for significantly improving detection efficiency and accurately classifying defect types, providing a scientific basis for building maintenance decisions.

[0004] In recent years, deep learning-based object detection technology has achieved breakthroughs in general scenarios. Existing object detection models are categorized as two-stage and single-stage. Two-stage detection frameworks, such as Faster-RCNN, RFCN, and FPN, first extract image features using convolutional neural networks (CNNs) and generate multiple candidate bounding boxes on the feature map. Each candidate box then undergoes additional convolution for object classification and bounding box refinement. Finally, non-maximum suppression is used to filter out redundant bounding boxes and determine the final detection result. Two-stage detection frameworks offer high accuracy and perform better in fine-grained and small object detection. However, they are slow and resource-intensive, making them commonly used in scenarios requiring high precision. Single-stage detection frameworks, such as YOLO, SSD, and RetinaNet, directly predict object categories and localize image locations, generating candidate bounding boxes accordingly. Compared to two-stage detection, these approaches offer fast prediction speed and high computational efficiency, but they perform poorly in complex scenes and suffer from lower prediction accuracy. They are therefore suitable for scenarios requiring high real-time performance.

[0005] With the application of drone inspection technology in building exterior wall maintenance, automated defect detection based on high-resolution images has greatly improved inspection efficiency. While drone-captured images of building exterior walls offer high resolution, exterior wall defects typically appear as small objects, such as cracks, peelings, and water seepage. These objects exhibit irregular shapes, diverse scales, low coverage, blurred edge information, and difficult-to-distinguish noise and clutter, all at low resolution. Furthermore, the complex and diverse building backgrounds, susceptible to environmental factors such as lighting, texture, and shadows, present a significant obstacle to inspection. Deep learning-based object detection has achieved significant breakthroughs in general scenarios, but it struggles to effectively distinguish small pixel objects from complex backgrounds and lacks sensitivity to multi-scale and irregular objects. This inherent limitation hinders efficient and accurate defect detection in building exterior wall images. Therefore, addressing the specific characteristics of building exterior wall defect detection, it is imperative to develop a technical solution that optimizes feature extraction networks, introduces a multi-layer feature fusion mechanism, and constructs a lightweight model to improve the recognition accuracy of small, irregular objects and meet the practical needs of industrial inspection. This technological breakthrough will provide strong technical support for the automation and intelligence of building exterior wall defect detection, and promote technological innovation and practical development in the field of building maintenance. Summary of the Invention

[0006] In view of the particularity of building exterior wall defect detection, this patent proposes a building exterior wall defect detection method based on hybrid feature enhancement and attention optimization, including: proposing a data enhancement method for small targets to increase the relative proportion of small target defects in the image; developing an attention feature extraction network to enrich the detailed information of small targets; introducing a multi-level feature fusion mechanism to enhance the small target detection capability by fusing feature information of different scales.

[0007] A first aspect of the present invention provides a building exterior wall defect detection method based on hybrid feature enhancement and attention optimization, the method comprising the following steps:

[0008] S1, collect exterior wall images and perform data preprocessing;

[0009] S2, overlapping and slicing the exterior wall image after data preprocessing to obtain slice sub-images;

[0010] S3, screening the slice sub-images to obtain a mixed data set containing the same number of defect target images and background information sub-images;

[0011] S4. Build a feature pyramid sampling model to extract features, enhance features, and aggregate information for mixed datasets:

[0012] First, global feature extraction is performed to obtain a global feature map; multi-level feature extraction is performed on the global feature map to obtain feature maps of different semantic levels; the feature maps of different semantic levels are upsampled and fused to obtain fused feature maps of different levels;

[0013] S5. Construct an extended feature pyramid sampling model to extract the texture and super-resolution content in the fused feature map to obtain a feature texture map, and then fuse the feature texture map with the global feature map to obtain a small target defect feature map of the building exterior wall;

[0014] S6. Train the feature pyramid sampling model and the extended feature pyramid sampling model based on the mixed data set to obtain a training model, and establish a mixed data set to be predicted for prediction to obtain a defect feature map for positioning, locate the defect position in the slice sub-image based on the defect feature map, map the defect position in the slice sub-image, and obtain the defect position and category in the original image of the building exterior wall.

[0015] Furthermore, the data preprocessing specifically includes: correcting or deleting incomplete parts to perform data cleaning, and performing data labeling on the image.

[0016] Furthermore, the overlapping slices specifically include:

[0017] S2.1. Setting the slice size and overlap ratio according to the size of the exterior wall image and the size of the defects in the exterior wall image;

[0018] S2.2, each image in the exterior wall image According to the size and overlap ratio of the slices, the image is sliced in sequence by the sliding window method and divided into multiple overlapping slice sub-images. Finally, a set of sliced sub-images is obtained.

[0019] Furthermore, the slice sub-images are screened as follows:

[0020] S3.1. Select all slice images containing defects in the slice sub-image set Constitute a positive sample set;

[0021] S3.2. Count the number i of slice sub-images contained in the positive sample set;

[0022] S3.3. Randomly select images from all images in the slice sub-image set that do not contain defects, the number of images is equal to the number of the positive sample set. Constitute a negative sample set;

[0023] S3.4, combining the images of the positive sample set and the negative sample set to obtain Construct a mixed data set.

[0024] Furthermore, the feature pyramid sampling model specifically includes: a hybrid attention transformer module, a multi-level feature extraction module, a spatial pyramid pooling module and a multi-scale feature fusion module;

[0025] The hybrid attention transformer module is used to perform global feature extraction on the input image, including shallow feature extraction, deep feature extraction and image reconstruction;

[0026] The multi-level feature extraction module extracts features from the initial feature map and gradually reduces the scale to obtain feature maps of different semantic levels, including four layers of intermediate feature maps;

[0027] The spatial pyramid pooling module performs multi-scale pooling on the fourth layer intermediate feature map to enhance the global feature expression capability and obtain the final depth feature map;

[0028] The multi-scale feature fusion module uses the final depth feature map as the first fused feature map, upsamples the first fused feature map to the same scale as the third intermediate feature map, and then splices and fuses it to obtain a second fused feature map; upsamples the second fused feature map to the same scale as the second intermediate feature map, and then splices and fuses it to obtain a third fused feature map; and upsamples the third fused feature map to the same scale as the first intermediate feature map, and then splices and fuses it to generate a fourth fused feature map.

[0029] Furthermore, the hybrid attention transformer module specifically includes: shallow feature extraction, using a 3×3 convolutional layer to extract shallow features F0, and using the following formula to obtain F0:

[0030] F0=H SF (I LQ )

[0031] Among them, H DF (·) indicates deep feature extraction, which is specifically expressed as:

[0032]

[0033] F DF =H Conv (FN)

[0034] in, represents the i-th residual hybrid attention module RHAG, H Conv (·) represents a 3×3 convolutional layer;

[0035] Specifically, the residual hybrid attention module consists of multiple hybrid attention blocks HAB, an overlapping cross attention block OCAB and a convolution block; HAB combines the window self-attention mechanism and the channel attention mechanism, combines the two in parallel, and controls the weight of the channel attention block CAB output through the weight factor α, which is specifically expressed as:

[0036] X M =(S)W-MSA(X N )+αCAB(X N )+X

[0037] X N =LN(X)

[0038] P=MLP(X M ))+X M

[0039] Among them, X is the input feature, X N is the normalized input feature, X M is the intermediate feature, LN is the layer normalization operation, MLP is the multi-layer perceptron, W-MSA is the window self-attention mechanism, and CAB is the channel attention block.

[0040] Furthermore, the S5 specifically includes:

[0041] S5.1. Based on the extended feature pyramid network (EFPN), feature maps of different scales are fused to improve the detection capability of small objects and generate high-level feature expressions.

[0042] S5.2. Use the third fused feature map P3 as the main input feature and the fourth fused feature map P2 as the reference input feature;

[0043] S5.3, embedding the main input feature and the reference input feature into the super-resolution module FFT to extract the texture feature map;

[0044] Specifically, the super-resolution module FFT uses the following formula to obtain the feature texture map P3′:

[0045] P′3=E t (P2||E c (P3)↑ 2× )+E c (P3)↑ 2×

[0046] Among them, E t (·) represents the texture extractor, E c (·) represents content extractor, || represents feature cascade, ↑ 2× Indicates 2x magnification through sub-pixel convolution;

[0047] S5.4. Upsample the feature texture map P3′ and fuse it with the initial feature map Y2′ to obtain an extended pyramid map P2′.

[0048] Furthermore, mapping the defect position in the slice sub-image specifically includes:

[0049] S6.1. Based on the mixed data set, use the neural network to train to obtain a training model, input the mixed test set into the training model, and obtain an extended pyramid map P2′, a fourth fused feature map P2, a third fused feature map P3, a second fused feature map P4, and a first fused feature map P5 of the slice image;

[0050] S6.2, P2 ′ ,P2, P3, P4, and P5 are respectively input to the target detection head to generate basic detection information, including bounding boxes, category probabilities, and confidence scores;

[0051] S6.3. Filter the basic detection information based on a non-maximum suppression algorithm to remove low-confidence targets and redundant detection boxes;

[0052] S6.4. Perform final classification and location of defect targets based on the classification scores and regression information, and output the category, location, and confidence level of the building exterior wall defect;

[0053] S6.5. Further correct the target boundary based on the post-processing optimization strategy to improve the detection accuracy and obtain the building exterior wall defect category;

[0054] S6.6. Convert the prediction result of the slice sub-image into the original image of the building exterior wall to obtain the defect category of the original image of the building exterior wall.

[0055] On the other hand, the present invention specification also provides a building exterior wall defect detection device based on hybrid feature enhancement and attention optimization, including a memory and one or more processors, wherein the memory stores executable code, and when the processor executes the executable code, it implements the building exterior wall defect detection method based on hybrid feature enhancement and attention optimization.

[0056] On the other hand, the present invention specification also provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, it implements the method for detecting building exterior wall defects based on hybrid feature enhancement and attention optimization.

[0057] Beneficial effects of the present invention: The present invention provides a building exterior wall defect detection method based on hybrid feature enhancement and attention optimization. In the present invention, by overlapping and slicing high-resolution building exterior wall images and screening the overlapping slice images, a mixed slice data set of 1:1 mixed positive and negative samples is obtained. While improving the relative size of the defect target in the image, the present invention retains the background information in the image, further improves the accuracy of small target defect detection, and also improves the training and recognition efficiency of small target defect detection; in addition, the present invention combines the extended pyramid feature map with the hybrid attention exchanger to extract features from building exterior wall images, retains a large amount of large-scale feature map information, and further refines the feature details of small target defects, enriches the features of small targets in the feature map, and does not affect the detection accuracy of large targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a flow chart of the building exterior wall defect detection method based on hybrid feature enhancement and attention optimization;

[0059] Figure 2 It is a structural schematic diagram of the image processing method based on slice screening of the present invention;

[0060] Figure 3 Schematic diagram of the model structure of the feature extraction, fusion and detection model based on hybrid feature enhancement and attention optimization of the present invention;

[0061] Figure 4 2 is a schematic structural diagram of the super-resolution module FTT of the present invention;

[0062] Figure 5 It is a schematic diagram of a building exterior wall defect detection device based on hybrid feature enhancement and attention optimization provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The specific embodiments of the present invention are further described in detail below with reference to the accompanying drawings.

[0064] In the description of the present invention, it is understood that the numbers in the step descriptions do not specify the order of the steps, but are only used to explain each step of the present invention and are not used to limit the present invention.

[0065] First, the terms involved in the present invention are explained:

[0066] Spatial Pyramid Pooling (SPPF) module: extracts multi-scale features and achieves efficient computation and multi-scale feature fusion by optimizing pooling operations, thereby enhancing the model's ability to detect targets of different scales.

[0067] Extended Feature Pyramid Network (EFPN): An improved feature pyramid network that uses a super-resolution module based on key reference features to extract small target detail information for improving target detection performance.

[0068] Super-resolution module FFT: A key feature reference-based super-resolution module for super-resolving features and extracting credible regional details simultaneously.

[0069] Upsample module: expands the image size and realizes the mapping of the image from small resolution to large resolution.

[0070] Concat module: combines feature maps at different levels and concatenates input tensors on the specified axis to achieve feature fusion and enhance the model's ability to understand the target.

[0071] Fusion module C2f: performs feature conversion, branch processing, feature fusion, extracts and converts data features, and generates more representative output.

[0072] Convolution module Conv: extracts features through convolution kernel operation.

[0073] Hybrid Attention Transformer Module (HAT): Fusion of channel attention and self-attention mechanisms to activate more pixels, improve the performance of image super-resolution reconstruction, and effectively integrate global pixel information.

[0074] Residual Hybrid Attention Module RHAG: multiple hybrid attention blocks (HABs), an overlapping crisscross attention block (OCAB) and a 3×3 convolutional layer with residual connections.

[0075] Reconstruction module REC: It consists of a pixel shuffle upsampling and two convolution combinations, and uses the pixel shuffle method to upsample the fused features.

[0076] Non-maximum suppression (NMS) algorithm: retains the highest confidence bounding box for each object and removes bounding boxes with high overlap. In object detection, it extracts the object detection box with high confidence and suppresses the box with low confidence.

[0077] Secondly, in order to facilitate those skilled in the art to understand the technical solutions provided in the embodiments of the present invention, Figure 1 Shown is an embodiment of the present invention

[0078] A building exterior wall defect detection method based on hybrid feature enhancement and attention optimization, comprising:

[0079] S1. Collect the original image of the building exterior wall.

[0080] Specifically, exterior wall images of different buildings are collected through image acquisition equipment such as drones and cameras, and data cleaning and data annotation are performed on the exterior wall images. The annotated building exterior wall images are then divided proportionally.

[0081] Data cleaning processes the raw data, improving its quality by correcting or deleting incomplete parts. This ensures that the acquired images include common building exterior wall defects, such as peeling, corrosion, and cracks. The cleaned images are then divided into a training set and a test set in a 7:3 ratio.

[0082] Data annotation involves labeling the detection targets in the image. Specifically, the standard LabelImg software is used for annotation: In LabelImg, select the data images one at a time to obtain the building exterior wall image information, including image width, length, category name, and image name. Drag a box to annotate each image, and export the annotated image information to a txt file. The txt file includes the defect category label and bounding box coordinates contained in the image.

[0083] S2. Slice the training set and the test set of the original image to obtain a slice training set and a slice test set consisting of a plurality of slice sub-images.

[0084] like Figure 2 As shown, as a preferred embodiment of the present invention, the steps of slicing the training set and the test set of the original image to obtain a slice training set and a slice test set composed of a plurality of slice sub-images include:

[0085] S2.1. Setting the slice size and overlap ratio according to the size of the original image and the size of the defect in the original image;

[0086] Specifically, according to the building exterior wall defect annotation information and the original image information of the original image in step S1, combined with the proportion of the defects in the original image, the resolution of the target slice image is set to increase the proportion of the defect target in the slice image, and the slice overlap rate is set according to the characteristics of the defect target in the original image, the size and number of the original image.

[0087] S2.2, each image in the original image According to the size and overlap ratio of the slice, it is divided into multiple overlapping slice sub-images Finally, a set of sliced sub-images is obtained.

[0088] Specifically, according to the size and overlap ratio of the slices, the slice window size and the sliding step size are set, and for each image, the image is sliced in sequence using the sliding window method.

[0089] The sliding window method is a traditional method in target detection, which slices the image by moving a fixed-size window on the image, scanning the image step by step.

[0090] S3. Screen the slice sub-images to obtain a mixed data set.

[0091] like Figure 2 As shown, as a preferred embodiment of the present invention, the steps of screening the slice sub-images to obtain a mixed data set include:

[0092] S3.1. Select all slice images containing defects in the slice sub-image set Constitute a positive sample set;

[0093] Specifically, since the number of building exterior wall defect pixels in the original image is low, most of the slice sub-images in the slice sub-image set do not contain building exterior wall defects. The slice images containing defect targets in the slice images are defined as positive samples, and all positive sample images are screened to obtain the positive sample set.

[0094] S3.2. Count the number i of slice sub-images contained in the positive sample set;

[0095] S3.3. Randomly select images from all images in the slice sub-image set that do not contain defects, the number of images is equal to the number of the positive sample set. Constitute a negative sample set;

[0096] Specifically, the slice sub-image that does not contain building exterior wall defects contains a large amount of background information. Feature extraction of the background information helps to distinguish the foreground information and background information of small target defects.

[0097] S3.4, combining the images of the positive sample set and the negative sample set to obtain A mixed data set is constructed to obtain a training mixed data set and a test mixed data set.

[0098] Specifically, for the slice training set and the slice test set, the positive samples and the negative samples are mixed in a 1:1 ratio to obtain a mixed data set, and a training mixed data set and a test mixed data set are obtained. While obtaining foreground information and background information to improve the detection accuracy, the reduction in training efficiency caused by the large data set is avoided.

[0099] S4. Perform feature extraction, feature enhancement, and information aggregation on the mixed dataset image at multiple scales to construct a feature pyramid sampling model of the mixed dataset image, wherein the feature pyramid sampling model includes multiple feature maps arranged according to scales.

[0100] like Figure 3 As shown, as a preferred embodiment of the present invention, the steps of performing feature extraction, feature enhancement, and information aggregation on the mixed dataset image at multiple scales and constructing a feature pyramid sampling model for the mixed dataset image include:

[0101] S4.1. Based on the hybrid attention transformer module HAT, global features are extracted from the input image, and the image is transformed through the feature extraction module to obtain the initial global feature map C2′;

[0102] Specifically, the hybrid attention transformer module HAT consists of shallow feature extraction, deep feature extraction, and image reconstruction. Among them, the shallow feature extraction uses a 3×3 convolutional layer to extract the shallow feature F0, and the following formula is used to obtain F0:

[0103] F0=H SF (I LQ )

[0104] Among them, H DF (·) indicates deep feature extraction, which is specifically expressed as:

[0105]

[0106] F DF =H Conv (F N )

[0107] in, represents the i-th residual hybrid attention module RHAG, H Conv (·) represents a 3×3 convolutional layer;

[0108] Specifically, the residual hybrid attention module consists of multiple hybrid attention blocks (HABs), an overlapping cross attention block (OCAB), and a convolution block. HAB combines the window self-attention mechanism and the channel attention mechanism, combining the two in parallel and controlling the weight of the channel attention block (CAB) output through the weight factor α, specifically expressed as:

[0109] X M =(S)W-MSA(X N )+αCAB(X N )+X

[0110] X N =LN(X)

[0111] Y=MLP(X M ))+X M

[0112] Among them, X is the input feature, X N is the normalized input feature, X M is the intermediate feature, LN is the layer normalization operation, MLP is the multi-layer perceptron, W-MSA is the window self-attention mechanism, and CAB is the channel attention block.

[0113] HAB increases the window size, introduces a channel attention mechanism, and integrates a parallel structure to expand information utilization, capture global information, and optimize stability. OCAB introduces an overlapping window mechanism to enhance window information interaction and improves feature representation capabilities by introducing a cross-attention mechanism.

[0114] Image reconstruction uses a global residual module to fuse the shallow features and the deep features, obtains high-quality results through the reconstruction module, and uses the following formula to obtain a refined feature map:

[0115] I HQ =H REC (F0+F DF )

[0116] Among them, H REC (·) represents the reconstruction module, which consists of a pixel shuffle upsampling and two convolution combinations.

[0117] The refined feature map is input into a convolutional layer for further feature extraction and conversion to obtain the initial global feature map C2′.

[0118] S4.2. Extract features from the initial feature map based on a multi-level feature extraction module, and gradually reduce the scale to obtain feature maps at different semantic levels, wherein the multi-level feature extraction module sequentially includes C2, C3, C4, and C5 structures;

[0119] Specifically, feature extraction and fusion are performed on feature maps of different scales through the convolutional layer Conv and the fusion block C2f structure, and intermediate feature maps are gradually obtained through the feature extraction unit, and the scale of the intermediate feature maps is reduced in sequence.

[0120] First intermediate feature map C2: extracts and fuses the first-level feature map based on the preliminary feature map, performs feature enhancement and scale reduction;

[0121] Second intermediate feature map C3: Extracts and fuses the second-level feature map based on the first-level feature map, further reducing the scale and enhancing the feature expression capability;

[0122] Third intermediate feature map C4: Extracts and fuses the third-level feature map based on the second-level feature map to further improve the feature representation capability;

[0123] Fourth intermediate feature map C5: Extracts and fuses the fourth-level feature map based on the third-level feature map to ensure rich semantic information;

[0124] S4.3. Based on the spatial pyramid pooling module SPPF, multi-scale pooling is performed on the fourth-layer intermediate feature map C5 to enhance the global feature expression capability and obtain the final deep feature map;

[0125] Specifically, the basic idea of the spatial pyramid pooling layer is to divide the input image into different levels, use different-sized pooling windows for each level, perform pooling separately, and then concatenate the pooling results of each level to obtain the final deep feature map. This allows pooling of input images of different sizes while preserving image information.

[0126] S4.4. Based on the multi-scale feature fusion module, feature maps of different scales are sampled and fused to obtain fused feature maps P2, P3, P4, and P5 of different scales.

[0127] Specifically, the shallow feature map contains more feature information of small targets, and the deep feature map contains more feature information of large targets. By fusing the deep and shallow features, targets of different sizes can be detected at different scales. The steps include:

[0128] S4.4.1. Determine the final depth feature map as the first fused feature map P5;

[0129] S4.4.2. Upsample the first fused feature map P5 to the same scale as the third intermediate feature map C4, and then concatenate and fuse the third intermediate feature map C4 to generate a second fused feature map P4.

[0130] S4.4.3. Upsample the second fused feature map P4 to the same scale as the second intermediate feature map C3, and then concatenate and fuse the second intermediate feature map C3 to generate a third fused feature map P3.

[0131] S4.4.4. After upsampling the third fused feature map P3 to the same scale as the first intermediate feature map C2, concatenating and fusing the first intermediate feature map C2 to generate a fourth fused feature map P2;

[0132] Among them, the feature upsampling units at each level use the upsampling module Upsample, use the Concat module to splice different feature maps, and use the C2f module to fuse features.

[0133] S5. Generate a small target defect feature map of the building exterior wall based on the extended feature pyramid sampling model.

[0134] As a preferred embodiment of the present invention, the step of generating the small target defect feature map of the building exterior wall based on the extended feature pyramid sampling model includes:

[0135] S5.1, such as Figure 3 As shown in the figure, feature maps of different scales are fused based on the extended feature pyramid network EFPN to improve the detection ability of small targets and generate high-level feature expressions;

[0136] S5.2. Use the third fused feature map P3 as the main input feature and the fourth fused feature map P2 as the reference input feature;

[0137] S5.3, embedding the main input feature and the reference input feature into the super-resolution module FFT to extract the texture feature map;

[0138] like Figure 4 As shown in the figure, specifically, P3 is used as the main input and input into the content extractor. The main semantic content is extracted through three convolution blocks, and then the resolution of the content features is doubled through sub-pixel convolution. Its output is input into the texture extractor together with the reference input P2. Reliable regional textures are selected to detect small targets. Finally, the texture is fused with the super-resolution content features through residual connection to finally obtain the feature texture map P3′.

[0139] The super-resolution module FFT uses the following formula to obtain the feature texture map P3′:

[0140] P′3=E t (P2||E c (P3)↑ 2× )+E c (P3)↑ 2×

[0141] Among them, E t (·) represents the texture extractor, E c (·) represents content extractor, || represents feature cascade, ↑ 2× Indicates 2x magnification through sub-pixel convolution.

[0142] S5.4. Upsample the feature texture map P3′ and fuse it with the initial feature map C2′ to obtain an extended pyramid map P2′.

[0143] S6. Perform training based on the mixed training set to obtain a training model, and perform prediction on the mixed test set to obtain defect feature map positioning, locate the defect position in the slice sub-image based on the defect feature map, map the defect position in the slice sub-image, and obtain the defect position and category in the original image of the building exterior wall.

[0144] As a preferred embodiment of the present invention, the steps of training based on the mixed training set to obtain a training model, predicting the mixed test set to obtain defect feature map positioning, locating the defect position in the slice sub-image based on the defect feature map, mapping the defect position in the slice sub-image, and obtaining the defect position and category in the original image of the building exterior wall include:

[0145] S6.1. Based on the mixed training set, use the neural network to train to obtain a training model, input the mixed test set into the training model, and obtain an extended pyramid map P2′, a fourth fused feature map P2, a third fused feature map P3, a second fused feature map P4, and a first fused feature map P5 of the slice image;

[0146] S6.2. Input P2′, P2, P3, P4, and P5 into the object detection head to generate basic detection information, including bounding boxes, class probabilities, and confidence scores.

[0147] Specifically, P2 ′ , P2, P3, P4, and P5 contain feature map information of different scales. By inputting feature maps of different scales into the detection head, targets of different sizes can be detected, thereby improving the detection accuracy.

[0148] S6.3. Filter the basic detection information based on the non-maximum suppression (NMS) algorithm to remove low-confidence targets and redundant detection boxes;

[0149] Specifically, the non-maximum suppression algorithm in target detection sorts bounding boxes in descending order according to their confidence, selects the highest bounding box from the sequence, and adds it to the final detection result list. For the remaining bounding boxes, the intersection-over-union ratio with the selected bounding box is calculated, and the bounding boxes with low confidence are eliminated. The bounding box with the highest confidence among the remaining bounding boxes is selected in turn, and finally an optimal bounding box corresponding to each target is obtained.

[0150] S6.4. Perform final classification and location of defect targets based on the classification scores and regression information, and output the category, location, and confidence level of the building exterior wall defect;

[0151] S6.5. Based on the post-processing optimization strategy, the target boundary is further corrected to improve the detection accuracy and obtain the building exterior wall defect category.

[0152] S6.6. Convert the prediction result of the slice sub-image into the original image of the building exterior wall to obtain the defect category of the original image of the building exterior wall.

[0153] Specifically, after mapping the predicted bounding boxes in the sliced sub-image to the original image, the overlapping bounding boxes are integrated to obtain the final detection result, and the evaluation index of the detection result is calculated using the following formula:

[0154]

[0155] Among them, TP represents the number of true positives; TN represents the number of true negatives; FN represents the number of false positives; FP represents the number of false negatives, Accuracy represents the accuracy, Precision represents the precision, Recall represents the recall rate, and F1 Score represents the F1 score. The higher the value of each parameter, the better the corresponding effect.

[0156] Corresponding to the aforementioned embodiment of a building exterior wall defect detection method based on hybrid feature enhancement and attention optimization, the present invention also provides an embodiment of a building exterior wall defect detection device based on hybrid feature enhancement and attention optimization.

[0157] See also Figure 5 An embodiment of the present invention provides a building exterior wall defect detection device based on hybrid feature enhancement and attention optimization, including a memory and one or more processors, wherein the memory stores executable code, and when the processor executes the executable code, it is used to implement a building exterior wall defect detection method based on hybrid feature enhancement and attention optimization in the above embodiment.

[0158] The embodiment of the building exterior wall defect detection device based on hybrid feature enhancement and attention optimization provided by the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The device embodiment can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory for execution. From the hardware level, if Figure 5 As shown, it is a hardware structure diagram of a device with data processing capability for detecting building exterior wall defects based on hybrid feature enhancement and attention optimization provided by the present invention, except Figure 5In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiment is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0159] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0160] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present invention. A person of ordinary skill in the art can understand and implement the present invention without inventive work.

[0161] An embodiment of the present invention also provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, a building exterior wall defect detection method based on hybrid feature enhancement and attention optimization in the above embodiment is implemented.

[0162] The computer-readable storage medium may be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or memory. The computer-readable storage medium may also be an external storage device of any device with data processing capabilities, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit and an external storage device of any device with data processing capabilities. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and may also be used to temporarily store data that has been output or is to be output.

[0163] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the method for detecting building exterior wall defects based on hybrid feature enhancement and attention optimization.

[0164] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the contents disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered merely as exemplary, and the true scope and spirit of the present application are indicated by the claims.

[0165] It should be understood that the above general description and the detailed description that follows are exemplary and explanatory only and do not limit the present application. The present application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from the scope of the present application. The scope of the present application is limited only by the appended claims.

Claims

1. A building exterior wall defect detection method based on hybrid feature enhancement and attention optimization, characterized in that: The method comprises the following steps: S1, collect exterior wall images and perform data preprocessing; S2, overlapping and slicing the exterior wall image after data preprocessing to obtain slice sub-images; S3, screening the slice sub-images to obtain a mixed data set containing the same number of defect target images and background information sub-images; S4. Build a feature pyramid sampling model to extract features, enhance features, and aggregate information for mixed datasets: First, global feature extraction is performed to obtain a global feature map; multi-level feature extraction is performed on the global feature map to obtain feature maps of different semantic levels; the feature maps of different semantic levels are upsampled and fused to obtain fused feature maps of different levels; S5. Construct an extended feature pyramid sampling model to extract the texture and super-resolution content in the fused feature map to obtain a feature texture map, and then fuse the feature texture map with the global feature map to obtain a small target defect feature map of the building exterior wall; S6. Train the feature pyramid sampling model and the extended feature pyramid sampling model based on the mixed data set to obtain a training model, and establish a mixed data set to be predicted for prediction to obtain a defect feature map for positioning, locate the defect position in the slice sub-image based on the defect feature map, map the defect position in the slice sub-image, and obtain the defect position and category in the original image of the building exterior wall.

2. The building exterior wall defect detection method based on hybrid feature enhancement and attention optimization according to claim 1 is characterized in that: The data preprocessing specifically includes: correcting or deleting incomplete parts to perform data cleaning, and performing data labeling on the image.

3. The building exterior wall defect detection method based on hybrid feature enhancement and attention optimization according to claim 1 is characterized in that: The overlapping slices specifically include: S2.

1. Setting the slice size and overlap ratio according to the size of the exterior wall image and the size of the defects in the exterior wall image; S2.2, each image in the exterior wall image According to the size and overlap ratio of the slices, the image is sliced in sequence by the sliding window method and divided into multiple overlapping slice sub-images. Finally, a set of sliced sub-images is obtained.

4. The building exterior wall defect detection method based on hybrid feature enhancement and attention optimization according to claim 1 is characterized in that: The specific steps of screening the slice sub-images are as follows: S3.

1. Select all slice images containing defects in the slice sub-image set Constitute a positive sample set; S3.

2. Count the number i of slice sub-images contained in the positive sample set; S3.

3. Randomly select images from all images in the slice sub-image set that do not contain defects, the number of images is equal to the number of the positive sample set. Constitute a negative sample set; S3.4, combining the images of the positive sample set and the negative sample set to obtain Construct a mixed data set.

5. The building exterior wall defect detection method based on hybrid feature enhancement and attention optimization according to claim 1 is characterized in that: The feature pyramid sampling model specifically includes: a hybrid attention transformer module, a multi-level feature extraction module, a spatial pyramid pooling module and a multi-scale feature fusion module; The hybrid attention transformer module is used to perform global feature extraction on the input image, including shallow feature extraction, deep feature extraction and image reconstruction; The multi-level feature extraction module extracts features from the initial feature map and gradually reduces the scale to obtain feature maps of different semantic levels, including four layers of intermediate feature maps; The spatial pyramid pooling module performs multi-scale pooling on the fourth layer intermediate feature map to enhance the global feature expression capability and obtain the final depth feature map; The multi-scale feature fusion module uses the final depth feature map as the first fused feature map, upsamples the first fused feature map to the same scale as the third intermediate feature map, and then splices and fuses it to obtain a second fused feature map; upsamples the second fused feature map to the same scale as the second intermediate feature map, and then splices and fuses it to obtain a third fused feature map; and upsamples the third fused feature map to the same scale as the first intermediate feature map, and then splices and fuses it to generate a fourth fused feature map.

6. The building exterior wall defect detection method based on hybrid feature enhancement and attention optimization according to claim 5 is characterized in that: The hybrid attention transformer module specifically includes: shallow feature extraction, using a 3×3 convolutional layer to extract shallow features F0, and using the following formula to obtain F0: F0=H SF (I LQ ) Among them, H DF (·) indicates deep feature extraction, which is specifically expressed as: F DF =H Conv (F N ) in, represents the i-th residual hybrid attention module RHAG, H Conv (·) represents a 3×3 convolutional layer; Specifically, the residual hybrid attention module consists of multiple hybrid attention blocks HAB, an overlapping cross attention block OCAB and a convolution block; HAB combines the window self-attention mechanism and the channel attention mechanism, combines the two in parallel, and controls the weight of the channel attention block CAB output through the weight factor α, which is specifically expressed as: X M =(S)W-MSA(X N )+αCAB(X N )+X X N =LN(X) Y=MLP(X M ))+X M Among them, X is the input feature, X N is the normalized input feature, X M is the intermediate feature, LN is the layer normalization operation, MLP is the multi-layer perceptron, W-MSA is the window self-attention mechanism, and CAB is the channel attention block.

7. The building exterior wall defect detection method based on hybrid feature enhancement and attention optimization according to claim 5 is characterized in that: The S5 specifically includes: S5.

1. Based on the extended feature pyramid network (EFPN), feature maps of different scales are fused to improve the detection capability of small objects and generate high-level feature expressions. S5.

2. Use the third fused feature map P3 as the main input feature and the fourth fused feature map P2 as the reference input feature; S5.3, embedding the main input feature and the reference input feature into the super-resolution module FFT to extract the texture feature map; Specifically, the super-resolution module FFT uses the following formula to obtain the feature texture map P′3: P′3=E t (P2||E c (P3)↑ 2× )+E c (P3)↑ 2× Among them, E t (·) represents the texture extractor, E c (·) represents content extractor, || represents feature cascade, ↑ 2× Indicates 2x magnification through sub-pixel convolution; S5.

4. Upsample the feature texture map P′3 and fuse it with the initial feature map C′2 to obtain an extended pyramid map P′2.

8. The building exterior wall defect detection method based on hybrid feature enhancement and attention optimization according to claim 7 is characterized in that: Mapping the defect position in the slice sub-image specifically includes: S6.

1. Based on the mixed data set, use the neural network to train to obtain a training model, input the mixed test set into the training model, and obtain an extended pyramid map P′2, a fourth fused feature map P2, a third fused feature map P3, a second fused feature map P4, and a first fused feature map P5 of the slice image; S6.

2. Input P′2, P2, P3, P4, and P5 into the object detection head to generate basic detection information, including bounding boxes, class probabilities, and confidence scores. S6.

3. Filter the basic detection information based on a non-maximum suppression algorithm to remove low-confidence targets and redundant detection boxes; S6.

4. Perform final classification and location of defect targets based on the classification scores and regression information, and output the category, location, and confidence level of the building exterior wall defect; S6.

5. Further correct the target boundary based on the post-processing optimization strategy to improve the detection accuracy and obtain the building exterior wall defect category; S6.

6. Convert the prediction result of the slice sub-image into the original image of the building exterior wall to obtain the defect category of the original image of the building exterior wall.

9. A building exterior wall defect detection device based on hybrid feature enhancement and attention optimization, comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that: When the processor executes the executable code, it implements a building exterior wall defect detection method based on hybrid feature enhancement and attention optimization as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, a building exterior wall defect detection method based on hybrid feature enhancement and attention optimization as described in any one of claims 1 to 8 is implemented.

Images