Image recognition and positioning method for building facade diseases

Abstract

The application provides a kind of building facade disease image recognition and positioning method, by unmanned aerial vehicle with camera lens is directly opposite the angle of building facade and is shot to obtain multiple building facade local images;Data set is obtained from positive image sample and negative image sample;Building facade disease image recognition network model is constructed, and building facade disease image recognition network model includes skeleton network, neck network and detection head network;The optimal building facade disease image recognition network model is obtained;Disease prediction image is obtained, and scale invariant feature transform algorithm SIFT and random sample consensus algorithm RANSAC are used for splicing to obtain the final building facade overall recognition image.The method can realize the rapid and accurate identification of building facade disease, can improve the detection efficiency, has relatively accurate identification effect, has strong practicality and engineering significance.

Description

Technical Field

[0001] This invention relates to an image recognition and localization method for defects on building facades, belonging to the field of image recognition technology. Background Technology

[0002] The presence of facade defects is a pressing issue during the building's operational phase, typically attributed to mechanical and environmental factors. Typical defects include cracks, peeling paint, and water seepage. These defects affect the building's appearance and reduce its lifespan. More seriously, peeling paint can cause safety accidents and irreparable damage. Structural damage detection, a component of structural health monitoring, is crucial for ensuring the safe operation of buildings. As part of structural damage detection, facade defect detection allows governments and management to accurately understand the overall condition of building facades, thus aiding in the development of appropriate maintenance plans. This is an effective way to reduce building maintenance costs, extend building lifespan, and mitigate the impact of facade damage. Many countries and regions are developing policies for regular, standardized visual inspections. Facade defect detection has become a key component of building maintenance.

[0003] Currently, visual inspection is a simple and reliable method for assessing the condition of building facades. Traditional building facade inspections typically require professionals to arrive at the inspection site with specialized tools and conduct assessments using visual observation, hammering, and other techniques. These methods rely on the inspector's expertise and experience, which is subjective, dangerous, and inefficient. Due to the increasing number and scale of buildings, manual visual inspection methods are no longer sufficient for large-scale inspections. With technological advancements, many new methods (such as laser scanning, 3D thermal imaging, and SLAM) are being used for exterior wall defect detection via drones and robotic platforms. Compared to traditional technologies, these new methods are more convenient and safer, but they are time-consuming, and their identification accuracy and detection efficiency are relatively low, posing challenges in meeting the needs of large-scale inspections. Therefore, it is necessary to develop a more accurate and efficient method for detecting building facade defects to improve detection efficiency and reduce computational costs. Summary of the Invention

[0004] The purpose of this invention is to provide an image recognition and localization method for building facade defects, which solves the problem that the recognition accuracy and efficiency of existing technologies need to be improved.

[0005] The technical solution of this invention is:

[0006] A method for image recognition and localization of defects on building facades, comprising the following steps:

[0007] S1. Multiple partial images of the building facade are captured by a drone with its camera lens facing the building facade directly.

[0008] S2. Filter the acquired partial images of the building facade. Use partial images of the building facade with defects as positive image samples and label the defect types of the partial images of the building facade. Use partial images of the building facade without defects as negative image samples. Obtain the dataset from the positive image samples and negative image samples, and divide the dataset into training set and test set according to a set ratio.

[0009] S3. Construct a network model for recognizing building facade defects. The network model includes a skeleton network, a neck network, and a detection head network. The skeleton network performs convolution, pooling, and vector concatenation on the input local image of the building facade to output feature vectors at three scales. The feature vectors at the three scales are input into the neck network and after convolution, upsampling, and vector concatenation, a feature map is obtained. The feature map is then input into the detection head network to obtain a defect prediction image, which includes a prediction image with a prediction box or an image without a prediction box, i.e., an image without defects.

[0010] S4. Using the training set and employing the SIoU loss function that considers angle loss, distance loss, and shape loss, train the building facade defect image recognition network model constructed in step S3 to obtain the trained building facade defect image recognition network model. Input the test set into the trained building facade defect image recognition network model, and optimize the hyperparameters of the trained building facade defect image recognition network model based on the test set results to obtain the optimal building facade defect image recognition network model.

[0011] S5. Input multiple partial images of the building facade to be identified into the optimal building facade defect image recognition network model to obtain multiple defect prediction images. Then, use the scale-invariant feature transformation algorithm SIFT and the random sampling consensus algorithm RANSAC to stitch them together to obtain the final overall recognition image of the building facade.

[0012] Furthermore, in step S1, the defects on the building facade include cracks, peeling of exterior walls, and / or water seepage in the exterior walls.

[0013] Furthermore, in the skeleton network, after the input partial image of the building facade enters the skeleton network, it sequentially passes through four convolutional normalization activation modules (CBS), the first high-efficiency layer aggregation network module (ELAN1), the first max pooling module (MP1), the second high-efficiency layer aggregation network module (ELAN2), the second max pooling module (MP2), the third high-efficiency layer aggregation network module (ELAN3), the third max pooling module (MP3), and the fourth high-efficiency layer aggregation network module (ELAN4). The second high-efficiency layer aggregation network module (ELAN2) outputs feature vector T1, the third high-efficiency layer aggregation network module (ELAN3) outputs feature vector T2, and the fourth high-efficiency layer aggregation network module (ELAN4) outputs feature vector T3, which are then fed to the neck network.

[0014] Furthermore, in the neck network, feature vector T3 is processed by the spatial pyramid pooling module SPPELAN of the efficient layer aggregation network to output feature vector T4. Feature vector T4 is then processed by the first convolutional normalization activation module CBS1 and the first upsampling module UPsample1 before being output to the first vector concatenation module Concat1. Simultaneously, feature vector T4 is input to the fourth vector concatenation module Concat4. Feature vector T2 is processed by the second convolutional normalization activation module CBS2 before being output to the first vector concatenation module Concat1. The first vector concatenation module Concat1 performs tensor concatenation and outputs feature vector T5.

[0015] Feature vector T5 is passed through the first weighted efficient layer aggregation network module ELAN-W1, the third convolutional normalization activation module CBS3, and the second upsampling module UPsample2 before being output to the second vector concatenation module Concat2. At the same time, feature vector T5 is passed through the first weighted efficient layer aggregation network module ELAN-W1 before being output to the third vector concatenation module Concat3. Feature vector T1 is passed through the fourth convolutional normalization activation module CBS4 before being output to the second vector concatenation module Concat2. The second vector concatenation module Concat2 concatenates the vectors and outputs feature vector T6.

[0016] Feature vector T6 is processed by the second weighted high-efficiency aggregation network module ELAN-W2 to output feature vector T7. Feature vector T7 is directly input into the detection head network. At the same time, feature vector T7 is processed by the fourth maximum pooling module MP4 and output to the third vector concatenation module Concat3. The third vector concatenation module Concat3 performs tensor concatenation to obtain feature vector T8.

[0017] Feature vector T8 is processed by the third weighted efficient layer aggregation network module ELAN-W3 to output feature vector T9. Feature vector T9 is directly input into the detection head network. At the same time, feature vector T9 is processed by the fifth maximum pooling module MP5 and output to the fourth vector concatenation module Concat4. The fourth vector concatenation module Concat4 performs tensor concatenation to obtain feature vector T10. Feature vector T10 is processed by the fourth weighted efficient layer aggregation network module ELAN-W4 to output feature vector T11 to the detection head network.

[0018] Furthermore, in the SPPELAN module of the high-efficiency layer aggregation network, after the feature vector T3 is input into the SPPELAN module, it passes through the fifth convolutional normalization activation module CBS5 and outputs feature vector t1. Feature vector t1 is directly input into the fifth concatenation module Concat5, and simultaneously input into the sixth max pooling module MP6. The sixth max pooling module MP6 outputs feature vector t2. Feature vector t2 is directly input into the fifth concatenation module Concat5, and simultaneously input into the seventh max pooling module MP7. The seventh max pooling module MP7 outputs feature vector t3. Feature vector t3 is directly input into the fifth concatenation module Concat5, and simultaneously input into the eighth max pooling module MP8. The eighth max pooling module MP8 outputs feature vector t4. Feature vector t4 is directly input into the fifth concatenation module Concat5, and the fifth concatenation module Concat5 outputs feature vector t5. Feature vector t5 passes through the sixth convolutional normalization activation module CBS6 to obtain feature vector T4.

[0019] Furthermore, in the detection head network, feature vector T7 is input to the first classification module Classification1 after passing through the first residual convolution normalization activation module REP+CBM1; feature vector T9 is input to the second classification module Classification2 after passing through the second residual convolution normalization activation module REP+CBM2; and feature vector T11 is input to the third classification module Classification3 after passing through the third residual convolution normalization activation module REP+CBM3. Finally, the detection head network outputs disease prediction images, including prediction images with prediction boxes or images without prediction boxes, i.e., images without diseases.

[0020] Further, in step S5, the scale-invariant feature transformation algorithm SIFT and the random sampling consensus algorithm RANSAC are used to stitch together the identified local images of the building facade to obtain the final overall image of the building facade. Specifically,

[0021] S51. The image size with the predicted bounding box of the recognition result is H×W, and the coordinates of the center point of the predicted bounding box of the recognition result are (x, y, y). i ,y i Prediction box width w i Prediction box height h i and prediction category p i ;

[0022] S52. In the cross-platform computer vision library OpenCV, the scale-invariant feature transform function SIFT is called to perform scale space extreme value detection, feature point localization, feature point assignment and feature description on the disease prediction image output by the network model for building facade disease image recognition. Then, the random sampling consensus function RANSAC is called to complete the image feature detection and extraction, feature matching, perspective transformation and image fusion to obtain a complete building facade image.

[0023] S53, the coordinates (x) of the center point of the predicted bounding box of the recognition result. i ,y i The transformation is performed as follows: First, mark the origin (x0, y0) of the local image before stitching. Then, find the coordinates (X0′, Y0′) of (x0, y0) in the stitched image. Finally, calculate the coordinates (X0′, Y0′) of the center point of the prediction box in the complete building facade image. i ,Y i ):

[0024]

[0025] S53, Based on coordinates (X) i ,Y i Draw the width w i Height h i The predicted bounding boxes are then stitched onto the complete building facade image to obtain the final, complete recognition image with the predicted bounding boxes.

[0026] The beneficial effects of this invention are as follows: Compared with existing technologies, this image recognition and localization method for building facade defects can achieve rapid and accurate identification of building facade defects, improve detection efficiency, and has a more precise identification effect. It has strong practicality and engineering significance, can replace traditional manual inspection, and significantly reduce the risk of inspection. This method can display the location of defects in a complete image of the building facade, providing better visualization of the defect location and offering more data support for building facade inspection work. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating the image recognition and localization method for building facade defects according to an embodiment of the present invention.

[0028] Figure 2 This is a schematic diagram illustrating the network model for recognizing building facade defects in the embodiment.

[0029] Figure 3 This is a schematic diagram illustrating the SPPELAN spatial pyramid pooling module of the high-efficiency layer aggregation network in the embodiment;

[0030] Figure 4 The following are schematic diagrams illustrating the types of defects in the examples: (a) is a schematic diagram illustrating cracks, (b) is a schematic diagram illustrating water seepage in the exterior wall, and (c) is a schematic diagram illustrating peeling off the exterior wall.

[0031] Figure 5 This is a schematic diagram of the image recognition results in the embodiment;

[0032] Figure 6 This is a schematic diagram of image stitching in the embodiment;

[0033] Figure 7 This is a schematic diagram of the final recognized image in the embodiment. Detailed Implementation

[0034] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0035] An embodiment provides an image recognition and localization method for building facade defects, such as... Figure 1 This includes the following steps:

[0036] S1. Multiple partial images of the building facade are captured by a drone with its camera lens facing the building facade.

[0037] In step S1, the building facade defects include cracks, peeling of exterior walls, and / or water seepage. Images of the building facade are acquired by designing an automated flight path and shooting position using a drone.

[0038] S2. Filter the acquired partial images of the building facade. Select partial images of the building facade with defects as positive image samples and label the defect types on the partial images of the building facade. Select partial images of the building facade without defects as negative image samples. Obtain the dataset from the positive and negative image samples and divide the dataset into training set and test set according to a set ratio.

[0039] In step S2, the dataset is divided into a training set and a test set in an 8:2 ratio. The training set is used to train the model, and the test set is used to test the model.

[0040] S3. Construct a network model for image recognition of building facade defects, such as... Figure 2The building facade defect image recognition network model includes a skeleton network, a neck network, and a detection head network. The skeleton network performs convolution, pooling, and vector concatenation on the input local image of the building facade to output feature vectors at three scales. The feature vectors at the three scales are input into the neck network and subjected to convolution, upsampling, and vector concatenation to obtain a feature map. The feature map is then input into the detection head network to obtain a defect prediction image, which includes a prediction image with a prediction box or an image without a prediction box, i.e., an image without defects.

[0041] In step S3, the backbone network includes four sequentially configured convolutional normalization activation modules (CBS), a first high-efficiency layer aggregation network module (ELAN1), a first max pooling module (MP1), a second high-efficiency layer aggregation network module (ELAN2), a second max pooling module (MP2), a third high-efficiency layer aggregation network module (ELAN3), a third max pooling module (MP3), and a fourth high-efficiency layer aggregation network module (ELAN4). In the backbone network, the input partial image of the building facade enters the backbone network and passes through these four modules sequentially: CBS, ELAN1, MP1, ELAN2, MP2, ELAN3, MP3, and ELAN4. The second high-efficiency layer aggregation network module (ELAN2) outputs feature vector T1, the third high-efficiency layer aggregation network module (ELAN3) outputs feature vector T2, and the fourth high-efficiency layer aggregation network module (ELAN4) outputs feature vector T3, which is then fed into the neck network. Among them, feature vector T1 is the feature vector with the largest scale and the fewest eigenvalues ​​among feature vectors T1, T2 and T3, feature vector T2 is the feature vector with medium scale and medium eigenvalues, and feature vector T3 is the feature vector with the smallest scale and the most eigenvalues.

[0042] In step S3, the Neck network includes the Spatial Pyramid Pooling module SPPELAN of the efficient layer aggregation network, four Convolutional Normalized Activation Modules (CBS), two Upsampling modules UPsample, four Vector Concatenation modules Concat, two Max Pooling modules MP, and four Weighted Efficient Layer Aggregation Network modules ELAN-W.

[0043] In the Neck network, feature vector T3 is processed by the SPPELAN module of the efficient layer aggregation network to output feature vector T4. Feature vector T4 is then processed by the first convolutional normalization activation module CBS1 and the first upsampling module UPsample1 before being output to the first vector concatenation module Concat1. At the same time, feature vector T4 is input to the fourth vector concatenation module Concat4. Feature vector T2 is processed by the second convolutional normalization activation module CBS2 before being output to the first vector concatenation module Concat1. The first vector concatenation module Concat1 performs tensor concatenation and outputs feature vector T5.

[0044] Feature vector T5 is passed through the first weighted efficient layer aggregation network module ELAN-W1, the third convolutional normalization activation module CBS3, and the second upsampling module UPsample2 before being output to the second vector concatenation module Concat2. At the same time, feature vector T5 is passed through the first weighted efficient layer aggregation network module ELAN-W1 before being output to the third vector concatenation module Concat3. Feature vector T1 is passed through the fourth convolutional normalization activation module CBS4 before being output to the second vector concatenation module Concat2. The second vector concatenation module Concat2 concatenates the vectors and outputs feature vector T6.

[0045] Feature vector T6 is processed by the second weighted high-efficiency aggregation network module ELAN-W2 to output feature vector T7. Feature vector T7 can be directly input into the detection head network. At the same time, feature vector T7 is processed by the fourth maximum pooling module MP4 and output to the third vector concatenation module Concat3. The third vector concatenation module Concat3 performs tensor concatenation to obtain feature vector T8.

[0046] Feature vector T8 is processed by the third weighted efficient layer aggregation network module ELAN-W3 to output feature vector T9. Feature vector T9 can be directly input into the detection head network. At the same time, feature vector T9 is processed by the fifth maximum pooling module MP5 and output to the fourth vector concatenation module Concat4. The fourth vector concatenation module Concat4 performs tensor concatenation to obtain feature vector T10. Feature vector T10 is processed by the fourth weighted efficient layer aggregation network module ELAN-W4 to output feature vector T11 to the detection head network Head.

[0047] The detection head network consists of a first residual convolutional normalization activation module (REP+CBM1), a first classification module (Classification1), a second residual convolutional normalization activation module (REP+CBM2), a second classification module (Classification2), a third residual convolutional normalization activation module (REP+CBM3), and a third classification module (Classification3). In the detection head network, feature vector T7 is input to the first classification module (Classification1) after passing through the first residual convolutional normalization activation module (REP+CBM1); feature vector T9 is input to the second classification module (Classification2) after passing through the second residual convolutional normalization activation module (REP+CBM2); and feature vector T11 is input to the third classification module (Classification3) after passing through the third residual convolutional normalization activation module (REP+CBM3). Finally, the detection head network outputs a disease prediction image, which includes either a prediction image with a bounding box or an image without a bounding box (i.e., an image without disease).

[0048] like Figure 3 The Spatial Pyramid Pooling Module SPPELAN of the high-efficiency layer aggregation network includes the fifth convolutional normalization activation module CBS5, the sixth max pooling module MP6, the seventh max pooling module MP7, the eighth max pooling module MP8, the fifth splicing module Concat5, and the sixth convolutional normalization activation module CBS6. In the SPPELAN module of the high-efficiency layer aggregation network, feature vector T3 is input into the SPPELAN module. After passing through the fifth convolutional normalization activation module CBS5, feature vector t1 is output. Feature vector t1 is directly input into the fifth concatenation module Concat5, and simultaneously input into the sixth max pooling module MP6. The sixth max pooling module MP6 outputs feature vector t2. Feature vector t2 is directly input into the fifth concatenation module Concat5, and simultaneously input into the seventh max pooling module MP7. The seventh max pooling module MP7 outputs feature vector t3. Feature vector t3 is directly input into the fifth concatenation module Concat5, and simultaneously input into the eighth max pooling module MP8. The eighth max pooling module MP8 outputs feature vector t4. Feature vector t4 is directly input into the fifth concatenation module Concat5, and the fifth concatenation module Concat5 outputs feature vector t5. Feature vector t5 is then processed by the sixth convolutional normalization activation module CBS6 to obtain feature vector T4.

[0049] S4. Using the training set and employing the SIoU loss function that considers angle loss, distance loss, and shape loss, train the building facade defect image recognition network model constructed in step S3 to obtain the trained building facade defect image recognition network model. Input the test set into the trained building facade defect image recognition network model, and optimize the hyperparameters of the trained building facade defect image recognition network model based on the test set results to obtain the optimal building facade defect image recognition network model.

[0050] In step S4, the experimental GPU used was an RTX 3090, and the CPU used was an i7-9700. During the initial training, the hyperparameters were: Ir = 0.01, Irf = 0.1, batchsize = 16, and epoch = 300. After training the building facade defect image recognition network model, the test set was input into the trained building facade defect image recognition network model. Based on the test set results, the hyperparameters were optimized to obtain the optimal building facade defect image recognition network model.

[0051] S5. Input multiple partial images of the building facade to be identified into the optimal building facade defect image recognition network model to obtain multiple defect prediction images. Then, use the scale-invariant feature transformation algorithm SIFT and the random sampling consensus algorithm RANSAC to stitch them together to obtain the final overall recognition image of the building facade.

[0052] In step S5, the Scale Invariant Feature Transform (SIFT) algorithm and the Random Sample Consensus (RANSAC) algorithm are concatenated to obtain the final overall recognition image of the building facade. Specifically,

[0053] S51. The image size with the predicted bounding box of the recognition result is H×W, and the coordinates of the center point of the predicted bounding box of the recognition result are (x, y, y). i ,y i Prediction box width w i Prediction box height h i and prediction category p i ;

[0054] S52. In the cross-platform computer vision library OpenCV, the scale-invariant feature transform function SIFT is called to perform scale space extreme value detection, feature point localization, feature point assignment and feature description on the disease prediction image output by the network model for building facade disease image recognition. Then, the random sampling consensus function RANSAC is called to complete the image feature detection and extraction, feature matching, perspective transformation and image fusion to obtain a complete building facade image.

[0055] S53, the coordinates (x) of the center point of the predicted bounding box of the recognition result. i ,yi The transformation is performed as follows: First, mark the origin (x0, y0) of the local image before stitching. Then, find the coordinates (X0′, Y0′) of (x0, y0) in the stitched image. Finally, calculate the coordinates (X0′, Y0′) of the center point of the prediction box in the complete building facade image. i ,Y i ):

[0056]

[0057] S54, Based on coordinates (X) i ,Y i Draw the width w i Height h i The predicted bounding boxes are then stitched onto the complete building facade image to obtain the final, complete recognition image with the predicted bounding boxes.

[0058] In step S5, the optimal building facade defect image recognition network model obtains a local image with the prediction box of the recognition result. Therefore, the scale-invariant feature transformation algorithm SIFT and the random sampling consensus algorithm RANSAC are used to stitch the image together to obtain a complete building facade image.

[0059] This image recognition and localization method for building facade defects, compared with existing technologies, can achieve rapid, accurate, and automatic identification of building facade defects, improving detection efficiency and providing more precise identification results. It has strong practicality and engineering significance, can replace traditional manual inspection, and significantly reduces the risks associated with inspection. This method possesses many advantages, including high recognition accuracy and good adaptability. It can display the location of defects within a complete image of the building facade, providing excellent visualization of defect locations and offering more data support for building facade inspection work.

[0060] A specific example of the image recognition and localization method for building facade defects in this embodiment is as follows:

[0061] S1. The drone takes pictures of the building facade from an angle where the camera lens is directly facing the building facade.

[0062] S2. Filter the acquired building facade images, selecting those with existing facade defects as positive image samples, and label the defect types on the facade images, including cracks, etc. Figure 4 (a) Exterior wall peeling, such as Figure 4 (b) and / or external wall seepage, such as Figure 4 (c) uses images of building facades without defects as negative image samples, obtains a dataset from positive and negative image samples, and divides the dataset into training and testing sets according to a set ratio.

[0063] S3. Construct a network model for recognizing building facade defects. The network model for recognizing building facade defects includes a skeleton network, a neck network, and a detection head network.

[0064] S4. Using the training set and employing the SIoU loss function that considers angle loss, distance loss, and shape loss, train the building facade defect image recognition network model constructed in step S3 to obtain the trained building facade defect image recognition network model. Input the test set into the trained building facade defect image recognition network model, and optimize the hyperparameters of the trained building facade defect image recognition network model based on the test set results to obtain the optimal building facade defect image recognition network model.

[0065] S5. Input the building facade image to be identified into the optimal building facade defect image recognition network model to obtain an image with the prediction bounding box of the recognition result. Use the scale-invariant feature transformation algorithm SIFT and the random sampling consensus algorithm RANSAC to stitch the image with the prediction bounding box of the recognition result with the complete building facade image to obtain the final recognition image.

[0066] S51, such as Figure 5 The image size with the prediction bounding box of the recognition result is 800×600, the center point coordinates of the prediction box are (x1,y1)=(749,324), the width of the prediction box is w1=120, and the height of the prediction box is h1=216;

[0067] S52. In OpenCV, calling SIFT can complete the extremum detection, feature point localization, feature point assignment, and feature description of the image in scale space. Then, calling the RANSAC function can complete the feature detection and extraction, feature matching, perspective transformation, and image fusion of the image to obtain a complete building facade image, such as... Figure 6 ;

[0068] S53. Transform the coordinates (749, 324) of the center point of the prediction box: First, mark the origin (0,0) of the local image before stitching, then find the coordinates (1200,0) of (0,0) in the stitched image, and calculate the coordinates (1949, 324) of the center point of the prediction box in the complete building facade image.

[0069] S54. The predicted bounding boxes obtained from local image recognition are ultimately displayed uniformly in the stitched complete building facade image, as shown below. Figure 7 .

[0070] This method for image recognition and localization of building facade defects uses a trained deep learning model to identify these defects and then integrates their coordinates onto a complete building facade image through image stitching and coordinate transformation. This method enables rapid and accurate identification of building facade defects and allows for visual viewing of these defects across the entire building facade.

[0071] The above description of the embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make several improvements and modifications to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. A method for image recognition and localization of defects on building facades, characterized in that: Includes the following steps, S1. Multiple partial images of the building facade are captured by a drone with its camera lens facing the building facade directly. S2. Filter the acquired partial images of the building facade. Use partial images of the building facade with defects as positive image samples and label the defect types of the partial images of the building facade. Use partial images of the building facade without defects as negative image samples. Obtain the dataset from the positive image samples and negative image samples, and divide the dataset into training set and test set according to a set ratio. S3. Construct a network model for recognizing building facade defects. The network model includes a skeleton network, a neck network, and a detection head network. The skeleton network performs convolution, pooling, and vector concatenation on the input local image of the building facade to output feature vectors at three scales. The feature vectors at the three scales are input into the neck network and after convolution, upsampling, and vector concatenation, a feature map is obtained. The feature map is then input into the detection head network to obtain a defect prediction image, which includes a prediction image with a prediction box or an image without a prediction box, i.e., an image without defects. S4. Using the training set and employing the SIoU loss function that considers angle loss, distance loss, and shape loss, train the building facade defect image recognition network model constructed in step S3 to obtain the trained building facade defect image recognition network model. Input the test set into the trained building facade defect image recognition network model, and optimize the hyperparameters of the trained building facade defect image recognition network model based on the test set results to obtain the optimal building facade defect image recognition network model. S5. Input multiple partial images of the building facade to be identified into the optimal building facade defect image recognition network model to obtain multiple defect prediction images. Then, use the scale-invariant feature transformation algorithm SIFT and the random sampling consensus algorithm RANSAC to stitch them together to obtain the final overall recognition image of the building facade. In step S5, the Scale Invariant Feature Transform (SIFT) algorithm and the Random Sample Consensus (RANSAC) algorithm are concatenated to obtain the final overall recognition image of the building facade. Specifically, S51. The image size with the predicted bounding box of the recognition result is H×W, and the coordinates of the center point of the predicted bounding box of the recognition result are (x, y, y). i ,y i Prediction box width w i Prediction box height h i and prediction category p i ; S52. In the cross-platform computer vision library OpenCV, the scale-invariant feature transform function SIFT is called to perform scale space extreme value detection, feature point localization, feature point assignment and feature description on the disease prediction image output by the network model for building facade disease image recognition. Then, the random sampling consensus function RANSAC is called to complete the image feature detection and extraction, feature matching, perspective transformation and image fusion to obtain a complete building facade image. S53, the coordinates (x) of the center point of the predicted bounding box of the recognition result. i ,y i The transformation is performed as follows: First, mark the origin (x0, y0) of the local image before stitching, then find the coordinates (x0, y0) of (x0, y0) in the stitched image (x0, y0). ' ,Y0 ' The coordinates (X, X) of the center point of the prediction box in the complete building facade image are calculated. i ,Y i ): S54, Based on coordinates (X) i ,Y i Draw the width w i Height h i The predicted bounding boxes are then stitched onto the complete building facade image to obtain the final, complete recognition image with the predicted bounding boxes.

2. The image recognition and localization method for building facade defects as described in claim 1, characterized in that: In step S1, the defects on the building facade include cracks, peeling of exterior walls and / or water seepage in the exterior walls.

3. The image recognition and localization method for building facade defects as described in claim 1, characterized in that: In the skeleton network, the input partial image of the building facade enters the skeleton network and then passes through four convolutional normalization activation modules (CBS), the first high-efficiency layer aggregation network module (ELAN1), the first max pooling module (MP1), the second high-efficiency layer aggregation network module (ELAN2), the second max pooling module (MP2), the third high-efficiency layer aggregation network module (ELAN3), the third max pooling module (MP3), and the fourth high-efficiency layer aggregation network module (ELAN4). The second high-efficiency layer aggregation network module (ELAN2) outputs feature vector T1, the third high-efficiency layer aggregation network module (ELAN3) outputs feature vector T2, and the fourth high-efficiency layer aggregation network module (ELAN4) outputs feature vector T3, which are then fed to the neck network.

4. The image recognition and localization method for building facade defects as described in claim 3, characterized in that: In the neck network, feature vector T3 is processed by the spatial pyramid pooling module SPPELAN of the efficient layer aggregation network to output feature vector T4. Feature vector T4 is processed by the first convolutional normalization activation module CBS1 and the first upsampling module UPsample1 and then output to the first vector concatenation module Concat1. At the same time, feature vector T4 is input to the fourth vector concatenation module Concat4. The feature vector T2 is passed through the second convolution normalization activation module CBS2 and then output to the first vector concatenation module Concat1. The first vector concatenation module Concat1 performs tensor concatenation and outputs the feature vector T5. Feature vector T5 is passed through the first weighted efficient layer aggregation network module ELAN-W1, the third convolutional normalization activation module CBS3, and the second upsampling module UPsample2 before being output to the second vector concatenation module Concat2. At the same time, feature vector T5 is passed through the first weighted efficient layer aggregation network module ELAN-W1 before being output to the third vector concatenation module Concat3. Feature vector T1 is passed through the fourth convolutional normalization activation module CBS4 before being output to the second vector concatenation module Concat2. The second vector concatenation module Concat2 concatenates the vectors and outputs feature vector T6. Feature vector T6 is processed by the second weighted high-efficiency aggregation network module ELAN-W2 to output feature vector T7. Feature vector T7 is directly input into the detection head network. At the same time, feature vector T7 is processed by the fourth maximum pooling module MP4 and output to the third vector concatenation module Concat3. The third vector concatenation module Concat3 performs tensor concatenation to obtain feature vector T8. Feature vector T8 is processed by the third weighted efficient layer aggregation network module ELAN-W3 to output feature vector T9. Feature vector T9 is directly input into the detection head network. At the same time, feature vector T9 is processed by the fifth maximum pooling module MP5 and output to the fourth vector concatenation module Concat4. The fourth vector concatenation module Concat4 performs tensor concatenation to obtain feature vector T10. Feature vector T10 is processed by the fourth weighted efficient layer aggregation network module ELAN-W4 to output feature vector T11 to the detection head network.

5. The image recognition and localization method for building facade defects as described in claim 4, characterized in that: In the SPPELAN module of the high-efficiency layer aggregation network, feature vector T3 is input into the SPPELAN module. After passing through the fifth convolutional normalization activation module CBS5, feature vector t1 is output. Feature vector t1 is directly input into the fifth concatenation module Concat5, and simultaneously input into the sixth max pooling module MP6. The sixth max pooling module MP6 outputs feature vector t2. Feature vector t2 is directly input into the fifth concatenation module Concat5, and simultaneously input into the seventh max pooling module MP7. The seventh max pooling module MP7 outputs feature vector t3. Feature vector t3 is directly input into the fifth concatenation module Concat5, and simultaneously input into the eighth max pooling module MP8. The eighth max pooling module MP8 outputs feature vector t4. Feature vector t4 is directly input into the fifth concatenation module Concat5, and the fifth concatenation module Concat5 outputs feature vector t5. Feature vector t5 is then processed by the sixth convolutional normalization activation module CBS6 to obtain feature vector T4.

6. The image recognition and localization method for building facade defects as described in claim 4, characterized in that: In the detection head network, feature vector T7 is input to the first classification module Classification1 after passing through the first residual convolution normalization activation module REP+CBM1; feature vector T9 is input to the second classification module Classification2 after passing through the second residual convolution normalization activation module REP+CBM2; and feature vector T11 is input to the third classification module Classification3 after passing through the third residual convolution normalization activation module REP+CBM3. Finally, the detection head network outputs disease prediction images, including prediction images with prediction boxes or images without prediction boxes, i.e., images without diseases.

Images