A method for identifying wall defects based on thermal imaging and panoramic photography technology

Abstract

The present invention discloses a method for identifying wall defects based on thermal imaging and panoramic photography technology, including: carrying imaging equipment used for detection by drone; obtaining infrared thermal imaging images, and detecting facing bricks and wall defects in real time through images; taking multiple visible light images of the entire facade to be detected, and compiling a panoramic picture; using an alignment algorithm to align the visible light image with the infrared thermal imaging image, and integrating the information on the infrared thermal image obtained into a visible light long picture through neural network model analysis; calibrating the defect information on the long picture. The present invention integrates a high-definition visible light camera on a drone to take multiple photos of the facade and synthesize a panoramic long picture. Finally, the defect information predicted by the neural network model based on the infrared thermal image is reflected on the panoramic photo to achieve an intuitive display, which is convenient for construction personnel to quickly and accurately determine the defect situation and improve the detection efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of wall defect detection, and in particular to a method for identifying wall defects based on thermal imaging and panoramic photography technology. Background Art

[0002] In recent years, property and personal safety issues caused by housing safety defects have been reported from time to time in various places, and the problem of facade facing bricks falling off has attracted widespread attention from the society. In order to improve the engineering quality of buildings, engineers have found that the method of using infrared thermal imaging sensing technology to inspect each facade is efficient and easy. In the past, inspections required workers to reach the heights of buildings through construction machinery for inspection. The use of IR technology can avoid the use of high-altitude working facilities. Inspection personnel can use IR equipment to conduct inspections, and use its non-contact characteristics to conduct rapid and large-scale censuses, which has improved the level of building safety inspections and ensured the safety of the people.

[0003] The research and application history of domestic IR technology in building structure detection is short, and the research on infrared detection technology for wall defects suitable for my country's building needs is still blank. Worldwide, there is also a large gap in the combination of IR technology for building facade detection and artificial intelligence. The process of informatization is relatively slow. At present, technical personnel can only use IR equipment to analyze and determine defects. Although there have been improvements, there are still shortcomings such as high requirements for working conditions and professional level of personnel. It can be seen that the existing means will inevitably have misjudgments and lead to low detection efficiency. Therefore, the application of IR technology combined with visible light photography and artificial intelligence for detection has broad prospects. This method of integrating multiple modalities can greatly improve the level of detection.

[0004] Infrared thermal imaging technology

[0005] In nature, any object has the ability to continuously emit heat radiation to the surroundings. This property of objects can be used in the quality inspection of building finishes. By detecting the surface temperature of an object through IR technology, the surface and even internal characteristics of the object can be determined, especially the temperature can be accurately determined. In fact, infrared radiation is a kind of radio wave, usually called infrared, with a wavelength range of 0.75μm to 1μm. Infrared thermal imaging detection is to observe the surface temperature of an object to determine whether there are major structural defects in the workpiece or building.

[0006] Application of infrared thermal imaging technology in engineering inspection

[0007] When inspecting the laying condition of building facing tiles in engineering projects, due to the different thermal conductivity of the tile material and the air, when using IR technology to image the facades of the building, the air and leaked water in the hollow part of the tile present a different temperature from the surrounding tiles, resulting in an area with a significantly different color in the thermal image.

[0008] Ceramic tile hollowing is generally caused by insufficient mortar during tile laying, which results in a layer of air between the tile and the substrate. After heating or cooling, the temperature of the air layer changes differently from that of the surrounding cement, resulting in a temperature difference between the two, which is captured by the film. Based on this principle, IR equipment can be used to dynamically scan the finish to detect defects.

[0009] Visible light photography technology

[0010] The traditional manual infrared thermal imaging detection method also has high requirements for construction and inspection personnel. As a result, the detection technology is not universal in terms of personnel, and cannot be used for rapid and large-scale inspections, which is not convenient for detection and construction.

[0011] To solve the above problems, the present invention integrates a visible light camera on the detection end device to take pictures and obtain clear visible light photos. The wavelength of visible light is between 0.38μm and 0.75μm. The visible light camera uses CMOS to obtain the wavelength of visible light reflected by the object being photographed, and decodes the information obtained from each pixel through the chip, and finally outputs a visible light picture.

[0012] Application of visible light photography in engineering

[0013] During the ascent of the drone, the integrated visible light camera is used to take multiple photos of the facade being inspected, and a set of visible light photos with the facade as the main body is obtained. The perspective and distortion effects are eliminated through algorithms, and the parts with the same features of multiple photos are overlapped through feature point recognition algorithms, and finally integrated into a long picture. The defect information obtained by analysis is then calibrated on the processed long picture to obtain a long visible light picture that can intuitively show the location and range of the defect. At the same time, through the visible light picture, construction personnel can also directly determine defects such as external cracks. Summary of the invention

[0014] In order to solve the above technical problems, the purpose of the present invention is to provide a method for identifying wall defects based on thermal imaging and panoramic photography technology. The method is to use an artificial intelligence model to analyze the infrared thermal images of common construction defects of building finishes, and accordingly, to upgrade and develop the existing IR camera for the second time, so that it has the ability to detect various types of defects on the building facade. The present invention uses the YOLO algorithm to analyze the infrared thermal images obtained from the port to achieve better results. In order to ensure the stability of the equipment's analysis and detection under different temperature conditions, two modes, "Hot" and "Cold", are trained for selection. Users can select the mode they need in the guidance interface, and the IR camera and the visible light camera will be integrated on the drone at the same time to make it more adaptable.

[0015] The present invention uses an integrated visible light camera to take multiple photos of the facade being inspected during the ascent of the drone, and obtains a group of visible light photos with the facade as the main body, and eliminates the perspective and distortion effects through an algorithm, and overlaps the parts of multiple photos with the same features through a feature point recognition algorithm, and finally integrates them into a long picture, and then calibrates the defect information obtained by analysis on the processed long picture to obtain a long visible light picture that can intuitively display the location and range of the defect. At the same time, through the visible light picture, construction personnel can also directly determine defects such as external cracks. In order to achieve intuitive display and improve detection efficiency.

[0016] The purpose of the present invention is achieved through the following technical solutions:

[0017] A method for identifying wall defects based on thermal imaging and panoramic photography technology comprises the following steps:

[0018] Step A: The imaging equipment used for detection is carried by a drone;

[0019] Step B obtains infrared thermal imaging images and detects defects of facing bricks and walls in real time through the images;

[0020] Step C: taking multiple visible light images of the entire facade to be inspected, and compiling a panoramic picture;

[0021] Step D uses an alignment algorithm to align the visible light image with the infrared thermal imaging image, and integrates the information on the infrared thermal image into the visible light long image through neural network model analysis;

[0022] Step E marks the defect information on the long image.

[0023] Compared with the prior art, one or more embodiments of the present invention may have the following advantages:

[0024] The neural network model is used to analyze and integrate the information obtained from infrared thermal imaging technology and visible light images, and the wall defects are analyzed and calibrated in real time. At the same time, the integrated multi-modal imaging equipment is used to operate the drone, and the facade defects are analyzed and presented in real time and intuitively at high altitudes, which improves the efficiency and accuracy of facade safety inspection, avoids the problems of traditional inspection methods and semi-automatic inspection methods that manually analyze and calibrate infrared thermal images, and reduces the risk of misjudgment and missed judgment.

[0025] At the same time, the use of drone aerial photography technology and infrared thermal imaging technology has the characteristics of non-contact, and the use of drones equipped with imaging equipment for high-altitude analysis can be more flexible and more accurate. It also enables it to have the ability to conduct large-scale rapid inspections and reach locations that are difficult for workers to reach for inspections, thus avoiding the occurrence of blind spots in inspections to a large extent. At the same time, a long picture of the wall synthesized from visible light images is obtained. By real-time calibration of information such as defect location and type in the long picture, ordinary construction workers can understand the defects more intuitively and respond, avoiding the additional costs caused by repeated repairs. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flow chart of the method for identifying wall defects based on thermal imaging and panoramic photography technology;

[0027] Figure 2 It is a workflow diagram for carrying out imaging equipment;

[0028] Figure 3 is a schematic diagram of the integrated image after alignment;

[0029] Figure 4 It is the infrared thermal images of various defects taken from different distances in Cold mode;

[0030] Figure 5 It is the infrared thermal images of various defects taken from different distances in Hot mode;

[0031] Figure 6a and 6b Use LabelImg and LabelMe to label the pictures respectively;

[0032] Figure 7 is the training image after image enhancement;

[0033] Figure 8 It is the detection image output when training with the semantic segmentation algorithm;

[0034] Fig. 9 This is the network structure diagram of the YOLO algorithm;

[0035] Fig.10This is the network structure diagram of SPPF;

[0036] Fig.11 This is the network structure diagram of C2PSA;

[0037] Fig.12 This is the network structure diagram of the C3K2 module;

[0038] Fig.13 It is a graph of the loss function of different parameters during prediction. DETAILED DESCRIPTION

[0039] The present invention collects various thermal imaging images with wall defects, calibrates data, trains models, iterates and optimizes, and investigates panoramic photography technology, realizes the shooting of panoramic photos under the condition of ordinary visible light cameras, and manufactures drones equipped with thermal imaging cameras and visible light cameras. The camera data is wirelessly transmitted to the ground workstation through networking. The thermal imaging data returned from the drone is processed by the pre-trained wall defect recognition model at the ground workstation to automatically identify wall defects, and the local photos taken by the visible light camera are used to synthesize panoramic photos. After completing the above work, the information of the two is fused by using an alignment algorithm to achieve the effect of intuitively marking wall defects in panoramic photos for engineers to use.

[0040] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below in conjunction with embodiments and drawings.

[0041] like Figure 1 As shown, the method flow of identifying wall defects based on thermal imaging and panoramic photography technology includes the following steps:

[0042] Step 10: The imaging device used for detection is carried by a drone;

[0043] The imaging equipment used for detection is carried out by drones. The two carried imaging devices are connected to the Raspberry Pi development board mounted on the drone. After integration, they are wirelessly connected to the ground workstation, and data is obtained through Flask technology, which is controlled by the ground workstation. The imaging equipment is connected to the Raspberry Pi development board on the client, and visible light images and infrared thermal images are wirelessly transmitted through the Raspberry Pi development board. The server ground workstation POSTs the information sent by the development board. The host display device of the output module can obtain the image information captured by the equipment in real time and run the model for analysis.

[0044] Step 20: Obtain infrared thermal imaging images, and detect facing brick and wall defects in real time through the images;

[0045] The neural network model trained based on the YOLO algorithm is used to analyze the image in real time and record the monitored surface defects. The type and range of the local defects are then calibrated and saved through the program.

[0046] Step 30: taking multiple visible light images of the entire facade under inspection and compiling a panoramic picture;

[0047] The visible light camera is used to take multiple photos of the entire facade to be inspected, and a panoramic picture is compiled. Since it involves deforming the visible light image, the influence of distance perspective and lens distortion on the image is eliminated, the feature points of each picture are matched, and the pictures are moved and spliced ​​according to the matching information. Finally, the spliced ​​long picture is cropped and finally a panoramic long photo with the facade as the main content is synthesized. Based on the obtained visible light picture, combined with the defect information analyzed by the model on the infrared picture, the defects are calibrated in the long picture.

[0048] Step 40 uses an alignment algorithm to align the visible light image with the infrared thermal imaging image, and integrates the obtained information on the infrared thermal image into the visible light long image through neural network model analysis;

[0049] Since the infrared camera and the visible light camera are installed at different positions, the shooting directions of the two cameras are different, which results in a difference in the field of view of the photos taken. In order to avoid the impact on the calibration of the defect position on the long photo, the difference external parameters of the two cameras are calibrated on the basis of ensuring that the two cameras always remain on the same axis. During the alignment process, as long as the defect position information obtained from the acquired infrared thermal image is translated downward by a certain height, the information in the two pictures can be overlapped, thereby achieving the purpose of integrating the information on the infrared thermal image obtained by the model analysis into the visible light long picture.

[0050] h ir =h vl +Δh

[0051] Therefore, the images taken by the visible light camera and the thermal imaging camera can be aligned by translating the images and interpolating the visible light images (e.g. Figure 3 The image and information captured by the infrared camera are overlapped with the image captured by the visible light camera to prepare for the next function.

[0052] Step 50 marks the defect information on the long image.

[0053] After obtaining a visible light long image with the facade as the main body and the wall defect information obtained by analyzing the real-time infrared image, they are aligned. In order to display the information more intuitively, the defect position and range obtained by analysis are directly framed on the visible light long image, and finally a defect-visible light long image with accurate and intuitive information and no obvious errors is displayed in the result output window, clearly indicating the relevant information of the defect. Not only does it ensure the accuracy of identification, but it also allows on-site personnel to obtain specific information about the defects more effectively, making it easier for construction personnel to intuitively understand the situation and respond to the defects in a timely manner.

[0054] The above embodiment uses an infrared thermal imaging camera and a high-definition visible light camera, and connects the two devices to the Raspberry Pi development board, and uses a terminal computer as a ground workstation for control. The parameters of the infrared thermal imaging camera and the high-definition visible light camera, the ground workstation parameters, and some parameters of the Raspberry Pi are shown in Table 1, Table 2, Table 3, and Table 4 respectively:

[0055] Table 1

[0056]

[0057]

[0058] Table 2

[0059] Camera resolution 2560×1440

[0060] Table 3

[0061] System version Windows 10professional for workstation CPU Intel i5-1035G4 GPU AMD Radeon 630 Onboard RAM 16 GB

[0062] Table 4

[0063] SOC Broadcom BCM2711 CPU 64-bit 1.5GHz GPU Broadcom VideoCore Vl@500MHz USB interface USB2.0*2 / USB3.0*2 powered by 5V 3A Wifi network 802.11AC wireless 2.4GHz / 5GHz dual-band Wi-Fi

[0064] Sample production,In order to meet the requirements of neural network model training for training data, multiple facing brick paving models were artificially produced, which simulated common facing brick paving defects in the construction process, such as qualified products, small scattered hollows, large hollows, single-channel water seepage, and double-channel water seepage.

[0065] Data collection, in order to ensure the effectiveness of model detection, a large number of infrared thermal images of different wall defects were taken to obtain pictures as materials for training and testing of the neural network model. The infrared camera used was used to shoot them from a distance of 40cm and 80cm respectively. In order to restore the actual working conditions, the samples were controlled at 10℃, 25℃ (low temperature group) and 40℃, 55℃ (high temperature group) through temperature control equipment, simulating the low and high temperatures throughout the year and in a single day for shooting, and finally obtained about 1,000 sample materials.

[0066] Data preprocessing

[0067] Simulating the conditions of defect detection under different temperature conditions, in a cold environment, the temperature of the defect location is usually lower than the temperature of the surrounding environment, while in a hot environment, the opposite is true. To solve this problem, the content in the dataset is divided into two groups: cold environment (Cold) and hot environment (Hot), and two models are trained for switching between different detection scenarios (such as Figure 4 and Figure 5 shown).

[0068] After completing a large amount of data collection, use LabelImg and LabelMe software to annotate the defects in all images (such as Figure 6a and 6b ). During the research, it was found that when LabelImg was used for labeling (labeled data is more suitable for target detection algorithms), if the image contains rotation angles, the label box may contain a large number of non-defective areas. In order to improve the accuracy of labeling, LabelMe was used to label the data (labeled data is more suitable for semantic segmentation algorithms), and the algorithm direction was changed to semantic segmentation. At the same time, the models in two directions were trained, and the prediction effects of the two models were compared after testing, and the better model was selected for use to achieve better detection results.

[0069] In order to improve the generalization ability of the model, the images in the dataset are divided into a training set and a validation set in a ratio of 8:2. The data is enhanced by image processing, including rotation, scaling, cropping, and splicing, to enhance the difference between images and improve the model detection ability (such as Figure 7 shown).

[0070] By comparison, the semantic segmentation model adjusted using the YOLO algorithm can more accurately determine and display the location and range of defects (such as Figure 8 As shown), therefore, this embodiment uses the semantic segmentation algorithm as the final model algorithm for stronger and more targeted training and use.

[0071] Model Training

[0072] 1) YOLO algorithm

[0073] like Figure 7The figure shows the network structure of the YOLO algorithm. In the YOLO algorithm, Backbone is the backbone network, which is mainly responsible for extracting feature points from the channels of the input image and using multiple convolutions and pooling to extract and learn the features of the image. Neck, as the neck network, further processes the information in Backbone to enhance its ability to detect defective parts with multi-scale information fusion. Head is the detection head of the total network, which is responsible for the final output operation. Its core task is to draw the prediction box, class label and calculate the confidence of each prediction based on the information from the first two steps.

[0074] The YOLO11 model is used, which mainly analyzes the incoming infrared thermal image through a multi-layer algorithm to finally obtain the result.

[0075] The first step is to input and train the YOLO model. During input, the YOLO model reads the image and various pre-calibrated information. This includes the confidence level P of the target. c , and also includes the horizontal coordinate b of the center point of the target box on the image x and the vertical coordinate b y , indicating the center point of the target, YOLO will determine the coordinate axis with the upper left corner of the image as the origin and the lower right corner coordinate as (1,1), and get the coordinates of the center point, and at the same time get the height b of the target box h and width b w To get the range of the target frame, due to the characteristics of the coordinate system, in the whole picture, b x ,b y ,b h and b w It is always less than 1, and finally a target type c of the target is obtained. Therefore, the YOLO model obtains a tensor y of size 1*1*6 of the image information:

[0076] y=[P c b x b y b w b h c]

[0077] Afterwards, the YOLO algorithm will start the data preprocessing process to analyze the imported images and their marking information. YOLO will divide the incoming infrared thermal image into 361 small grids, and perform convolution and pooling on each small grid to analyze the information of each channel. First, the YOLO algorithm will use the image mapping operation to evenly map the information in the external label accompanying the image to each small grid. Then the algorithm will check the small grids one by one through the classification and positioning algorithms. When encountering a small grid with a target frame, YOLO will re-map and mark the center point position and frame range of the target frame in the small grid. If there is a target frame range in the divided image, but there is no marked center point, then YOLO will selectively ignore these contents, that is, set the confidence of the position to 0, and finally obtain a 19*19*6 three-dimensional matrix.

[0078] After that, YOLO starts the training process. It mainly obtains the best set of parameters by analyzing the labeled data in the existing training data set, and saves the parameters in the weight file best.pt. YOLO uses an innovative convolution algorithm. It no longer uses the window sliding method based on the set step size, but directly divides the infrared thermal image into small grids for analysis, that is, the divided small grids are directly used as convolution windows for use, which reduces the convolution subspace and improves efficiency. It is gradually compressed to expand the amount of information that can be obtained from the same number of pixels. At the same time, the maximum pooling operation is introduced in the convolution layer, and the divided small grids are used as the pooling window of the operation. The maximum value of the pixel information in a certain pixel range is extracted to represent a certain range of information. This can further improve the degree of information complexity in the image.

[0079] During the verification process, the model will call the various parameters and weights in the best.pt file, and use them as the parameters of the various operations in the convolution layer in the test to operate on the tested image, and then obtain a 19*19*6 tensor, and then perform parameter calculations in the tensor to finally obtain the target parameter y in each small grid. Since the range of the target may have exceeded the range of the small grid during the detection process, b w and b h The value of may be greater than 1. Finally, the confidence level P c By multiplying it by the probability that the target is of this category, we can get the probability that the object in the image is of this defect type. When the obtained probability is greater than or equal to the set IOU threshold of 0.7, the algorithm will define a prediction box for the target and record its data, and then hand over the prediction box and data to the next algorithm for analysis.

[0080] YOLO11 uses the fast-spatial pyramid pooling layer SPPF to replace the original pooling scheme to complete the pooling operation of the image pixels ( Fig.10The network structure diagram of SPPF) is shown in Figure 2. Through the multi-scale feature fusion function, the feature points of infrared thermal images can be better extracted and matched, thereby improving the efficiency of image recognition:

[0081] First, the SPPF layer receives the feature map from the previous layer as input, and then performs a pooling operation on the image. Unlike the SPP algorithm, which uses pooling kernels of different sizes but generally larger, the SPPF uses multiple small pooling kernels to replace a single large kernel for parallel operations in different pooling layers, reducing the amount of calculation for each layer. It also retains the function of multi-scale feature fusion and linking, and improves the ability to recognize image feature points. After the fast pooling operation, the SPPF layer splices and integrates the outputs of different small pooling layers to form a feature vector of fixed length as the output of the entire layer.

[0082] In YOLO11, feature extraction and processing are performed through the C2PSA module (such as Fig.11 The network structure of C2PSA) extracts more effective information from the low-scale semantics of infrared thermal images by combining the information of the incoming pictures, and combines it with the multi-head attention mechanism to enhance the feature extraction capability. You can also selectively add a residual structure to achieve a more accurate perception of the image semantics and optimize the training effect. Among them, C2PSA introduces the PSA mechanism, which is an optimization mechanism based on the SE attention mechanism. Its core is to adaptively assign weights to different information by the program, reduce the components of manual intervention to improve the actual performance of the network model. PSA also optimizes the effectiveness of multi-scale processing. Moreover, in YOLO11, the C2PSA module is called to use multiple PSA modules to continuously enhance the feature extraction capability of the model. By introducing the attention mechanism, the C2PSA module can further optimize and enrich the feature map of the previous layer, improve the accuracy and richness of the image information, and improve the performance of the model.

[0083] The YOLO model also includes a convolution module C3K2 module (such as Fig.12The network structure of the C3K2 module is used to partially replace the original convolution operation to deeply extract and process the information of the image. C3K2 introduces variable convolution kernel technology. When the c3k parameter is true, the model uses the C3 module. Use multi-scale convolution kernels to set the size of the convolution kernel used in this step of the convolution operation. Improving the flexibility of the convolution operation can expand the field of vision of computer vision perception, enable the model to have stronger associative ability, and improve the model's ability to detect complex scenes. Among them, C3K2 divides the input features into two parts, one part is directly transmitted through ordinary convolution operations, and the other part is extracted through multiple C3K convolutions or Bottleneck structures for deep features. Finally, the features of the two parts are spliced ​​and integrated, and the convolution operation is fused, which keeps the lightweight and enables the model to effectively extract deep information.

[0084] The above implementation also optimizes the learning and training process through the Adam optimizer, which is an optimization algorithm for the stochastic gradient descent process that can iteratively update the neural network weights based on the training data. During the training process, Adam will design adaptive learning rates for different parameters.

[0085] The YOLO series models provided in this embodiment use a variety of loss functions to determine and correct the target detection results. According to the information in the tensor y, it designs functions such as positioning loss, confidence loss and classification loss, calculates the loss of the data respectively, and calculates the difference between the predicted value and the true value to correct various weights and parameters of the model. At the same time, in order to achieve accurate prediction, the model is converted into a semantic segmentation model. In order to achieve this goal, a special segmentation loss function is added to the model for loss calculation of image segmentation tasks.

[0086] The predictions are compared with the true annotations at the pixel level. By assigning each pixel to a specific class, the information on each pixel is compared to achieve highly accurate comparison and prediction.

[0087] Among them, the most important is the IOU loss function. In the semantic segmentation task, the function determines whether the prediction is accurate by matching the predicted box with the true labeled box. The threshold is set to 0.7. Only when the intersection of the predicted box and the true labeled box is greater than or equal to 0.7 is it considered a valid match, and the prediction is also considered a valid prediction. The weight parameters are adjusted (such as Fig.13 Shown are the loss functions for different parameters during prediction).

[0088] During the implementation of the above embodiment, two sets of YOLO models were developed. Among them, the target detection algorithm can detect multiple targets in the image and provide a bounding rectangle and category information, but it cannot provide accurate contour information of the target, and it is difficult to detect overlapping targets or small targets. The semantic segmentation algorithm can provide a category label for each pixel, can classify the image at the pixel level, and can distinguish different instances in the same category. It also provides information about the bounding rectangle prediction box of the target to ensure the accuracy of the detection. Moreover, using labelMe for labeling can achieve high-precision polygonal labeling. As a result, the model has the ability to fuse low-scale semantic information with high-scale image information at multiple scales, and recognize the image at the pixel level to achieve extremely high range detection accuracy.

[0089] In the above embodiment, the project team collected a large amount of data under the YOLO algorithm framework and trained two models for identifying wall defects in high temperature and low temperature environments, with high accuracy and rich feedback of wall defect information. The wall defect information was innovatively presented to construction workers in the form of panoramic photos, with intuitive effects and rich details; thermal imaging cameras and visible light cameras were installed on the drone, and a ground workstation was set up. The work is highly integrated, and no high-risk operations are required during the detection phase, protecting the personal safety of practitioners.

[0090] Although the embodiments disclosed in the present invention are as above, the above contents are only embodiments adopted for facilitating the understanding of the present invention and are not intended to limit the present invention. Any technician in the technical field to which the present invention belongs can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present invention, but the patent protection scope of the present invention shall still be subject to the scope defined in the attached claims.

Claims

1. A method for identifying wall defects based on thermal imaging and panoramic photography technology, characterized in that: The following steps are involved: Step A: The imaging equipment used for detection is carried by a drone; Step B obtains infrared thermal imaging images and detects defects of facing bricks and walls in real time through the images; Step C: taking multiple visible light images of the entire facade to be inspected, and compiling a panoramic picture; Step D uses an alignment algorithm to align the visible light image with the infrared thermal imaging image, and integrates the information on the infrared thermal image into the visible light long image through neural network model analysis; Step E marks the defect information on the long image.

2. The method for identifying wall defects based on thermal imaging and panoramic photography technology according to claim 1, characterized in that: In step A, the imaging device transmits the visible light image and the infrared thermal image to the ground service end through the client, and the display device of the output module obtains the image information captured by the imaging device in real time and analyzes it through the trained neural network model.

3. The method for identifying wall defects based on thermal imaging and panoramic photography technology according to claim 1, characterized in that: The step B includes: training a neural network model through the acquired infrared thermal imaging images, and using the neural network model trained based on the YOLO algorithm to perform real-time analysis on the images to obtain the type and location information of the surface defects.

4. The method for identifying wall defects based on thermal imaging and panoramic photography technology according to claim 3 is characterized in that: The temperature control device is used to control the thermal imaging images into low temperature group and high temperature group for data collection, and the content of the data collection is divided into two groups: cold environment data and hot environment data. The neural network model is trained with the cold environment data and the hot environment data respectively to switch between different detection scenarios; After data collection, the defect data in the thermal imaging image is annotated using LabelImg and LabelMe. The data set contains images with rotation angles. When LabelImg is used for annotation, the annotation box contains non-defective areas. In this way, the data is annotated using LabelMe, and the algorithm is converted into a semantic segmentation algorithm. At the same time, the semantic segmentation model is trained.

5. The method for identifying wall defects based on thermal imaging and panoramic photography technology according to claim 4, characterized in that: The YOLO algorithm is used to adjust the semantic segmentation model to determine and display the location and scope of defects, including: In the YOLO algorithm, Backbone is used as the backbone network to extract feature points from the channels of the input image, and uses multiple convolutions and pooling to extract and learn the features of the image; Neck is used as the neck network to further process the information in Backbone and enhance its multi-scale information fusion detection capability for defective parts; Head is responsible for the final output operation, drawing the prediction box, calculating the category label and the confidence of each prediction.

6. The method for identifying wall defects based on thermal imaging and panoramic photography technology according to claim 5, characterized in that: The YOLO11 model is used to analyze the incoming infrared thermal image through a multi-layer algorithm: Input and training of the YOLO model. During input, the YOLO model reads the image and various pre-calibrated information, including the target confidence P c , the horizontal coordinate b of the center point of the target frame on the image x and the vertical coordinate b y ; YOLO determines the coordinate axis by taking the upper left corner of the image as the origin and the lower right corner as (1,1), obtains the coordinate of the center point, and obtains the height b of the target frame. h and width b w To get the range of the target frame, due to the characteristics of the coordinate system, in the whole picture, b x ,b y ,b h and b w It is always less than 1, and finally the probability c of the target being of this type is obtained; thus, the YOLO model obtains a tensor y of size 1*1*6 for the image information: y=[P c b x b y b w b h c] Subsequently, the YOLO algorithm analyzes the imported images and their marking information through data preprocessing, divides the incoming infrared thermal image into 361 small grids vertically and horizontally, performs convolution and pooling on each small grid, and analyzes the information of each channel.

7. The method for identifying wall defects based on thermal imaging and panoramic photography technology according to claim 6, characterized in that: The YOLO algorithm maps the information in the external label that accompanies the image to each small grid evenly through image mapping, and checks the small grids one by one through classification and positioning algorithms. When encountering a small grid with a target box, YOLO re-maps and marks the center point position and frame range information of the target box in the small grid. If there is a target box range in the divided image but no marked center point, YOLO will selectively ignore it, that is, set the confidence of the position to 0, and finally obtain a 19*19*6 three-dimensional matrix; Analyze the labeled data in the training dataset and get the best set of parameters, saving the parameters in the best.pt file; The parameters and weights in the best.pt file are called and used as the parameters of the various operations in the convolution layer in the test to operate on the test image, and a 19*19*6 tensor is obtained again. The parameters are calculated in the tensor to obtain the target parameter y in a small grid. Since the range of the target relative to each small grid exceeds the range of the small grid during the detection process, b w and b h The value of is greater than 1, and the confidence level P c By multiplying it by the probability c that the target is of this type, we can get the probability that the object in the image is of this defect type. When the probability is greater than or equal to the set IOU threshold of 0.7, the YOLO algorithm will define a prediction box for the target and record its data, and then calculate and analyze the prediction box and data, that is, integrate and output the information of the two images.

8. The method for identifying wall defects based on thermal imaging and panoramic photography technology according to claim 6, characterized in that: The YOLO11 includes an SPPF module, a C2PSA module, and a C3K2 module; The SPPF performs semantic segmentation on the image, receives the feature map from the previous layer as input, and then performs a pooling operation on the image; after pooling, the SPPF layer splices and integrates the outputs of different small pooling layers to form a feature vector of fixed length as the output of the whole layer; The C2PSA module extracts information from the semantics of the infrared thermal image by inputting information of the image; The C3K2 module is used to divide the input features into two parts, one of which is directly transmitted through a convolution operation, and the other part is deep feature extracted through multiple C3K convolutions or Bottleneck structures. Finally, the features of the two parts are spliced ​​and integrated to complete the extraction of deep information.

9. The method for identifying wall defects based on thermal imaging and panoramic photography technology according to claim 1, characterized in that: The step C includes preprocessing the images, matching the feature points of the preprocessed images, moving and splicing the images according to the matching information, cropping the spliced ​​long images, and finally synthesizing a panoramic long photo with the facade as the main content.

10. The method for identifying wall defects based on thermal imaging and panoramic photography technology according to claim 2, characterized in that: The step D includes translating the defect location information obtained from the acquired infrared thermal image downward and interpolating the visible light image to achieve the overlap of the information in the infrared thermal image and the visible light image, thereby integrating the information on the infrared thermal image obtained by the model through analysis into the visible light long image: h ir =h vl +Δhwhere h ir It represents the corrected vertical position of the defect position in the infrared image, that is, the final position of the defect information in the infrared image during the alignment process; h vl It represents the initial vertical position of the corresponding defect position in the infrared image, that is, the initial height or position of the corresponding point in the visible light image; Δh represents the difference or offset in the vertical direction between the infrared image and the visible light image, which is the vertical translation of the defect position in the infrared image relative to the corresponding position in the visible light image.

Images