Method for detecting and risk assessment of high-rise building outer wall falling anomaly based on infrared thermal image

Abstract

A kind of high-rise building outer wall falling abnormality detection and risk assessment method based on infrared thermal image, is the obvious difference between the heat dissipation characteristics of outer wall between the beam-column structure of the frame structure high-rise building constituting bearing system and the infill wall constituting the enclosure, with the help of building facade infrared thermal image, the outer wall hollow defect and falling risk of high-rise building are detected and evaluated. By analyzing the boundary invasion and regional connectivity of the boundary position between the building beam-column and infill wall area on the infrared thermal image, the feature difference between the abnormal area and normal area on the infrared thermal image is extracted, the falling risk grade of high-rise building outer wall is accurately evaluated, and the high-rise building with prominent outer wall falling risk is screened out. The present application realizes the defect screening and risk assessment of a large number of high-rise building outer wall falling by analyzing and processing the infrared thermal image of high-rise building outer wall, and provides decision support for selective further in-depth detection and maintenance.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of building safety anomaly detection of frame structure, and relates to building outer wall falling anomaly detection, in particular to a high-rise building outer wall hollow defect detection based on infrared thermal image and a falling risk assessment method caused by hollow defect, which is suitable for detecting the hollow defect damage of building surface layer and superficial layer. BACKGROUND

[0002] High-rise buildings are an important part of urban construction, which not only improves the efficiency of land use, but also brings higher population carrying capacity, and is a manifestation of economic and technological capacity. The high-rise building with unique design and unique location will often become a landmark building of a city, and even the construction of higher floor limit building is also the goal of each city to compete for. However, the rise of high-rise buildings also brings great hidden dangers to urban safety. The outer wall insulation material used in early high-rise buildings has great defects, and since the completion of the building, it has been subjected to environmental stress such as sun, rain, thermal expansion and freezing and thawing for a long time, gradually aging and damaging, and will naturally fall off when it reaches a certain degree.

[0003] Usually, the quality detection of building outer wall is to detect the adhesion of outer wall, facing layer, insulation layer and building main body. The traditional detection methods include visual inspection method, hammering method, pulling method, and impact wave detection method. Among them, the visual inspection method and the hammering method are experience evaluation methods, which are used by technical personnel to observe the outer wall structure through naked eyes and auxiliary hammering to determine whether the outer wall structure has the risk of hollowing, cracking and falling off; the pulling method is to open a square or circular groove on the outer side of the external thermal insulation system, and then to measure the stress by pulling the metal disc solidified by epoxy resin glue on the surface, so as to judge the health condition of the outer wall; the impact wave detection method is to cause stress wave at the defect position of internal insulation layer by applying a small impact on the surface of the building, and to analyze the frequency peak value and surface displacement frequency spectrum of the feedback stress wave fluctuation characteristics, so as to determine whether there is a defect in the structure and obtain the approximate size of the defect. The above detection methods explore the correlation between the health degree of building materials and vibration, sound wave and appearance, so as to determine and locate the defects, which has certain reference value, but the detection efficiency is low, the accuracy and reliability depend on the subjective experience of the operator, and some methods may cause further damage to the building outer wall. When working on high-rise buildings, the hammering method and the pulling method also need to set up scaffolding, and the technical personnel often face high safety risks.

[0004] In recent years, infrared thermal imaging technology has been gradually applied to the field of building anomaly detection. Under sunlight, the health status of building materials is closely related to temperature changes. Infrared thermal imaging technology can present the temperature field distribution of the building outer wall surface on the infrared thermal image, which is represented by different colors. Its remote, non-contact, and large-scale measurement characteristics make the use of this method for building damage detection analysis effective, and it can also achieve non-destructive detection. Currently, the identification of abnormal area location and size based on infrared thermal images is often done by observing abnormal high-temperature points in the image, commonly known as hot spots. This is because the air in the hollow area changes the heat transfer coefficient, and the heat accumulates under sunlight, causing the temperature of the hollow area to be higher than that of the surrounding normal area. Or it is done by observing abnormal low-temperature points in the image, commonly known as cold spots. This is because the water in the crack area changes the heat transfer coefficient, and the heat capacity increases, causing the temperature of the crack area to be lower than that of the surrounding normal area.

[0005] The infrared imaging method detection process is often done by hand-held equipment on-site shooting by staff, and through the staff's prior knowledge and reliance on the temperature distribution extraction function of the infrared thermal imager, the temperature abnormal position in the wall area is found spontaneously. Although it has certain effectiveness, there is still a large error, and the reliability of the detection result depends on the equipment performance and the technical level of the operator. This method of distinguishing abnormalities by temperature difference only has good detection effect for small-scale abnormal points with significant temperature difference. According to the abnormal information obtained by infrared imaging, traditional detection methods are still needed for detailed and comprehensive detection to assess the damage degree in detail. The process is complex, time-consuming and labor-intensive. In actual field environment, background noise is very serious, and windows, metal frames, and air conditioning outdoor units under sunlight can all become temperature abnormal points, seriously interfering with the recognition accuracy. In densely populated cities, there are many buildings, and high-rise buildings are hundreds of thousands. The aging speed and degree of building damage of different areas, different material structures, and different use intensities are quite different. Although the infrared thermal imaging detection method assisted by manual method has certain detection speed, it is still a drop in the bucket compared to such a large building group. The digital-driven abnormal automatic recognition method based on traditional infrared imaging anomaly detection is difficult to work effectively. It is no different from repairing the sheep after the loss when the building outer wall falls off causing personal injury or property damage, and then analyzing and repairing the exposed building.

[0006] On the other hand, the temperature anomaly of the outer wall defect is often submerged in the entire infrared thermal image of the outer wall, but for high-rise buildings with frame structures, the material difference between the beams and columns and the infill walls will present different heat dissipation characteristics under the action of sunlight. Focusing on the clear temperature uniformity distribution of the building outer wall surface, it is expected to detect the outer wall defect anomaly more accurately and efficiently.

[0007] In summary, the new smart city should be able to intelligently perceive the health condition of the city building, monitor the damage degree of the building in real time, and timely alarm the management and dispatch personnel to prevent problems. The existing building defect detection method cannot realize large-area, high-speed and high-precision detection and analysis of the city building group, and the mining of city elements and information is still not thorough enough, and a digital detection method for quickly and effectively detecting the damage degree of the outer wall of high-rise building is urgently needed. SUMMARY

[0008] The purpose of the present application is to overcome the defects of the prior art, and to provide a high-rise building outer wall falling abnormality detection and risk assessment method based on infrared thermal image. By combining infrared thermal image and visible light image, according to the distribution characteristics of the building beam column area and the infilled wall area of the frame structure, the abnormal change of the infilled wall area boundary position in the building outer wall infrared thermal image is extracted, and the building outer wall falling abnormality detection and falling risk assessment are realized.

[0009] To achieve the above purpose, the present application combines infrared thermal imager and visible light imaging device to shoot the outer wall of the city high-rise building of frame structure, carries out image segmentation based on visible light image, locates the maximum effective area of the building outer wall surface in the image, in the corresponding outer wall infrared thermal image, extracts the boundary pixel distribution characteristics between the building beam column and the infilled wall area by image processing algorithm, and then judges whether there is abnormality such as boundary intrusion and region connection in the area representing hollow defect, counts the total number of abnormal areas, assesses the falling risk grade of the outer surface of the building, generates the building outer wall diagnosis report, and thus quickly screens out the city problem building group.

[0010] Specifically, the technical solution adopted by the present application is: a high-rise building outer wall falling abnormality detection and risk assessment method based on infrared thermal image, and the steps of the method are as follows:

[0011] A. Establish a city high-rise building geographic distribution map, collect visible light images and corresponding infrared thermal images of the outer walls of each high-rise building.

[0012] The above-mentioned urban high-rise buildings refer to high-rise buildings with frame structure. First, the location distribution of urban high-rise buildings is investigated, and a geographic distribution map of high-rise buildings is established. Considering the endurance of the patrol vehicle and the operation and maintenance cost, the urban area is divided into multiple blocks according to the area, and the high-rise building concentrated area is taken as the center and the high-rise building scattered area as the block boundary. Each block is independently operated and maintained, and an operation and maintenance point is set. According to the geographic distribution map of high-rise buildings, the optimal patrol detection route is drawn based on the large-format complete wall surface on the east and west sides of the building as the main target. The patrol vehicle is equipped with FLIR M500 super high performance multi-sensor thermal imager system. The thermal imager has 14 times continuous optical thermal zoom and 30 times electronic zoom color high-definition camera, which can detect thermal signals at the nearest 8.3nm and the farthest 15.4km, and can perform near and ultra-long range target detection and identification well. Its 360° azimuth and + / - 90° pitch angle and automatically adjustable digital image detail enhancement can generate thermal images with clear texture and wide range. The street automatic patrol vehicle runs periodically according to the fixed patrol track, and takes pictures of the outer walls of high-rise buildings along the way. When reaching the preset fixed point position, the direction and elevation angle of the camera are adjusted to obtain the outer wall images of the surrounding high-rise buildings. The obtained images not only have rich color distribution information of the building outer wall, but also have outer wall surface temperature distribution information, and record the basic properties of the vehicle at this time, such as latitude and longitude coordinates, thermal imager horizontal inclination angle, vertical inclination angle, camera direction, shooting distance, target wall surface direction, etc. The shooting time should be arranged as much as possible in sunny and cloudy days with more sunshine to obtain the best temperature distribution image of the building outer wall; the shooting angle should be kept as positive as possible, and the distance should be moderate, so that the proportion of the building outer wall in the image is appropriate.

[0013] B. Split the visible light image, extract the largest effective area in the image representing the building outer wall surface, and obtain the infrared thermograph rectification image (referred to as infrared rectification image) of the infrared thermograph area corresponding to the largest effective area.

[0014] To ensure the timeliness of image acquisition and image processing, the image acquisition process is configured in the inspection vehicle terminal, and the image processing process and abnormal state assessment are performed in the cloud. After the high-rise outer wall image is collected, the obtained image is transmitted to the Faster-R-CNN neural network model trained with the existing building outer wall data set, the visible light image is segmented, the area representing the building outer wall in the image is extracted, the background noise interference such as sky and street in the image is eliminated, the processing area is reduced to the building outer wall surface, and the infrared thermal image is sorted and cropped. The Faster-R-CNN neural network segments multiple candidate regions according to prior knowledge, and extracts the largest effective region representing the building outer wall surface. The largest building wall candidate region obtained by image recognition segmentation should account for more than 40% of the entire image area, so as to be considered as an effective target object, which can avoid the condition that multiple small invalid building wall surface regions are recognized in the same image, and then obtain the effective infrared thermal image of the building wall region. Using the matlab data analysis platform, the image is subjected to basic image processing operations such as Gaussian filtering, grayscale, dilation and erosion, and image noise is filtered out, which can enhance the clarity of the object shape structure and the region contour performance in the image.

[0015] The infrared correction image is obtained by using the improved Hough transform to extract the four straight lines belonging to the building outer wall boundary in the infrared thermal image region corresponding to the largest effective region of the building outer wall surface, calculating the coordinates of the intersection points of the straight lines, obtaining the four corner points of the building outer wall surface, and then performing perspective transformation to eliminate the perspective distortion phenomenon in the image. The target building outer wall surface orthographic infrared thermal image can be obtained. Specifically, the improved Hough transform and perspective transformation are used to calculate the coordinates of the four corner points A, B, C and D of the building outer wall surface through the four edge lines intersecting with each other on the outer contour of the building outer wall surface in the infrared thermal image region. The corrected building outer wall corner point coordinates are A', B', C' and D', and the four groups of mapping point coordinates are substituted into the perspective transformation equation set to solve the parameter size in the perspective transformation matrix. This parameter is the perspective transformation parameter conforming to the image, and all pixel coordinates in the region formed by the four edge lines in the image are subjected to perspective transformation based on the calculated perspective transformation matrix, so as to obtain the infrared correction image of the building outer wall region. This process is a conventional technique in the art, and will not be described here.

[0016] C. The boundary pixel distribution characteristics of the building beam column and infill wall region in the infrared correction image are extracted by the image processing algorithm. Specifically, the image processing algorithm includes the following steps, and the implementation process of each step is a conventional technique in the art:

[0017] (1) The R, G, B component piecewise linear adjustment is used to enhance the color of the infrared corrected image, mainly to enhance the image contrast, strengthen the outline and boundary of the building beam column structure and the infilled wall, and make the color of each temperature uniform area more bright;

[0018] (2) The sobel operator is used to filter the color-enhanced image to calculate the color component change gradient image;

[0019] (3) The extended minimum value transformation method is used to set the minimum value threshold to segment and remove the low gradient area, extract the beam column area and infilled wall area boundary outline;

[0020] (4) The labeled watershed algorithm is used to extract the beam column and infilled wall area single-pixel width boundary pixel coordinate matrix.

[0021] The infrared corrected image of the building outer wall area obtained in step B has a clear boundary between the building beam column and the infilled wall area, but the sharpness between the temperature uniform areas is insufficient, the transition area is wide and not bright, which is not conducive to the subsequent boundary coordinate extraction, so the R, G, B component piecewise linear adjustment is used for image enhancement. In the infrared corrected image, the R, G, B three component values of the temperature uniform area representing the position of the building infilled wall are all high, wherein R∈[240, 255], G∈[160, 250], B∈[120, 180], the area color is mainly white, while the R component value of the reddish area at the position of the building beam column structure is high, consistent with the infilled wall, the green and blue components are low, R∈[240, 255], G∈[40, 120], B∈[50, 100], the area color is mainly red. And in the transition process from the white area of the infilled wall to the red area of the beam column structure, the R component is approximately constant, and the G component and the B component show a significant increasing trend. Therefore, based on 0-255, the R, G, B components in the infilled wall area and the R component in the beam column area are segmented and linearly increased, and the G, B components in the beam column area are segmented and linearly decreased, so as to strengthen the color of the two areas, sharpen the boundary, and the contrast between the areas is more bright.

[0022] In order to extract the single-pixel width boundary of the building beam column and the infilled wall area, the application adopts a marked watershed algorithm to calculate the area boundary position, and adopts a gradient amplitude transformation to pre-process the image. Since the infrared correction image is still an RGB three-channel image after segmented linear adjustment of R, G and B components, direct gray scaling will lose a large amount of temperature distribution information, therefore, first, the sobel operator is used to filter the color-enhanced infrared image, and then the color change gradient graph is calculated, and the extended minimum value transformation method is used to set the minimum value threshold size, so that a three-channel binary image can be obtained without damaging the image information. The gradient graph and the binary graph are combined for morphological reconstruction, if the pixel value in the binary image is 1, the gradient value at the corresponding position in the gradient graph is set to the minimum value of the region, and the modified gradient graph is clear in the outline of the boundary of each temperature region. At this time, the obtained region outline is composed of multiple pixels, which is not conducive to the next step of boundary accumulation, therefore, a marked watershed algorithm is used to obtain the single-pixel width boundary of the beam column and the infilled wall area outline.

[0023] D. According to the pixel distribution of the boundary position of the building beam column and the infilled wall area in the infrared correction image, a pixel accumulation broken line graph is established, whether the abnormal boundary invasion and region connection representing the hollow defect in the beam column and the infilled wall area is detected, the total number of abnormal regions is counted, and then the building external surface falling risk level is evaluated, the building external wall health diagnosis report is generated, so that the city problem building group can be quickly screened out.

[0024] According to the histogram theory in the image theory, the application first proposes the boundary pixel accumulation theory. The histogram is a precise graphical representation reflecting the distribution of numerical data, according to a certain characteristic index of the data as the bottom edge; the frequency of objects meeting this characteristic index is counted as the height, and the distribution of the data can be observed visually. In the application, the frequency of the boundary pixels in each row and each column is counted for the pixels located at the boundary position in the image, a pixel accumulation broken line graph is established, the physical meaning of the data distribution in the broken line graph is explored, and through the related data characteristics, the positions of the infilled wall area, the beam column area and the abnormal boundary invasion, region connection and the like can be located.

[0025] In the obtained infrared correction image of the building outer wall, a boundary pixel longitudinal accumulation broken line graph is established with a pixel row coordinate as a bottom side and a boundary pixel frequency under the same row coordinate as a height. Local accumulation maximum value points in the accumulation graph representing the horizontal direction boundaries of the building beam column and infill wall area in the original thermal image are extracted, and a longitudinal normal accumulation interval between the two ends of the local accumulation maximum value in the vertical direction is defined. The accumulation of the boundary pixels outside the interval can be determined as an anomaly, thereby obtaining abnormal boundary pixels. According to the Euclidean distance between the pixels, a mean-shift clustering algorithm is used, and the normal accumulation interval span is used as a clustering bandwidth. Positionally adjacent abnormal boundary pixels are clustered into a class, and the number of class clusters in the clustering result represents the number of abnormal regions invaded by the boundary outside the longitudinal normal accumulation interval.

[0026] In the longitudinal normal accumulation interval, the widths of the zero accumulation intervals are counted, and the maximum value of the zero accumulation interval width is obtained by comparison. The zero accumulation intervals with a width less than 60% of the maximum value are considered as a boundary invasion anomaly. In addition, the shortest distance between each local accumulation maximum value and the adjacent zero accumulation interval, except for the local accumulation maximum values at the two ends of the normal accumulation interval, is calculated. If the distance exceeds the maximum width of the zero accumulation interval, it is considered that the local accumulation maximum value is abnormal, that is, there is an internal region connectivity anomaly in the corresponding region in the horizontal direction.

[0027] In the obtained boundary pixel coordinate matrix of the outer wall thermal image region, a boundary pixel transverse accumulation broken line graph is established with a pixel column coordinate as a bottom side and a boundary pixel frequency under the same column coordinate as a height. Local accumulation maximum value points in the accumulation region representing the vertical direction boundaries of the building beam column and infill wall area in the original thermal image are extracted, and a transverse normal accumulation interval between the two sides of the local accumulation maximum value in the horizontal direction is defined. The accumulation outside the interval can be determined as an anomaly, thereby obtaining abnormal boundary pixels. According to the Euclidean distance between the pixels, a mean-shift clustering algorithm is used, and the minimum distance between the zero accumulation regions in the longitudinal accumulation graph is used as a clustering bandwidth. Positionally adjacent abnormal boundary pixels are clustered into a class, and the number of class clusters in the clustering result represents the number of abnormal regions invaded by the boundary outside the transverse normal accumulation interval.

[0028] Therefore, the total number of abnormal regions of the building outer wall correction thermal image is the sum of the number of abnormal regions invaded by the boundary outside the longitudinal normal accumulation interval, the number of zero accumulation interval boundary invasion anomalies with a width less than 60% of the maximum zero accumulation interval width in the longitudinal normal accumulation interval, the number of internal region connectivity anomalies with a shortest distance between the local accumulation maximum value and the adjacent zero accumulation interval exceeding the maximum value of the zero accumulation interval width in the longitudinal normal accumulation interval, and the number of abnormal regions invaded by the boundary outside the transverse normal accumulation interval.

[0029] According to the distribution characteristics of the position of the horizontal direction boundary of the temperature uniform region of the beam column and the filled wall and the relationship between the same and the floor, in the obtained longitudinal stacking graph of the building outer wall, it can be known that the local stacking maximum value point represents the upper and lower boundaries of each floor, and therefore, the number of local stacking maximum value points has a double relationship with the number N of floors in the corrected image. Therefore, the number of floors of the current building outer wall infrared corrected image can be extracted according to the information in the longitudinal stacking broken line graph, and in combination with the total number of abnormal regions of the building outer wall infrared corrected image obtained in the above abnormality detection process, the risk rating of the outer wall caused by the hollowing defect can be performed on the target wall surface, so as to judge the damage degree of the building outer wall in the hollowing abnormality, wherein the risk rating A of the falling of the building outer wall is defined as the ratio of the total number of abnormal regions in the infrared corrected image to the number of floors. According to the above abnormal region detection process, the maximum number of effective abnormal regions existing in the infrared corrected image should be the sum of the number of abnormal regions outside the longitudinal normal stacking interval (the maximum number of this abnormal region is 2), the number of region intrusion and region connection abnormality in the longitudinal normal stacking interval (the maximum number of effective abnormal regions is L-1, L is the number of floors), and the number of abnormal regions outside the horizontal normal stacking interval (the maximum number of this abnormal region is 2*L), that is, 3*L+1. Therefore, the risk rating A of the falling of the outer wall is approximately considered to be between [0, 3]. Further, A∈[0, 0.5] is defined as low falling risk, A∈[0.5, 1] is defined as medium falling risk, and A∈[1, 3] is defined as high falling risk, so as to evaluate the risk rating of the abnormality detection result of the photographed building outer wall infrared thermal image.

[0030] According to the latitude and longitude coordinates and the shooting angle and other attributes of the photographed image, the geographical position of the problem building is located, and an abnormality detection report of the building outer wall is generated through the FLIR Tools software, the building outer wall image and the corresponding hollowing defect data are presented according to the requirement, and finally a high-rise building detection list of the outer wall hollowing defect abnormality is generated. The management library of the whole life cycle of the city high-rise building can be established, the annual inspection list, the high-risk list and the repair list are set according to different building conditions, the data support is provided for further management analysis, detection and repair. The buildings with large hidden dangers in the city are discovered in time, and the repair is arranged in advance.

[0031] The beneficial effects of the present application are as follows:

[0032] 1. According to the regular characteristic of the high-rise building frame structure, the abnormal changes of the beam column region and the filled wall region boundary in the building outer wall infrared thermal image are extracted to represent the hollowing defect, and the relationship between the surface hollowing defect of the outer wall and the abnormal changes of the temperature region boundary in the thermal image is established.

[0033] 2. Based on histogram theory, a method for establishing a polygonal line graph of boundary pixel accumulation in a temperature uniform region is proposed. By using the longitudinal and transverse distribution characteristics of boundary pixel accumulation, the location of infill walls and beam-column areas can be located, and abnormalities such as boundary intrusion and regional connectivity, which represent hollow defects, can be detected, effectively judging the health status of the building's exterior wall surface.

[0034] 3. A rapid initial screening method for defects in the exterior walls of high-rise buildings in the city was designed. Combined with urban automatic patrol vehicles, a data-driven approach was adopted to automatically identify abnormal conditions of building exterior walls, generate building exterior wall anomaly identification reports, establish a list of problematic buildings, dynamically monitor the safety status of the exterior surface layer of high-rise buildings, control the risk of exterior brick wall detachment, and eliminate the hidden danger of falling objects from heights in a timely manner.

[0035] The invention will be further described and explained below with reference to the accompanying drawings. Attached Figure Description

[0036] Figure 1 This is an infrared thermal image of the exterior wall of a building.

[0037] Figure 2 This is an infrared thermal image of the exterior wall frame structure of a building with no foreign objects.

[0038] Figure 3 This is a boundary model diagram of the exterior wall beam and column area and the infill wall area of ​​a defective building.

[0039] Figure 4 This is a diagram showing the division of the building's exterior wall area.

[0040] Figure 5 This is a vertically stacked line graph of the marked building exterior wall boundary pixels.

[0041] Figure 6 A horizontally stacked line graph of the marked building exterior wall boundary pixels.

[0042] Figure 7 The results show the clustering of abnormal boundary pixels at both ends of the building's exterior wall in the vertical direction, with 2 abnormal clusters.

[0043] Figure 8 The results show the clustering of abnormal boundary pixels on both sides of the building's exterior wall in the horizontal direction, with 6 abnormal clusters. Detailed Implementation

[0044] The present application is based on the obvious difference between the heat dissipation characteristics of the infilled wall and the beam column of the high-rise building with frame structure, and the infrared thermal image of the building outer wall is used to detect the abnormality of the hollow defect of the outer wall of the high-rise building and evaluate the risk of falling caused by the hollow defect. First, the boundary invasion and regional connectivity of the boundary position between the building beam column and the infilled wall region on the infrared thermal image are analyzed, so as to extract the feature difference between the abnormal region and the normal region on the infrared thermal image, accurately evaluate the falling risk level of the outer wall of the high-rise building, and screen out the high-rise building with high risk of falling of the outer wall. Specifically, the steps of the high-rise building outer wall falling abnormality detection and risk assessment method based on infrared thermal image of the present application are as follows:

[0045] A. Establish the geographical distribution map of urban high-rise buildings, collect the visible light images of the outer walls of the high-rise buildings and the corresponding infrared thermal images.

[0046] The above-mentioned urban high-rise buildings refer to high-rise buildings with frame structure. The city is divided into several blocks according to the area, each block is independently operated and maintained, a patrol detection route is formulated in each block, and an automatic patrol vehicle carrying a visible light imaging device and an infrared thermal imaging device is periodically operated according to a fixed running track to shoot the outer walls of the high-rise buildings along the way.

[0047] B. Segment the visible light image, extract the largest effective region representing the building outer wall surface in the image, and obtain the infrared thermal image correction image (infrared correction image) of the infrared thermal image region corresponding to the largest effective region.

[0048] In order to ensure the timeliness of image acquisition and image processing, image acquisition is carried out on the vehicle side, and image processing process and abnormal state evaluation are carried out on the cloud. After collecting the high-rise building outer wall image, the Faster R-CNN target detection algorithm is used to segment the visible light image, the processing region is reduced to the building outer wall surface, the corresponding infrared thermal image of the effective region representing the building outer wall surface is identified, and the infrared thermal image corresponding to the visible light image extracted from the building outer wall region is more than 40% of the whole infrared thermal image. The building outer wall region is identified as effective, and the largest effective region representing the building outer wall surface is extracted. The matlab data analysis platform is used to perform basic image processing operations such as Gaussian filtering, graying, dilation and corrosion on the extracted effective region infrared thermal image, filter out image noise, and then use improved Hough transform and perspective transform to obtain the infrared correction image of the building outer wall region.

[0049] C. Through the image processing algorithm, the boundary pixel distribution characteristics of the building beam column and the infilled wall region in the infrared correction image are extracted. The specific process is: the R, G, B component segmented linear adjustment is used for color enhancement of the infrared correction image, mainly to enhance the contrast of the image, strengthen the outline and boundary of the building beam column structure and the infilled wall, and make the color of each temperature uniform region more bright; the sobel operator is used to filter the color enhanced image, and the color component change gradient image is calculated; the extended minimum value transformation method is used to set the minimum value threshold to segment and remove the low gradient region, and the beam column region and the infilled wall region boundary contour are extracted; the single-pixel width boundary pixel coordinate matrix of the beam column and the infilled wall region is extracted by using the marked watershed algorithm, and the boundary pixel distribution characteristics of the building beam column and the infilled wall region are obtained.

[0050] D. According to the boundary position pixels of the building beam column and the infilled wall region in the infrared correction image, the pixel accumulation polyline graph is established, the abnormality of the region is detected, such as boundary intrusion and region connection, the total number of abnormal regions is counted, the risk grade of building external surface falling is evaluated, and the building external wall health diagnosis report is generated.

[0051] Figure 1 For the infrared thermal image of the building external wall of a frame structure, there are anomalies such as thermal spots and boundary damage in the image. The red thermal spot pointed by the white arrow on the right side has a sharp temperature jump with the adjacent temperature uniform region at the boundary. The traditional temperature anomaly detection method can detect the anomaly at this place, but the region pointed by the gray arrow on the left side has a significant sign of intrusion of the temperature uniform region of the infilled wall into the beam column region. The intrusion part and the lower infilled wall region belong to the same temperature uniform region, and the traditional temperature anomaly detection method cannot effectively detect the anomaly at this place through threshold segmentation and contour extraction, and there is a situation of missed judgment and misjudgment. Since the traditional infrared detection method depends on local temperature anomaly, there are defects, so in the field of building external wall detection, the distribution characteristics of high-rise building frame structure can be combined to set constraints that conform to the natural law of the scene, analyze the relationship between surface damage and boundary damage in the infrared thermal image, establish a model of boundary damage of temperature uniform region, and effectively judge whether there is surface defect in the building external wall.

[0052] Figure 2 The black line marked region is the building infilled wall region, which is commonly made of hollow bricks, has small specific heat capacity, and heats up quickly under sunlight, and has uniform temperature distribution, which appears as a white wide rectangular region on the thermal image; the gray-white line marked region is the building beam column region, which is commonly made of concrete pouring and steel, has large specific heat capacity, and heats up slowly under sunlight, and has uniform temperature distribution, which appears as a red narrow frame region on the infrared thermal image (since the submitted is not a color image, Figure 2 , this region appears gray).

[0053] In the infrared thermal image of the building outer wall with good health and no obvious abnormalities, the boundary of the filling wall and beam column area should be regular and orderly, and the temperature uniform area should not have significant boundary invasion damage or regional connectivity phenomenon.

[0054] Due to the difference in material and structure and the difference in construction process between the beam column structure and the filling wall, the environmental stress resistance of the material at the boundary is weaker than that of the two, and the first natural defect such as cracking and hollowing occurs under the erosion of environmental stress. It can be found from the infrared thermal image used in the previous study that the temperature abnormal area is often distributed near the boundary of the two, and there are few defects in the area. The boundary of the abnormal area is usually connected with the temperature area of the two, and the surface anomaly starts from here or is related to it.

[0055] When the beam column structure side has hollowing, a high-temperature area is formed, which is connected with the adjacent filling wall temperature area, showing that the high-temperature area invades the low-temperature area. Therefore, if there is a boundary blur and invasion phenomenon between the building surface filling wall and the beam column in the infrared thermal image, it indicates that the internal structure is damaged or hollow. It can be determined that there is a higher probability of hollowing defects at this place.

[0056] Since the infrared thermal image can detect the temperature change of the building outer wall surface hollowing defect and the outer wall skin peeling caused by it, there is a difference between the temperature of the abnormal area and the normal area, so the area boundary invasion and the area connectivity are proposed to characterize the abnormality of the building outer wall hollowing defect.

[0057] Based on the structural characteristics of the building outer wall, a plane model of the boundary between the filling wall and the beam column area on the outer surface of the building is constructed. The connected domain formed by the boundary is the filling wall area inside, and the beam column area outside, which has four boundary abnormalities: longitudinal boundary invasion at both ends in the vertical direction, lateral boundary invasion on both sides in the horizontal direction, internal boundary invasion, and internal regional connectivity. Figure 3 In the model shown, there are two longitudinal boundary invasion abnormalities at both ends in the vertical direction, six lateral boundary invasion abnormalities on both sides in the horizontal direction, one internal boundary invasion abnormality, and one internal regional connectivity abnormality.

[0058] Since the abnormal situation of the building outer wall cannot be predicted in the abnormal detection of the building outer wall, in order to detect the four boundary abnormalities and the number of abnormal areas, Figure 3 The building outer surface image shown is divided into three areas, as shown in Figure 4The boundary pixel longitudinal and transverse accumulation fold line graphs are established according to the boundary pixel distribution characteristics of the building beam column and the filled wall region, and the boundary pixel longitudinal and transverse accumulation fold line graph method is proposed. According to the established pixel accumulation fold line graph, the positions of the filled wall and the beam column region are located by the longitudinal and transverse distribution characteristics of the boundary pixel accumulation, the abnormality of the boundary intrusion and the region connection existing in the filled wall and the beam column region is detected respectively, and the risk level of the building outer surface falling off in the infrared correction image is effectively judged. The risk level A of the building outer surface falling off in the infrared correction image is defined as the ratio of the total number of abnormal regions of the infrared correction image to the number of building layers contained in the image, the number of building layers of the infrared correction image is defined as 1 / 2 of the local accumulation maximum value point number of the longitudinal normal accumulation interval in the boundary pixel longitudinal accumulation fold line graph, if the calculated number of layers is a decimal, then the integer part is taken; and A∈[0, 0.5] is defined as low falling off risk, A∈[0.5, 1] is defined as medium falling off risk, and A∈[1, 3] is defined as high falling off risk.

[0059] In the infrared correction image of the building outer wall, the boundary pixel longitudinal accumulation fold line graph is established with the pixel row coordinate as the bottom side and the boundary pixel frequency under the same row coordinate as the height, and the boundary pixel transverse accumulation fold line graph is established with the pixel column coordinate as the bottom side and the boundary pixel frequency under the same column coordinate as the height. According to the boundary pixel longitudinal and transverse accumulation fold line graphs, the positions of the filled wall and the beam column region are located by the longitudinal and transverse distribution characteristics of the boundary pixel accumulation, the abnormality of the boundary intrusion and the region connection existing in the filled wall and the beam column region is detected respectively, and the risk level of the building outer surface falling off in the infrared correction image is effectively judged. Figure 3 The boundary pixel longitudinal and transverse accumulation fold line graphs drawn from the building outer surface image shown in Figure 5 、 Figure 6 are shown in

[0060] The region boundary is the boundary pixel dense area, so the row and column boundary pixel frequency of the boundary is large, which will present a local accumulation maximum value in the accumulation fold line graph. In the longitudinal accumulation fold line graph of the beam column region, the boundary pixel on each row is 0, which will present a zero accumulation interval in the accumulation fold line graph.

[0061] In the Figure 5 , the positions of the zero accumulation interval in the longitudinal normal accumulation interval are detected, and the data points are marked with the symbol "o". The zero accumulation interval list Nones[zero_1, zero_2……] is extracted, zero_i=[up_i, low_i]. The zero accumulation interval zero_i is stored in the list, up_i represents the upper limit of the i-th zero accumulation interval, and low_i represents the lower limit of the zero accumulation interval. If the interval width is 1, it is saved as a single value. The maximum value of the interval width of the zero accumulation interval None_max and the average value of the interval width of the zero accumulation interval None_aver can be obtained.

[0062] Extract the list of longitudinal local accumulation maxima Peaks_V=[peak_v_1, peak_v_2……], peak_v_i=(num_i, loc_i), num_i represents the size of the maximum value (i.e. pixel accumulation value), loc_i represents the longitudinal position of the maximum value (i.e. the row number). Calculate the average value of the local accumulation maxima peak_v_aver as a threshold for segmentation, and filter out the local accumulation maxima in the list whose accumulation value is lower than peak_v_aver. And with the maximum value of the zero interval width None_max as the interval tolerance, compare each local accumulation maxima peak_v_j in Peaks_V after average value segmentation within the interval [loc_j-None_max, loc_j+None_max] with the local accumulation maxima that may exist in it, and retain the larger value, thereby filtering out adjacent accumulation maxima. The list of retained local accumulation maxima is denoted as Peaks_V_bound. The local accumulation maxima retained after two screenings are denoted by the symbol "*". These peaks can represent the transverse boundaries of each infilled wall region.

[0063] For the abnormality outside the normal accumulation interval in the vertical direction of the infilled wall, take the top and bottom local accumulation maxima positions loc_max and loc_min as the upper and lower limits of the interval to obtain the longitudinal accumulation interval Veitical_pile. Detect whether there is abnormal accumulation outside the accumulation interval and mark it with the symbol "◇", which can determine whether there is a boundary intrusion phenomenon at the top and bottom of the infilled wall region. From the returned abnormal accumulation interval, the row coordinates of the abnormal positions can be located, and further column traversal of the region where these rows are located can be performed to find whether there are boundary pixels and the column coordinates corresponding to the pixels. Since the boundary pixel positions of the same abnormal region are close to each other, the Euclidean distance between abnormal pixels is used as a feature, and the transverse accumulation polyline interval span is set to the size of the bandwidth K value. Using the mean-shift clustering algorithm, abnormal pixels belonging to the same abnormal region can be clustered into one class. The clustering result is shown in FIG. 8, and the number of boundary intrusion abnormalities outside the two end longitudinal normal accumulation regions in the vertical direction of the building outer wall is num1=2. Figure 7

[0064] For the abnormality inside the vertical direction accumulation interval of the infilled wall, since the beam column region presents a zero accumulation interval, the two sides are usually the infilled wall boundary, which presents a local accumulation maximum. Therefore, if the zero accumulation interval width is lower than 0.6*None_max, it indicates that the beam column region has a more serious boundary intrusion. The greater the intrusion degree, the narrower the zero accumulation interval. Figure 5 ​Three zero accumulation intervals can be observed in the image, among which the zero accumulation interval located at line 32 has a width of 1, which is much smaller than the other two zero accumulation intervals, thus obtaining the number num2=1 of beam-column regions in the area that are obviously invaded, i.e., the number of abnormal boundary invasion within the longitudinal normal accumulation interval.

[0065] Without considering the local accumulation maxima at the top and bottom of the vertical direction (located at loc_max and loc_min) in Peaks_V_bound, the pixel distance dist_m between the remaining local accumulation maxima peak_v_m and its nearest zero accumulation interval zero_n is calculated. In the normal position, the zero interval span is large, and the corresponding local accumulation maxima on both sides are very close to the zero interval, and even if there is boundary invasion, the distance will not exceed None_max. However, if there is a filling wall region connection, the zero accumulation interval representing the beam-column region at this position disappears and becomes a low accumulation region, and the distance between the accumulation peak at this position and its nearest zero accumulation interval will be at least close to one filling wall width plus one beam-column region width. Therefore, with None_max as the threshold, the local accumulation maxima with dist_m greater than the threshold can be determined as abnormal. Due to symmetry, the number of distance abnormal maxima is counted and divided by 2, and if the resulting peak number is odd, the result is rounded up after being divided by 2, and the number of internal region connection abnormalities can be obtained. Figure 5 In the image, two local accumulation maxima located at line 92 and line 103 are observed to be far beyond the maximum zero interval width from the adjacent zero accumulation interval, thus determining that there is an internal region connection abnormality at this position. As described above, the number of internal region connection abnormalities in the longitudinal normal accumulation interval num3=1 can be obtained by the calculation method.

[0066] In Figure 6 In the image, the list of horizontal local accumulation maxima Peaks_H, [peak_h_1, peak_h_2……] is extracted, peak_h_i=(num_i, loc_i), num_i represents the size of the maximum value, and loc_i represents the position of the maximum value. The average value peak_h_aver of the horizontal accumulation maxima is calculated as a threshold for segmentation, and the accumulation maxima greater than peak_h_aver in the list are marked with the symbol "*"; the maximum value loc_max and the minimum value loc_min of the position of the horizontal accumulation maxima in the list are used as the upper and lower limits of the interval, and the main horizontal accumulation interval of the building filling wall Horizontal_pile can be located. The regions outside the accumulation interval are reversely detected, and the existence of accumulation is abnormal, which is marked with the symbol "◇".

[0067] The returned abnormal stacking intervals can be used to locate the column number of the abnormal location. Further, the regions containing these columns are traversed to check for boundary pixels and their corresponding row numbers. This allows for precise location of the abnormal location of the lateral intrusion. Since the boundary pixels of the same abnormal region are close together, using the pixel Euclidean distance as a feature, and the minimum spacing between zero-stack intervals in the vertical stacked line graph as the bandwidth K value, the mean-shift clustering algorithm can be used to cluster them into one class, thus obtaining the number of external anomalies in the lateral area of ​​the building's exterior wall. Figure 6 It can be observed that outside the local maxima on both sides of the accumulation interval, there is still significant pixel accumulation, indicating obvious lateral boundary intrusion on both sides. From the lateral accumulation line graph, the column range of the abnormal pixels is [4,8], [40,44]. By performing row traversal on the corresponding columns in the image and detecting the boundary pixels, the set of lateral abnormal boundary pixels is obtained. Then, according to the above clustering method, the abnormal pixels with close distances are clustered into one class. The clustering result is as follows. Figure 8 As shown, the number of intrusion anomalies at the outer boundary of the normal horizontal accumulation area on both sides of the building's exterior wall is num4=6.

[0068] The total number of hollow defects detected above is num = num1 + num2 + num3 + num4 = 10. Furthermore, the number of local stacked maxima in the Peaks_V_bound list obtained from the vertical stacked line graph is N = 10. Therefore, the number of infill wall layers in the obtained image is L = N / 2 = 5. Consequently, the risk level of the target building's exterior wall can be calculated as A = num / L = 2, indicating a fairly serious exterior wall anomaly.

Claims

1. A method for detecting and risk assessment of high-rise building outer wall falling anomaly based on infrared thermal image, characterized in that, The method steps are as follows: A. Establish a geographic distribution map of urban high-rise buildings, collect visible light images of the outer walls of each high-rise building and corresponding infrared thermal images; B. Segment the visible light image, extract the maximum effective area representing the building outer wall surface in the image, and obtain the infrared correction image of the infrared thermal image area corresponding to the maximum effective area; C. Extract the boundary pixel distribution characteristics of the building beam column and infill wall area in the infrared correction image through image processing algorithms; D. According to the pixel distribution of the boundary position of the infill wall area in the infrared correction image, establish a pixel accumulation broken line graph, detect whether there is a boundary intrusion and area connection anomaly representing the hollow defect between the beam column and the infill wall area, count the total number of abnormal areas, and then evaluate the building outer surface falling risk level corresponding to the infrared correction image, and generate a building outer wall health diagnosis report; The specific process of detecting the area boundary anomaly in step D is: according to the pixel distribution of the boundary position of the infill wall and the beam column area in the infrared correction image, taking the pixel row coordinate as the base, and taking the boundary pixel frequency in the same row coordinate in the correction image as the height, a boundary pixel longitudinal accumulation broken line graph is established, and the abnormality of boundary intrusion outside the longitudinal normal accumulation interval is detected; And in the longitudinal normal accumulation interval, the width of each zero accumulation interval is counted, the maximum value of the zero accumulation interval width is calculated, if the width of a zero accumulation interval is less than 60% of the maximum value of the zero accumulation interval width, it is determined that the infrared image area corresponding to the position of the zero accumulation interval exists boundary intrusion anomaly; if the shortest distance between a local accumulation maximum value in the longitudinal normal accumulation interval and the adjacent zero accumulation interval exceeds the maximum value of the zero accumulation interval width, it is determined that the area boundary corresponding to the local accumulation maximum value on the image exists internal area connection anomaly; at the same time, taking the pixel column coordinate as the base, and taking the boundary pixel frequency in the same column coordinate in the correction image as the height, a boundary pixel transverse accumulation broken line graph is established, and the abnormality of boundary intrusion outside the transverse normal accumulation interval is detected; When counting the total number of abnormal areas, the number of abnormal areas of boundary intrusion outside the longitudinal normal accumulation interval is obtained by using the mean-shift clustering algorithm with the span of the normal accumulation interval in the longitudinal accumulation graph as the clustering bandwidth according to the Euclidean distance between each abnormal pixel outside the normal accumulation interval detected in the longitudinal accumulation broken line graph, and the abnormal boundary pixels adjacent in position are clustered into one class, and the number of class clusters in the clustering result is the number of abnormal areas; The number of abnormal areas of boundary intrusion outside the transverse normal accumulation interval is obtained by using the mean-shift clustering algorithm with the minimum distance between each zero accumulation area in the longitudinal accumulation graph as the clustering bandwidth according to the Euclidean distance between each abnormal pixel outside the normal accumulation interval detected in the transverse accumulation broken line graph, and the abnormal boundary pixels adjacent in position are clustered into one class, and the number of class clusters in the clustering result is the number of abnormal areas; The total number of abnormal regions is the sum of the number of abnormal regions invaded from the boundary of the longitudinal normal accumulation interval, the number of abnormal regions invaded from the boundary of the longitudinal normal accumulation interval, the number of abnormal regions invaded from the boundary of the longitudinal normal accumulation interval, the number of abnormal regions invaded from the boundary of the longitudinal normal accumulation interval, and the number of abnormal regions invaded from the boundary of the longitudinal normal accumulation interval.

2. The infrared thermographic-based high-rise building outer wall spalling anomaly detection and risk assessment method according to claim 1, characterized in that, The risk level A of the building outer surface falling off in the infrared correction image is defined as the ratio of the total number of abnormal regions in the infrared correction image to the number of building layers contained in the image, the number of building layers in the infrared correction image is defined as 1 / 2 of the number of local accumulation maximum points in the longitudinal normal accumulation interval in the longitudinal accumulation fold line of the boundary pixel, if the calculated number of layers is a decimal, then take the integer part; and A∈[0, 0.5] is low falling risk, A∈[0.5, 1] is medium falling risk, A∈[1, 3] is high falling risk. 3.The infrared thermographic-based high-rise building outer wall falling anomaly detection and risk assessment method according to claim 1, characterized in that, The specific process of step C is: (1) Color enhancement is performed on the infrared correction image by using R, G, B component piecewise linear adjustment; (2) The color-enhanced image is filtered by using a sobel operator to calculate the color component change gradient image; (3) The minimum threshold is set by using the extended minimum value transformation method to segment and remove low gradient regions, and the beam column region and the infilled wall region boundary contour are extracted; (4) The single-pixel width boundary pixel coordinate matrix of the beam column and the infilled wall region is extracted by using the marked watershed algorithm.

4. The infrared thermographic-based high-rise building outer wall spalling anomaly detection and risk assessment method according to claim 1, characterized in that, The effective area image of the building outer wall surface extracted in step B should account for more than 40% of the area of the visible light image.

5. The infrared thermographic-based high-rise building outer wall spalling anomaly detection and risk assessment method according to claim 1, characterized in that, The infrared correction image in step B is obtained by using improved Hough transformation and perspective transformation processing on the infrared thermal image region corresponding to the largest effective region of the extracted building outer wall surface. 6.The infrared thermographic-based high-rise building outer wall falling anomaly detection and risk assessment method according to claim 1, characterized in that, Before obtaining the infrared correction image in step B, the matlab data analysis platform is used to filter the image noise of the infrared thermal image corresponding to the effective region of the building outer wall surface, and the image object shape structure definition and region contour performance are enhanced.

7. The infrared thermographic-based high-rise building outer wall falling anomaly detection and risk assessment method according to claim 1, characterized in that, In step A, when collecting the outer wall image, the city is divided into several blocks according to the area size, and the automatic line patrol vehicle equipped with double optical thermal imaging equipment in each block is periodically operated according to the fixed running track to shoot the outer wall of the high-rise building along the way.

8. The infrared thermographic-based high-rise building outer wall spalling anomaly detection and risk assessment method according to claim 1, characterized in that, In step B, Faster R-CNN target detection algorithm is used to segment the visible light image.

Images