Building surface damage detection method and device and nonvolatile storage medium

Abstract

The invention discloses a building surface damage detection method and device and a nonvolatile storage medium. The method comprises the following steps: acquiring image data of a building surface; the detection model is adopted to determine initial damage information of the image data, the damage information at least comprises a damage position, a damage type and a segmentation mask, and the segmentation mask is used for determining a damage area; determining the damage area and the damage degree of the image data based on the initial damage information, and taking the initial damage information, the damage area and the damage degree as detection results; and generating a visual report based on the detection result, and pushing the visual report to the user terminal. According to the invention, the technical problems of low efficiency, strong subjectivity and difficulty in large-scale long-term monitoring due to dependence on manual detection in a building surface damage detection method in the prior art are solved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and more particularly, to a method, apparatus, and non - volatile storage medium for detecting damage to the surface of a building. Background Art

[0002] The detection of damage to the surface of a building is an important link in ensuring building safety. Over time, buildings are affected by various factors such as the natural environment (e.g., weathering, corrosion) and human factors (e.g., improper construction, overloading), resulting in various damages such as cracks, peeling, and rust on the surface. If these damages are not detected and treated in a timely manner, they may lead to a decline in the stability of the building structure and even cause serious safety accidents.

[0003] In related technologies, the methods for detecting damage to the surface of a building mainly rely on manual observation and recording, and have the following significant drawbacks: 1. Low efficiency: Manual detection requires checking each part of the building surface one by one, which is time - consuming. Especially in the case of large - scale or high - rise buildings, the detection workload is huge and the efficiency is extremely low. 2. Strong subjectivity: The results of manual detection often depend on the experience and judgment of the detection personnel. Different detection personnel may draw different conclusions, resulting in the inconsistency and unreliability of the detection results. 3. Difficult to conduct large - scale long - term monitoring: Due to limitations in manpower and resources, it is difficult to comprehensively and continuously monitor buildings, especially for aging buildings that require regular inspections or buildings in harsh environments. Therefore, the methods for detecting damage to the surface of a building in related technologies rely on manual detection, and have problems such as low efficiency, strong subjectivity, and difficulty in conducting large - scale long - term monitoring.

[0004] In view of the above problems, no effective solution has been proposed yet. Summary of the Invention

[0005] Embodiments of the present application provide a method, apparatus, and non - volatile storage medium for detecting damage to the surface of a building, so as to at least solve the technical problems that the methods for detecting damage to the surface of a building in related technologies rely on manual detection, and have problems such as low efficiency, strong subjectivity, and difficulty in conducting large - scale long - term monitoring.

[0006] According to one aspect of the embodiments of the present application, a method for detecting damage to the surface of a building is provided, including: acquiring image data of the building surface; using a detection model to determine initial damage information of the image data, where the damage information at least includes a damage location, a damage type, and a segmentation mask, and the segmentation mask is used to determine the damage area; determining the damage area and damage degree of the image data based on the initial damage information, and using the initial damage information, the damage area, and the damage degree as detection results; generating a visualization report based on the detection results and pushing the visualization report to a user terminal.

[0007] In some embodiments of the present application, determining initial damage information of image data using a detection model includes: inputting the image data into the detection model; extracting multi-level image feature maps in the image data based on a first network layer in the detection model, where the resolution of each layer of image feature maps decreases layer by layer; performing feature fusion on the image feature maps based on a second network layer in the detection model to obtain a plurality of fusion points, where the second network layer is the next network layer of the first network layer; adjusting the weights of feature channels at each fusion point based on a third network layer in the detection model to obtain a fused feature map; and determining the initial damage information based on the fused feature map.

[0008] In some embodiments of the present application, performing feature fusion on multi-level image feature maps based on a second network layer in the detection model to obtain a plurality of fusion points includes: the second network layer performing an upsampling operation on the multi-level image feature maps along a top-down path to obtain a plurality of first image feature maps, where the upsampling operation includes at least bilinear interpolation or transposed convolution, and the top-down path is from the image feature map corresponding to the layer with the lowest resolution to the image feature map corresponding to the layer with the highest resolution; and performing feature fusion on the plurality of first image feature maps and the image feature maps with the same resolution in the multi-level image feature maps to obtain a plurality of fusion points, where each fusion point corresponds to a fused image feature map.

[0009] In some embodiments of the present application, determining the damage area and damage degree of the image data based on the initial damage information includes: calculating the number of pixels at the damage location based on the segmentation mask in the initial damage information, and converting the number of pixels into a physical size to obtain the damage area; and determining the damage degree according to the damage degree standard based on the damage type and the damage area, where different loss levels correspond to different damage degrees.

[0010] In some embodiments of the present application, generating a visualization report based on the detection result includes: generating a damage distribution map based on the initial damage information in the detection result, where the damage distribution map is used to display the damage location and damage type; generating a damage report based on the detection result, where the damage report includes at least the damage area, damage degree, damage type, and damage location; and using the damage distribution map and the damage report as the visualization report.

[0011] In some embodiments of the present application, the detection model is obtained by training an initial detection model by inputting a structured data set stored in a database, and the structured data set is obtained in the following manner: collecting sample image data on the surface of a building at preset time intervals; annotating the sample image data and storing the annotated sample image data in the database, where the annotation content of the annotated sample image data includes the damage type and the damage location; and using all the annotated sample image data stored in the database as the structured data set.

[0012] In some embodiments of the present application, the method further includes: establishing a damage database, where the damage database is used to store the detection results obtained each time; updating the damage database after each new detection result is generated; obtaining a damage data set in the damage database, where the damage data set is composed of all the detection results in the damage database; predicting the damage risk information of the image data in a future preset time based on the damage data set, where the damage risk information at least includes the predicted loss position and the predicted loss type; generating an alarm message based on the damage risk and pushing it to the user terminal.

[0013] According to another aspect of the embodiments of the present application, there is also provided a detection device for building surface damage, including: an acquisition module, configured to acquire image data of the building surface; a first determination module, configured to determine the initial damage information of the image data by using a detection model, where the damage information at least includes the damage position, the damage type, and a segmentation mask, and the segmentation mask is used to determine the damage area; a second determination module, configured to determine the damage area and the damage degree of the image data based on the initial damage information, and use the initial damage information, the damage area, and the damage degree as the detection results; a push module, configured to generate a visualization report based on the detection results and push the visualization report to the user terminal.

[0014] According to another aspect of the embodiments of the present application, there is also provided a non-volatile storage medium, in which a program is stored, and when the program runs, it controls the device where the non-volatile storage medium is located to execute the above-mentioned detection method for building surface damage.

[0015] According to another aspect of the embodiments of the present application, there is also provided an electronic device, including: a memory and a processor, where the processor is configured to run the program stored in the memory, and when the program runs, it executes the above-mentioned detection method for building surface damage.

[0016] According to another aspect of the embodiments of the present application, there is also provided a computer program product, including computer instructions, and when the computer instructions are executed by a processor, they implement the above-mentioned detection method for building surface damage.

[0017] In the embodiments of the present application, image data of the building surface is acquired; a detection model is used to determine the initial damage information of the image data, where the damage information at least includes the damage location, damage type, and segmentation mask, and the segmentation mask is used to determine the damage area; based on the initial damage information, the damage area and damage degree of the image data are determined, and the initial damage information, damage area, and damage degree are used as the detection results; based on the detection results, a visualization report is generated and pushed to the user terminal. By using the detection model to determine the initial damage information of the image data and based on the initial damage information to determine the damage area and damage degree of the image data, the detection results are finally obtained, achieving the purpose of accurately positioning the damage location on the building surface, and further solving the technical problems in the related art that the detection method for damage on the building surface relies on manual detection, has low efficiency, strong subjectivity, and is difficult to perform large-scale long-term monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and form a part of the present application. The illustrative embodiments and descriptions thereof of the present application are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0019] Figure 1 is a hardware structure block diagram of a computer terminal for implementing a method for detecting damage on the building surface according to an embodiment of the present application;

[0020] Figure 2 is a flowchart of a method for detecting damage on the building surface according to an embodiment of the present application;

[0021] Figure 3 is a flowchart of another method for detecting damage on the building surface according to an embodiment of the present application;

[0022] Figure 4 is a schematic structural diagram of a device for detecting damage on the building surface according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0024] The information collected in the embodiments of this application is information and data authorized by the user or fully authorized by all parties. Moreover, for the processing of relevant data, such as collection, storage, use, processing, transmission, provision, disclosure, and application, all comply with the relevant laws, regulations, and standards of the relevant regions, necessary confidentiality measures are taken, it does not violate public order and good customs, and a corresponding operation entry is provided for users to choose to authorize or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered.

[0025] It should be noted that the terms "first", "second", etc. in the description, claims, and above-mentioned drawings of this application are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of this application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0026] To better understand the embodiments of this application, the technical terms involved in the embodiments of this application are explained as follows:

[0027] Artificial Intelligence of Things (abbreviated as AIoT): A concept that combines artificial intelligence (AI) technology and Internet of Things (IoT) technology. The goal of AIoT is to analyze and process the data collected by IoT devices through AI technology to achieve more intelligent decision-making and automated operations.

[0028] EfficientNet: An efficient convolutional neural network architecture designed to improve the efficiency and performance of convolutional neural networks. It optimizes in three dimensions: depth, width, and resolution through a compound scaling method to achieve better performance and efficiency.

[0029] Feature Pyramid Network (FPN): It is an architecture used to enhance the feature extraction ability of convolutional neural networks, especially in object detection tasks. The design of FPN enables the model to utilize both high-level semantic information and low-level detail information simultaneously, thereby improving the accuracy and robustness of detection. FPN combines high-level and low-level features in a top-down manner to construct a feature pyramid, enabling the model to effectively detect objects of different scales. Specifically, FPN adds a lateral connection after each convolutional layer, downsamples the high-level features and fuses them with the low-level features, so that the features of each layer can contain both semantic information and spatial details, improving the accuracy and robustness of detection.

[0030] Squeeze-and-Excitation Network (SENet): It is a module used to improve the performance of convolutional neural networks, aiming to enhance the model's representation ability by adaptively recalibrating the weights of feature channels.

[0031] In related technologies, the building surface damage detection method mainly relies on manual observation and recording, which has the following significant drawbacks: 1. Low efficiency: Manual detection requires checking each part of the building surface one by one, which is time-consuming. Especially in the case of large or high-rise buildings, the detection workload is huge and the efficiency is extremely low. 2. Strong subjectivity: The results of manual detection often depend on the experience and judgment of the detection personnel. Different detection personnel may draw different conclusions, resulting in the inconsistency and unreliability of the detection results. 3. Difficult to conduct large-scale long-term monitoring: Due to limitations in manpower and resources, it is difficult to comprehensively and continuously monitor buildings. Especially for aging buildings that need to be regularly inspected or buildings in harsh environments. Therefore, the building surface damage detection method in related technologies relies on manual detection, with problems such as low efficiency, strong subjectivity, and difficulty in conducting large-scale long-term monitoring. Therefore, there are technical problems in the building surface damage detection method in related technologies relying on manual detection, including low efficiency, strong subjectivity, and difficulty in conducting large-scale long-term monitoring. To solve this problem, relevant solutions are provided in the embodiments of this application, which are described in detail below.

[0032] According to the embodiments of this application, an embodiment of a method for detecting building surface damage is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0033] The method embodiments provided by the embodiments of the present application can be executed in a computer terminal or a similar computing device. Figure 1 The following shows a hardware block diagram of a computer terminal for implementing a method for detecting damage to a building surface. As Figure 1 shown, the computer terminal 10 may include one or more processors 102 (shown as 102a, 102b, ……, 102n in the figure) (the processor 102 may include, but is not limited to, processing devices such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may further include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply, and / or a camera. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 may further include more or fewer components than those Figure 1 shown, or have a different configuration from that Figure 1 shown.

[0034] It should be noted that the above one or more processors 102 and / or other data processing circuits are generally referred to as "data processing circuits" in this article. The data processing circuit may be embodied in software, hardware, firmware, or any combination thereof, in whole or in part. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any one of the other elements in the computer terminal 10. As involved in the embodiments of the present application, the data processing circuit is used for processor control (such as the selection of a variable resistance terminal path connected to an interface).

[0035] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method for detecting damage to the building surface in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implements the above-mentioned method for detecting damage to the building surface. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, a flash memory, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely disposed relative to the processor 102, and these remote memories may be connected to the computer terminal 10 through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0036] The transmission device 106 is used to receive or send data via a network. Specific examples of the above network may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a Radio Frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0037] The display can be, for example, a touch-screen liquid crystal display (LCD), which enables the user to interact with the user interface of the computer terminal 10.

[0038] Under the above operating environment, an embodiment of a method for detecting building surface damage is provided in an embodiment of the present application. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0039] As Figure 2 shown, it is a flowchart of a method for detecting building surface damage provided by an embodiment of the present application, including:

[0040] Step S202, obtaining image data of the building surface.

[0041] The following are specific embodiments:

[0042] Install high-definition intelligent cameras around the building and at key positions (such as exterior walls, load-bearing structures, vulnerable areas, etc.). These cameras should have a wide viewing angle and high resolution to ensure that the building surface conditions can be comprehensively and clearly captured. The cameras are connected to a central processing unit (such as a server or cloud platform) through a wired or wireless network (such as Wi-Fi, 4G / 5G, etc.) to ensure real-time data transmission; the central processing unit supports running complex artificial intelligence algorithms, such as deep learning models such as the Hybrid Damage Detection Network (abbreviated as HybridDamageNet). Perform damage detection on the image data, generate damage information, and may trigger warnings or maintenance suggestions according to the detection results. Obtain the image data of the building surface collected by the high-definition intelligent cameras.

[0043] Step S204, use a detection model to determine the initial damage information of the image data. The damage information at least includes the damage location, damage type, and segmentation mask, where the segmentation mask is used to determine the damage area.

[0044] In the technical solution provided in step S204, there are various implementation methods for using a detection model to determine the initial damage information of the image data. For example: input the image data into the detection model; extract multi-level image feature maps in the image data based on the first network layer in the detection model, where the resolution of each layer of the image feature map decreases layer by layer; perform feature fusion on the image feature maps based on the second network layer in the detection model to obtain multiple fusion points, where the second network layer is the next network layer after the first network layer; adjust the weights of the feature channels at each fusion point based on the third network layer in the detection model to obtain a fused feature map; determine the initial damage information based on the fused feature map. Through multi-level feature extraction and fusion, the content of the image can be more comprehensively understood, and the accuracy and robustness of damage detection can be improved. Especially in complex environments and lighting conditions, it can effectively identify damage details and is suitable for refined detection of building surface damage.

[0045] In the above steps, there are various implementation methods for performing feature fusion on multi-level image feature maps based on the second network layer in the detection model to obtain multiple fusion points. For example: the second network layer performs an upsampling operation on the multi-level image feature maps along a top-down path to obtain multiple first image feature maps, where the upsampling operation at least includes bilinear interpolation or transposed convolution, and the top-down path is from the image feature map corresponding to the lowest resolution layer to the image feature map corresponding to the highest resolution layer; perform feature fusion on the multiple first image feature maps and the image feature maps with the same resolution in the multi-level image feature maps to obtain multiple fusion points, where each fusion point corresponds to a fused image feature map.

[0046] The following are specific embodiments:

[0047] Preprocess the image data. For example, the preprocessing first performs denoising to eliminate noise interference in the image. Then, enhance the contrast of the image data to improve the clarity and detail performance of the image. Finally, perform image correction on the image data to eliminate problems such as perspective distortion and ensure the accuracy and consistency of the image.

[0048] Input the preprocessed image data into a detection model (e.g., an AI model generated based on HybridDamageNet for building surface damage detection and classification tasks, which integrates three advanced deep learning techniques: EfficientNet, FPN, and SENet). The first network layer in the detection model (e.g., EfficientNet) extracts feature maps at different levels from the image data. The first network layer is used to extract the image features of the image data. The architecture design of EfficientNet enables it to generate multi-level and multi-scale feature maps, which are stacked by multiple compound blocks. Each compound block contains a depthwise separable convolution (for efficiently capturing texture information), a pointwise convolution (for feature transformation), and a SENet module (for adaptive adjustment of feature channel weights). During the model training process, the input image data passes through these compound blocks, gradually extracting multi-level feature maps. The feature maps generated by the bottommost compound blocks have a low resolution but contain global information, which is suitable for identifying large damage areas. While the feature maps generated by the topmost compound blocks have a high resolution and can capture small details, such as tiny cracks. EfficientNet adjusts the depth, width, and resolution parameters in different compound blocks through a compound scaling method, thereby obtaining a series of image feature maps with different resolutions and feature depths. As the network layer deepens, the resolution of the feature maps decreases layer by layer. In HybridDamageNet, these feature maps will be passed to the second network layer (e.g., FPN) for feature fusion so that the model can perform more accurate damage localization and classification based on multi-scale information. FPN performs an upsampling operation on the feature maps generated by EfficientNet along the top-down path: starting from the EfficientNet output feature map with the lowest resolution, which is usually at the deepest layer of the network. The feature map has a low resolution but contains rich semantic information. An upsampling operation is performed to obtain multiple first image feature maps. When performing the upsampling operation, bilinear interpolation or transposed convolution is used to increase the resolution of the current feature map to match the resolution of the feature map in the previous (shallower) layer. Bilinear interpolation is an upsampling method based on weighted averaging of surrounding pixel values and is suitable for maintaining the smoothness and details of the image. Transposed convolution increases the resolution of the feature map by learning weights and is often used to restore image details. The multiple first image feature maps are fused with the image feature maps with the same resolution in the multi-level image feature maps to obtain the multiple fusion points. Specifically: the upsampled first image feature map is horizontally concatenated with the image feature map from the corresponding level of EfficientNet (i.e., the same resolution). Usually, feature map concatenation is used, that is, the two feature maps are merged along the channel axis. After each upsampling and horizontal connection, a fusion point is generated, and each fusion point contains a fused image feature map.These image feature maps contain multi-scale information of the damage, providing a more comprehensive image understanding for subsequent damage detection.

[0049] At each fusion point, the third network layer in the model (e.g., SENet) adaptively adjusts the weights of the feature channels of the fused image feature map corresponding to each fusion point, highlighting important damage features while suppressing background or irrelevant feature information. Specifically: At each fusion point, first perform global average pooling (Global Average Pooling, abbreviated as GAP) on the fused image feature map, compressing the image feature map of each channel into a fixed-size vector, which contains the global information of the image feature map. Next, input the vector obtained by GAP into a fully connected layer, and through a series of non-linear transformations (e.g., through the rectified linear unit (Rectified Linear Unit, abbreviated as ReLU) and the sigmoid function), generate a weight vector. The length of this weight vector is equal to the number of channels of the image feature map, and each element corresponds to the weight of a feature channel. Multiply the weight vector obtained in the previous step with the fused feature map channel by channel to achieve adaptive weight adjustment of the feature channels. Channels with higher weights will be amplified, while channels with lower weights will be suppressed, thereby highlighting those features more relevant to damage detection. This adjustment improves the quality of the image feature map, enabling the detection model to more accurately locate and classify damage. Finally, based on the fused feature map, the detection model outputs the initial information of the damage, including the location, type, and segmentation mask of the damage. The segmentation mask is used to precisely define the shape and scope of the damage area. Specifically: Further process each fused feature map, for example, refine the fused feature map through a convolutional layer to generate more accurate damage candidate boxes. Use one or more convolutional layers and a region proposal network (Region Proposal Network, abbreviated as RPN) to generate damage candidate regions, that is, image regions that may contain damage. The RPN generates multiple candidate boxes and the confidence score of each damage candidate box by sliding a window on the fused feature map. Set a threshold and only retain the candidate boxes with a confidence score higher than this threshold. Calculate the intersection over union (Intersection over Union, abbreviated as IoU) between the retained candidate boxes pairwise. IoU is the ratio of the intersection area to the union area of two candidate boxes and is used to measure the overlap degree of the two boxes. Then, based on non-maximum suppression (Non-Maximum Suppression, abbreviated as NMS), remove redundant detection results, and finally obtain the detected damage location, type, and segmentation mask. Specifically: Select the candidate box with the highest confidence score from the candidate box list and add it to the final damage detection result. Delete all candidate boxes whose overlapping regions with other candidate boxes in the list exceed the preset intersection over union threshold. Repeat the above process for the remaining candidate boxes until all candidate boxes have been processed.After NMS processing, only one most accurate candidate box is retained for each damaged area, and this box contains the precise location and type information of the damage. Finally, combining the segmentation mask, the initial damage information is obtained, including the damage location (through the candidate box coordinates), the damage type, and the segmentation mask (used to precisely represent the damage range, which is a binary image with the same size as the original image, but only the damaged area is the foreground and the rest is the background).

[0050] Step S206: Determine the damage area and damage degree of the image data based on the initial damage information, and use the initial damage information, damage area, and damage degree as the detection results.

[0051] In the technical solution provided in step S206, there are various ways to determine the damage area and damage degree of the image data based on the initial damage information. For example: calculate the number of pixels at the damage location based on the segmentation mask in the initial damage information, and convert the number of pixels into a physical size to obtain the damage area; determine the damage degree according to the damage degree standard based on the damage type and damage area.

[0052] Step S208: Generate a visualization report based on the detection results and push the visualization report to the user terminal.

[0053] In the technical solution provided in step S208, there are various ways to generate a visualization report based on the detection results. For example: generate a damage distribution map based on the initial damage information in the detection results, where the damage distribution map is used to display the damage location and damage type; generate a damage report based on the detection results, where the damage report at least includes the damage area, damage degree, damage type, and damage location; use the damage distribution map and the damage report as the visualization report. The visualization report can intuitively display the damage situation, facilitate non-professionals to understand, and is suitable for the reporting and communication of building maintenance, improving the efficiency and accuracy of information transmission.

[0054] The following are specific embodiments:

[0055] First, according to the segmentation mask in the initial damage information, determine the number of pixels in the damaged area. The segmentation mask is a binary image with the same size as the original image, where the damaged area is the foreground (pixel value is 1) and the rest is the background (pixel value is 0). By calculating the number of pixels with a pixel value of 1 in the segmentation mask, the size of the damaged area can be obtained. Subsequently, convert the number of pixels into physical dimensions. Since the installation position and angle of the intelligent camera are known, the conversion coefficient between pixels and actual physical dimensions can be calculated through the internal parameters of the camera (such as focal length, image sensor size) and external parameters (such as installation height, distance from the building surface). Multiply the number of pixels in the damaged area by this conversion coefficient to obtain the physical dimension of the damaged area, such as square meters or square centimeters. According to the damage type and area, refer to the preset damage degree criteria to grade each damage. The preset damage degree criteria include damage types (such as cracks, spalling, corrosion), and the damage area thresholds for damage types. When the damage area threshold for a damage type is less than the first preset threshold, it is determined as mild damage; when the damage area threshold for a damage type is greater than or equal to the first preset threshold and less than or equal to the second preset threshold, it is determined as moderate damage; when the damage area threshold for a damage type is greater than the second preset threshold, it is determined as severe damage. For example, a crack area less than 1 square meter is defined as mild damage, 1 - 5 square meters as moderate damage, and greater than 5 square meters as severe damage. The damage grading criteria for spalling and corrosion may be different and are adjusted according to actual application requirements. Take the initial damage information, damage area, and damage degree as the detection results.

[0056] Based on the initial damage information (including damage location, type, and segmentation mask) in the detection results, draw the damaged area on the original image (for example, using the matplotlib plotting library to achieve). For different damage types, different colors or markers can be used to distinguish. For example, cracks are marked in red, spalling in yellow, and corrosion in green. To make the damage distribution map more intuitive, labels of damage types and coordinates of damage locations can also be added to the image to ensure that maintenance personnel can quickly locate and identify the damage. Additionally, a damage distribution map can be generated (for example, using the matplotlib library (a Python plotting library for data visualization) to generate a pie chart or bar chart), showing the proportion of different damage types in the entire detection results, as well as the damage locations and types of their respective damaged areas, to help quickly understand the overall distribution of damage. Generate a damage report based on the detection results (for example, using the ReportLab library (a Python library for generating PDF documents) to create a PDF-format damage report). In the report, list in detail the information of each damage, including damage location, type, area, and degree.

[0057] In each of the above steps, the detection model is obtained by training the initial detection model by inputting the structured data set stored in the database into the initial detection model. The structured data set is obtained in the following manner: sample image data of the building surface is collected at preset time intervals; the sample image data is labeled, and the labeled sample image data is stored in the database. Among them, the labeling content of the labeled sample image data includes the damage type and the damage location; all the labeled sample image data stored in the database is used as the structured data set. Through continuous training and optimization, the detection model can continuously learn and adapt to new damage patterns, improving the accuracy and adaptability of detection, and is suitable for the continuous optimization and upgrading of the building surface damage detection model.

[0058] The following are specific embodiments:

[0059] Set the automatic acquisition frequency of the camera to collect sample image data of the building surface at preset time intervals (for example, every hour) to ensure diverse image data is obtained under different lighting and weather conditions. To construct a high-quality structured data set, the following methods can be used for sample image data acquisition and processing: Automatically collect images during the day and at night, and under different weather conditions such as sunny, cloudy, and rainy days. Use image processing algorithms (such as histogram equalization, contrast enhancement, color correction, etc.) to optimize the image quality to ensure that the images are clearly visible under various lighting conditions. Label the collected sample image data, including the damage type (such as cracks, spalling, rust, etc.) and the specific location of the damage. The labeling of the sample image data is carried out in response to the labeling instructions of professionals, or semi-automatic or automatic labeling techniques can also be used, such as pre-labeling methods based on template matching or deep learning. Input the labeled sample image data into the database. Input the structured data set stored in the database into the initial detection model, and train the initial detection model (such as Mask Region-based Convolutional Neural Network, abbreviated as Mask R-CNN) until the detection model (i.e., the AI model generated based on HybridDamageNet) is obtained after reaching the preset number of iterations. In practical applications, for example, when complex damage types are detected, the detection model can automatically adjust parameters to further improve the detection accuracy. At the same time, the detection model is optimized based on the sample image data collected during each preset time period (for example, 3 months).

[0060] To provide forward-looking guidance for building maintenance, effectively prevent potential safety hazards, and reduce maintenance costs, the above detection model can also predict the damage risk information of image data at a preset future time. For example, the damage risk information of image data at a preset future time is predicted in the following manner: establish a damage database, where the damage database is used to store the detection results obtained each time; update the damage database after each new detection result is generated; obtain the damage data set in the damage database, where the damage data set consists of all the detection results in the damage database; predict the damage risk information of image data at a preset future time based on the damage data set, where the damage risk information includes at least the predicted loss location and the predicted loss type; generate an alarm message based on the damage risk and push it to the user terminal. The damage risk information also includes the prediction of the value of the risk indicator corresponding to the predicted loss type. Once the damage risk information indicates an abnormal situation (i.e., the predicted value of the risk indicator is greater than the preset threshold), an early warning message is immediately sent to the relevant personnel through the Internet of Things platform. For example, when the predicted value of the corresponding risk indicator in the damage risk information is the crack propagation speed on the building surface and the crack propagation speed on the building surface exceeds the preset threshold, the alarm mechanism will be automatically triggered, and the management personnel will be notified in a timely manner through the mobile phone APP or text message, etc.

[0061] Through the above steps, the damage condition of the building surface can be detected quickly and accurately, which is applicable to the regular inspection and maintenance of various buildings. Especially in the monitoring of large buildings, bridges, tunnels and other structures, the detection efficiency can be significantly improved, and the errors and workload of manual inspection can be reduced. An efficient, accurate and sustainable building surface damage detection solution is provided, comprehensively improving the level of building safety management.

[0062] The embodiment of the present application also provides a flowchart of another method for detecting building surface damage, as Figure 3 shown, including:

[0063] 1. Camera Deployment Module: Install high-definition cameras and connect them to the central processing unit. This part shows the deployment around the building. High-definition cameras are installed at key positions to cover all outer surfaces of the building. The cameras are connected to the central processing unit via wired or wireless networks to ensure that image data can be transmitted to the processing system in real time and stably. This is the data input end of the entire detection process and the basis for real-time monitoring and damage detection. 2. Data Acquisition Module: Collect images under different lighting conditions, annotate the images, and form a dataset (that is, collect sample image data of the building surface at preset time intervals, annotate the sample image data, and store the annotated sample image data in the database). This module is responsible for the automatic acquisition of image data, setting the acquisition frequency of the cameras, ensuring that images are obtained under different lighting and weather conditions, and increasing the diversity and representativeness of the data. The collected original images will undergo preprocessing, including denoising, contrast enhancement, and image correction, to improve the image quality and ensure that damage features can be clearly recognized by the detection model. 3. AI Model Training Module: Use HybridDamageNet to generate an AI model (that is, the above-mentioned detection model). 4. Loss Assessment Module: Use the data output by the AI model to form complete damage information (that is, use the detection model to determine the initial damage information of the image data, determine the damage area and damage degree of the image data based on the initial damage information, and use the initial damage information, damage area, and damage degree as the detection results). 5. Result Output Module: Generate visual results and reports (that is, generate a visual report based on the detection results and push the visual report to the user terminal).

[0064] The embodiment of the present application also provides a structural schematic diagram of a surface damage detection device, as Figure 4 shown, including:

[0065] An acquisition module 402, configured to acquire image data of the building surface.

[0066] A first determination module 404, configured to use a detection model to determine the initial damage information of the image data, where the damage information includes at least the damage position, damage type, and segmentation mask, and the segmentation mask is used to determine the damage area.

[0067] The first determination module 404 is further configured to input the image data into the detection model; extract multi-level image feature maps from the image data based on the first network layer in the detection model, where the first network layer is used to extract the image features of the image data, and the resolution of each layer of image feature maps decreases layer by layer; perform feature fusion on the image feature maps based on the second network layer in the detection model to obtain a plurality of fusion points, where the second network layer is the next network layer of the first network layer; adjust the weights of the feature channels at each fusion point based on the third network layer in the detection model to obtain a fused feature map; and determine the initial damage information based on the fused feature map.

[0068] The first determination module 404 is further configured to perform an upsampling operation on the multi-level image feature maps by the second network layer according to a top-down path to obtain a plurality of first image feature maps, where the upsampling operation includes at least bilinear interpolation or deconvolution, and the top-down path is from the image feature map corresponding to the layer with the lowest resolution to the image feature map corresponding to the layer with the highest resolution; perform feature fusion on the plurality of first image feature maps and the image feature maps with the same resolution in the multi-level image feature maps to obtain a plurality of fusion points, where each fusion point corresponds to a fused image feature map.

[0069] The second determination module 406 is configured to determine the damage area and damage degree of the image data based on the initial damage information, and use the initial damage information, the damage area, and the damage degree as the detection result.

[0070] The second determination module 406 is further configured to calculate the number of pixels at the damage position based on the segmentation mask in the initial damage information, and convert the number of pixels into a physical size to obtain the damage area; determine the damage degree according to the damage degree standard based on the damage type and the damage area, where different loss levels correspond to different damage degrees.

[0071] The push module 408 is configured to generate a visualization report based on the detection result and push the visualization report to the user terminal.

[0072] The push module 408 is further configured to generate a damage distribution map based on the initial damage information in the detection result, where the damage distribution map is used to display the damage position and the damage type; generate a damage report based on the detection result, where the damage report includes at least the damage area, the damage degree, the damage type, and the damage position; and use the damage distribution map and the damage report as the visualization report.

[0073] It should be noted that Figure 4 the detection device for building surface damage shown is used to execute Figure 2 the detection method for building surface damage shown, so Figure 2The relevant explanations in the method for detecting building surface damage also apply to the device for detecting building surface damage, which will not be elaborated here.

[0074] It should be noted that each module in the above-mentioned device for detecting building surface damage can be a program module (for example, a set of program instructions for implementing a specific function), or a hardware module. For the latter, it can be presented in the following forms, but not limited to this: the manifestation form of each of the above modules is a processor, or the functions of each of the above modules are implemented by a processor.

[0075] The embodiment of the present application also provides a non-volatile storage medium. The non-volatile storage medium includes a stored program. When the program runs, it controls the device where the non-volatile storage medium is located to execute the above method for detecting building surface damage. For example, obtaining image data of the building surface; using a detection model to determine the initial damage information of the image data, where the damage information at least includes the damage location, damage type, and segmentation mask, and the segmentation mask is used to determine the damage area; determining the damage area and damage degree of the image data based on the initial damage information, and taking the initial damage information, damage area, and damage degree as the detection result; generating a visualization report based on the detection result and pushing the visualization report to the user terminal.

[0076] The embodiment of the present application also provides an electronic device. The electronic device includes a processor, and the processor is used to run a program. When the program runs, it executes the above method for detecting building surface damage. For example, obtaining image data of the building surface; using a detection model to determine the initial damage information of the image data, where the damage information at least includes the damage location, damage type, and segmentation mask, and the segmentation mask is used to determine the damage area; determining the damage area and damage degree of the image data based on the initial damage information, and taking the initial damage information, damage area, and damage degree as the detection result; generating a visualization report based on the detection result and pushing the visualization report to the user terminal.

[0077] According to another aspect of the embodiment of the present application, there is also provided a computer program product, including a computer program, which implements the above method for detecting building surface damage when executed by a processor. For example, obtaining image data of the building surface; using a detection model to determine the initial damage information of the image data, where the damage information at least includes the damage location, damage type, and segmentation mask, and the segmentation mask is used to determine the damage area; determining the damage area and damage degree of the image data based on the initial damage information, and taking the initial damage information, damage area, and damage degree as the detection result; generating a visualization report based on the detection result and pushing the visualization report to the user terminal.

[0078] In the above-mentioned embodiments of the present application, the descriptions of the various embodiments each have their own emphases. For the parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0079] In several embodiments provided by the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are merely illustrative. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the units or modules can be in electrical or other forms.

[0080] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0081] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0082] If the above-mentioned integrated units are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the relevant technology, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. And the aforementioned storage medium includes: USB flash drives, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), mobile hard disks, magnetic disks or optical discs and other various media that can store program codes.

[0083] The above are only the preferred embodiments of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A method for detecting surface damage of a building, characterized in that, Including: Obtaining image data of the building surface; Using a detection model to determine initial damage information of the image data, wherein the damage information at least includes damage location, damage type, and a segmentation mask, and the segmentation mask is used to determine the damage area; Determining the damage area and damage degree of the image data based on the initial damage information, and taking the initial damage information, the damage area, and the damage degree as detection results; Generating a visualization report based on the detection results and pushing the visualization report to a user terminal.

2. The method according to claim 1, characterized in that Using a detection model to determine initial damage information of the image data includes: Inputting the image data into the detection model; Extracting multi-level image feature maps in the image data based on a first network layer in the detection model, wherein the resolution of each layer of image feature maps decreases layer by layer; Performing feature fusion on the image feature maps based on a second network layer in the detection model to obtain a plurality of fusion points, wherein the second network layer is the next network layer of the first network layer; Adjusting the weights of feature channels at each fusion point based on a third network layer in the detection model to obtain a fused feature map; Determining the initial damage information based on the fused feature map.

3. The method according to claim 2, wherein The performing feature fusion on the multi-level image feature maps based on the second network layer in the detection model to obtain a plurality of fusion points includes: The second network layer performs an upsampling operation on the multi-level image feature maps along a top-down path to obtain a plurality of first image feature maps, wherein the upsampling operation at least includes bilinear interpolation or transposed convolution, and the top-down path is from the image feature map corresponding to the lowest resolution layer to the image feature map corresponding to the highest resolution layer; Performing feature fusion on the plurality of first image feature maps and the image feature maps with the same resolution in the multi-level image feature maps to obtain the plurality of fusion points, wherein each fusion point corresponds to a fused image feature map.

4. The method according to claim 1, characterized in that, Determining the damage area and damage degree of the image data based on the initial damage information includes: Calculating the number of pixels at the damage location based on the segmentation mask in the initial damage information, and converting the number of pixels into a physical size to obtain the damage area; Determining the damage degree according to the damage degree standard based on the damage type and the damage area.

5. The method according to claim 1, wherein, Generating a visualization report based on the detection results includes: Generating a damage distribution map based on the initial damage information in the detection results, wherein the damage distribution map is used to display the damage location and damage type; Generating a damage report based on the detection results, wherein the damage report at least includes the damage area, the damage degree, the damage type, and the damage location; Taking the damage distribution map and the damage report as the visualization report.

6. The method according to claim 1, wherein The detection model is obtained by training an initial detection model by inputting a structured data set stored in a database into the initial detection model, and the structured data set is obtained through the following method: Collect sample image data of the building surface at preset time intervals; Annotate the sample image data and store the annotated sample image data in the database, wherein the annotation content of the annotated sample image data includes the damage type and the damage location; Use all the annotated sample image data stored in the database as the structured data set.

7. The method according to claim 1, characterized in that, The method further includes: Establish a damage database, where the damage database is used to store the detection results obtained each time; Update the damage database each time a new detection result is generated; Obtain a damage data set from the damage database, where the damage data set consists of all detection results in the damage database; Predict the damage risk information of the image data at a future preset time based on the damage data set, where the damage risk information at least includes the predicted loss location and the predicted loss type; Generate an alarm message based on the damage risk and push it to the user terminal.

8. A detection device for surface damage of a building, characterized in that, It includes: An acquisition module for acquiring image data of the building surface; A first determination module for using a detection model to determine the initial damage information of the image data, where the damage information at least includes the damage location, the damage type, and a segmentation mask, and the segmentation mask is used to determine the damage area; A second determination module for determining the damage area and the damage degree of the image data based on the initial damage information, and using the initial damage information, the damage area, and the damage degree as the detection result; A push module for generating a visualization report based on the detection result and pushing the visualization report to the user terminal.

9. A non-volatile storage medium, characterized in that, A program is stored in the non-volatile storage medium, and when the program runs, it controls the device where the non-volatile storage medium is located to execute the detection method for building surface damage according to any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes: A memory and a processor, and the processor is used to run the program stored in the memory, where when the program runs, it executes the detection method for building surface damage according to any one of claims 1 to 7.

11. A computer program product, comprising computer instructions, characterized in that, When the computer instruction is executed by the processor, it implements the detection method for building surface damage according to any one of claims 1 to 7.

Images