A method and device for unmanned aerial vehicle visual detection of a metal corrosion area of a building surface

Abstract

The application discloses a kind of building surface metal corrosion area unmanned aerial vehicle vision detection method and device, the method steps include: step S01. The detection model for identifying metal corrosion area in image is constructed, and detection model at least includes multi-scale occlusion perception feature extraction module and corrosion feature enhancement attention module;Step S02. The detection model constructed is trained, dynamic perception loss function is used during training to train, and the trained detection model is obtained after training is completed;Step S03. The image of the building surface to be measured is obtained by unmanned aerial vehicle with image acquisition equipment, and the detection image is input into the trained detection model, and the detection result of the building surface metal corrosion area is output.The application has the advantages of simple implementation method, high detection efficiency and precision, and strong robustness.

Description

Technical Field

[0001] This invention relates to the field of UAV visual inspection technology, and in particular to a UAV visual inspection method and device for metal corrosion areas on building surfaces. Background Technology

[0002] Metal corrosion on building surfaces is a significant factor affecting building safety. Traditional detection methods primarily rely on manual visual inspection, which suffers from drawbacks such as low efficiency, high risk, and strong subjectivity. With the development of UAV technology, corrosion detection based on UAV visual images is gradually being applied in engineering practice. Existing target detection methods typically employ detection networks such as Faster R-CNN and SSD. These networks lack sufficient detection accuracy in complex scenes. While improved models such as YOLOv5 and YOLOv8 can improve detection speed to some extent, their ability to detect small or occluded targets is insufficient. Although the YOLOv11 model can balance speed and accuracy, it struggles to handle changes in lighting, partial occlusion, and small corrosion areas, and is prone to missed detections when applied to metal corrosion areas.

[0003] Using existing target detection methods to detect metal corrosion areas on building surfaces presents the following problems:

[0004] 1. The surface environment of buildings is complex and variable. Metal components are often distributed at different heights and angles, and the lighting conditions vary greatly, resulting in unstable image quality. Traditional target detection algorithms are prone to false detections and false negatives in such complex backgrounds.

[0005] 2. Building surfaces are also prone to partial occlusion, such as trees and ancillary facilities obscuring metal components, which increases the difficulty of detecting corrosion areas. Traditional convolutional networks perform poorly in handling occlusion problems.

[0006] 3. Metal corrosion manifests in various forms, including pitting corrosion, surface corrosion, and crack corrosion. Different types of corrosion have significant differences in characteristics and a wide range of sizes. Existing single feature extraction methods are insufficient to fully capture all types of corrosion characteristics, which in turn affects the final detection accuracy. Summary of the Invention

[0007] The technical problem to be solved by the present invention is as follows: In view of the above-mentioned problems existing in the prior art, the present invention provides a UAV visual inspection method and device for metal corrosion areas on building surfaces that is simple to implement, has high detection efficiency and accuracy, and is robust. It can accurately detect the location, type and severity of metal corrosion areas on building surfaces, while improving the robustness of the detection.

[0008] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows:

[0009] A UAV visual inspection method for metal corrosion areas on building surfaces, comprising the following steps:

[0010] Step S01. Construct a detection model for identifying metal corrosion regions in an image. The detection model includes at least a multi-scale occlusion perception feature extraction module and a corrosion feature enhancement attention module. The multi-scale occlusion perception feature extraction module obtains an initial feature map extracted by a feature extraction network and performs adaptive convolution processing based on the feature density of the initial feature map to obtain a density-adapted feature map. The density-adapted feature map is then processed using spatial attention and channel attention mechanisms to perceive occlusion regions, reduce the feature values ​​of occlusion regions, and initially enhance corrosion features and corrosion boundary features to obtain an occlusion perception processed feature map. The corrosion feature enhancement attention module performs multi-level corrosion visual feature decomposition on the feature map output by the multi-scale occlusion perception feature extraction module and fuses the decomposed corrosion visual features to obtain an enhanced corrosion feature map.

[0011] Step S02. Train the constructed detection model. During the training process, a dynamic perceptual loss function is used for training. After the training is completed, the trained detection model is obtained. The dynamic perceptual loss function is to evaluate the severity of corrosion based on the state of each corrosion area and adjust the weight of the loss term according to the severity of corrosion.

[0012] Step S03. Obtain images of the building surface to be tested using a drone equipped with an image acquisition device. Input the images to be tested into the trained detection model and output the detection results of the metal corrosion area on the building surface. The detection results include the location of the corrosion area, corrosion type, and severity assessment information.

[0013] Furthermore, in the multi-scale occlusion perception feature extraction module, the density-adapted feature map is obtained by adaptive convolution processing based on the feature density of the initial feature map using the DSConv module. The steps include:

[0014] Calculate the local variance of the initial feature map to evaluate the feature density and generate a density map D;

[0015] The convolution operation is selected based on the value of the density map D, where the density value is higher than a first threshold. For dense regions, perform convolution operations using standard n×n convolutions, where the density value is below the second threshold. For sparse regions, n×n dilated convolution is used for convolution operations, with density values ​​between the second threshold. With the first threshold For the region between, standard n×n convolution is used for convolution operations, and n×n dilated convolution is used for convolution operations, and then the results are fused to obtain the final convolution operation result, where n represents the size of the convolution kernel.

[0016] Furthermore, the calculation expression for fusing the convolution operation performed using standard n×n convolution with the convolution operation performed using n×n dilated convolution to obtain the final convolution operation result is as follows:

[0017]

[0018] in, Represents the initial feature map. This represents the output feature map of the DSConv module. Represents a standard n×n convolution. This represents an n×n dilated convolution. , Indicates the expansion rate. The fusion coefficient, calculated based on density D, is expressed as follows: .

[0019] Furthermore, in the multi-scale occlusion perception feature extraction module, the OA module uses spatial attention and channel attention mechanisms to perceive occluded regions, reduce the feature values ​​of occluded regions, and initially enhance erosion features and erosion boundary features. The OA module includes a spatial attention branch, a channel attention branch, a local-global boundary enhancer, and a feature fusion unit. The spatial attention branch generates a spatial attention map based on the density-adapted feature map to assign weights to each pixel location to represent the probability of occlusion at the corresponding location. The channel attention branch obtains channel statistical features from the density-adapted feature map and then performs shared multi-channel attention. The layer perceptron generates channel weight vectors to assess the importance of different channels. The local-global boundary enhancer extracts local edge features by using depthwise separable convolution on the density-adapted feature map and obtains global context information through dilated convolution. The results of the depthwise separable convolution and the dilated convolution are fused to form fused edge features to enhance the erosion boundary features. The feature fusion unit is used to combine the density-adapted feature map, the spatial attention map output by the spatial attention branch, the channel weight vector output by the channel attention branch, and the fused edge features output by the local-global boundary enhancer to obtain the occlusion-aware feature map.

[0020] Furthermore, the calculation expression for the feature map after occlusion perception processing is as follows:

[0021]

[0022] in, This is the feature map after occlusion perception processing output by the OA module. This is the input feature map of the OA module, i.e., the density-adapted feature map output by the DSConv module. The spatial attention graph output by the spatial attention branch. By analyzing the input feature map It is generated using two n×n convolutional layers and the Sigmoid activation function. The channel weight vector is the output of the channel attention branch. By analyzing the input feature map Channel statistical features are obtained using global average pooling and max pooling, and generated using a shared multilayer perceptron. The fused edge features are output by the local-global boundary enhancer.

[0023] Furthermore, the multi-scale occlusion perception feature extraction module also includes an AFE module for adaptive feature enhancement of the feature map after occlusion perception processing. The AFE module includes a multi-scale feature extraction unit, a scale attention mechanism, and an adaptive feature fusion unit. The multi-scale feature extraction unit extracts feature information of different scales in the feature map after occlusion perception processing through parallel convolutional branches of different scales. The scale attention mechanism generates weight coefficients for each scale branch. The adaptive feature fusion unit performs weighted fusion of the multi-scale features based on the attention weights to obtain the feature map after feature enhancement.

[0024] Furthermore, the corrosion feature enhancement attention module includes a multi-level corrosion feature decomposer, a context fusion enhancer, and a cross-layer feature aggregation unit. The multi-level corrosion feature decomposer decomposes the feature map output by the multi-scale occlusion perception feature extraction module into multiple visual feature channels to achieve corrosion visual feature filtering. The visual feature channels include color anomaly channels, texture change channels, and structural defect channels. The features of each visual feature channel are fused through a self-attention mechanism to generate a comprehensive corrosion feature representation. The context fusion enhancer calculates the correlation between each position and all other positions in the comprehensive corrosion feature representation output by the multi-level corrosion feature decomposer using a self-attention mechanism, generates an attention map, and performs weighted aggregation based on the attention map to enhance the contrast between the corrosion region and the background. The cross-layer feature aggregation unit performs cross-layer aggregation on the feature map output by the context fusion enhancer using a feature pyramid network structure and adjusts the weights of features at different levels according to the size characteristics of the corrosion region, finally obtaining the enhanced corrosion feature map output.

[0025] Furthermore, the calculation expression for the dynamic sensing loss function is as follows:

[0026]

[0027]

[0028]

[0029]

[0030] in, This represents the total dynamic sensing loss. Represents classification loss. Indicates location loss. Indicates confidence loss. Indicates boundary loss. , as well as These are the weighting coefficients for each loss term. , as well as These are parameters adjusted to control the sensitivity of each loss term's weight to severity. The corrosion severity score is calculated based on the size, density, and texture complexity of the corroded area.

[0031] Furthermore, the detection model also includes an image preprocessing module before the scale occlusion perception feature extraction module. The image preprocessing module includes an image quality assessment unit for screening images that meet quality requirements and an illumination compensation unit for performing illumination compensation. The illumination compensation unit performs multi-level decomposition of the input image using wavelet transform to suppress noise in high-frequency components and performs brightness equalization processing to adjust the image brightness distribution.

[0032] A drone-based visual inspection device for metal corrosion areas on building surfaces includes a processor and a memory. The memory stores a computer program, and the processor executes the computer program to perform the method described above.

[0033] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention constructs a detection model, which includes a multi-scale occlusion perception feature extraction module and a corrosion feature enhancement attention module. The multi-scale occlusion perception feature extraction module can be used to comprehensively extract features in complex building surface environments. The corrosion feature enhancement attention module enhances the visual features of corrosion, which can significantly improve the recognition ability of various corrosion types and improve the accuracy and robustness of detecting metal corrosion areas on building surfaces with occlusion and small targets in complex building environments. At the same time, during the model training process, the dynamic perception loss function can dynamically adjust the weights according to the corrosion type and severity through corrosion severity assessment and adaptive weight allocation mechanism, thereby improving the sensitivity to severely corroded areas and improving the detection accuracy and robustness of the model. It can achieve accurate localization and classification of corrosion areas on the surface of metal structures such as building exterior walls, bridges, and towers. Attached Figure Description

[0034] Figure 1 This is a schematic diagram illustrating the implementation process of the UAV visual inspection method for metal corrosion areas on building surfaces in this embodiment.

[0035] Figure 2 This is a schematic diagram of the structural principle of the DSConv module in a specific application embodiment of the present invention.

[0036] Figure 3 This is a schematic diagram of the structural principle of the OA module in a specific application embodiment of the present invention.

[0037] Figure 4 This is a schematic diagram of the structural principle of the AFE module in a specific application embodiment of the present invention.

[0038] Figure 5 This is a schematic diagram of the structural principle of the CRA module in a specific application embodiment of the present invention.

[0039] Figure 6 This is a schematic diagram of the structural principle of the illumination compensation FM unit in a specific application embodiment of the present invention.

[0040] Figure 7 This is a schematic diagram illustrating the implementation process of the dynamic sensing loss function in a specific application embodiment of the present invention.

[0041] Figure 8 This is a schematic diagram of the overall network architecture principle built based on the YOLOv11 network in a specific application embodiment of the present invention. Detailed Implementation

[0042] The present invention will be further described below with reference to the accompanying drawings and specific preferred embodiments, but this does not limit the scope of protection of the present invention.

[0043] As disclosed in this invention, unless the context clearly indicates otherwise, words such as "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. The terms "first," "second," and similar terms used in this invention disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, words such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connected" or "linked" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0044] like Figure 1 As shown, the steps of the UAV visual inspection method for metal corrosion areas on building surfaces in this embodiment include:

[0045] Step S01. Construct a detection model for identifying metal corrosion regions in an image. The detection model includes at least a multi-scale occlusion perception feature extraction module and a corrosion feature enhancement attention module. The multi-scale occlusion perception feature extraction module obtains an initial feature map extracted by a feature extraction network and performs adaptive convolution processing based on the feature density of the initial feature map to obtain a density-adapted feature map. The density-adapted feature map is then processed using spatial attention and channel attention mechanisms to perceive occlusion regions, reduce the feature values ​​of occlusion regions, and initially enhance corrosion features and corrosion boundary features, resulting in an occlusion perception-processed feature map. The corrosion feature enhancement attention module performs multi-level corrosion visual feature decomposition on the feature map output by the multi-scale occlusion perception feature extraction module and fuses the decomposed corrosion visual features to obtain an enhanced corrosion feature map.

[0046] Step S02. Train the constructed detection model. During the training process, the dynamic perceptual loss function is used for training. After the training is completed, the trained detection model is obtained. The dynamic perceptual loss function (CSDL) assesses the severity of corrosion based on the state of each corrosion region and adjusts the relative weights of classification loss, localization loss and confidence loss according to the severity of corrosion.

[0047] Step S03. Obtain images of the building surface to be tested using a drone equipped with an image acquisition device. Input the images to be tested into the trained detection model and output the detection results of the metal corrosion area on the building surface. The detection results include the location of the corrosion area, the corrosion type, and the severity assessment information.

[0048] This embodiment constructs a detection model, which includes a multi-scale occlusion perception feature extraction module and a corrosion feature enhancement attention module. The multi-scale occlusion perception feature extraction module is applicable to complex building surface environments for comprehensive feature extraction. The corrosion feature enhancement attention module enhances the visual features of corrosion, significantly improving the recognition ability of various corrosion types. Combining the multi-scale occlusion perception feature extraction network and the corrosion feature enhancement attention module effectively improves the accuracy and robustness of detecting occluded and small-target metal corrosion areas on building surfaces in complex architectural environments. Furthermore, during model training, the dynamic perception loss function, through corrosion severity assessment and adaptive weight allocation mechanisms, dynamically adjusts weights according to corrosion type and severity, enhancing sensitivity to severely corroded areas and thus improving the model's detection accuracy and robustness. This embodiment can achieve precise localization and classification of corrosion areas on the surfaces of metal structures such as building exterior walls, bridges, and towers, improving the detection accuracy and robustness of metal corrosion areas on building surfaces, solving problems such as occlusion and diverse corrosion morphologies in building surface metal corrosion detection, and achieving accurate detection of corrosion areas at different scales.

[0049] In this embodiment, YOLOv11 can be used as the backbone network, combined with multi-scale feature extraction and occlusion perception mechanism to form an improved target detection network architecture, and a detection model can be constructed. Multi-scale occlusion perception feature extraction module is used to extract multi-scale occlusion perception features to achieve accurate detection of corrosion areas at different scales.

[0050] In this embodiment, the multi-scale occlusion perception feature extraction module uses the DSConv (Density Sensitive Convolution) module to perform adaptive convolution processing based on the feature density of the initial feature map to obtain a density-adapted feature map, thus applying adaptive convolution processing to erosion features of different densities. The initial feature map is the feature map extracted by the backbone network. For example... Figure 2 As shown, the steps of the DSConv module to obtain a density-adapted feature map by adaptive convolution based on the feature density of the initial feature map include:

[0051] Calculate the local variance of the initial feature map to evaluate the feature density and generate a density map D;

[0052] The convolution operation is selected based on the value of the density map D, where the density value is higher than a first threshold. For dense regions (such as surface erosion), perform a standard n×n convolution operation with a density value below the second threshold. For sparse regions (such as pitting), use n×n dilation convolution for convolution operation, with density values ​​between the second threshold. With the first threshold For the region between, standard n×n convolution is used for convolution operations, and n×n dilated convolution is used for convolution operations, and then the results are fused to obtain the final convolution operation result, where n represents the size of the convolution kernel.

[0053] This embodiment, through the aforementioned DSConv module, overcomes the limitation of the fixed receptive field in traditional convolutional networks. It can adaptively adjust the receptive field according to the density of corrosion features, that is, it adaptively adjusts the receptive field for different corrosion morphologies. Standard convolution is used for dense surface corrosion regions, while dilated convolution is used for sparse pitting corrosion regions, achieving a smooth transition between the two. This improves the ability to capture various corrosion features, thereby more accurately capturing corrosion features at different scales. It can solve the problem that the difficulty of recognition varies due to the different feature distribution densities of different corrosion types (such as pitting corrosion, surface corrosion, etc.).

[0054] In this embodiment, the calculation expression for the final convolution result obtained by fusing the convolution operation performed using standard n×n convolution and the convolution operation performed using n×n dilated convolution is as follows:

[0055] (1)

[0056] in, Represents the initial feature map. This represents the output feature map. Represents a standard n×n convolution. This represents an n×n dilated convolution. , Indicates the expansion rate. The fusion coefficient, calculated based on density D, is expressed as follows: .

[0057] In a specific application example, taking a 3×3 convolution kernel as an example, the DSConv module is implemented as follows:

[0058] Feature density estimation: The feature density is evaluated by calculating the local variance of the feature map. The standard deviation of the feature values ​​in the 3×3 neighborhood is calculated to generate the density map D.

[0059] Adaptive convolution selection: Automatically selects appropriate convolution operations based on the value of density map D: for density values ​​higher than a threshold For dense regions, use standard 3×3 convolution; for density values ​​below a threshold... For sparse regions, such as pitting, use a 3×3 dilated convolution (dilation rate r=2); for density values ​​between and The region between them is linearly fused with the two convolution results, mathematically expressed as: .

[0060] In this embodiment, the multi-scale occlusion perception feature extraction module uses a spatial attention mechanism and channel attention through the OA module to perceive occluded areas, reduce the feature values ​​of occluded areas, and initially enhance erosion features and erosion boundary features. The OA module includes a spatial attention branch, a channel attention branch, a local-global boundary enhancer, and a feature fusion unit. The spatial attention branch generates a spatial attention map based on the density-adapted feature map to assign weights to each pixel position to represent the probability of occlusion at the corresponding position. The channel attention branch obtains channel statistical features from the density-adapted feature map and generates channel weight vectors through a shared multilayer perceptron to evaluate the importance of different channels. The local-global boundary enhancer extracts local edge features by using depthwise separable convolution on the density-adapted feature map and obtains global context information through dilated convolution. The results of depthwise separable convolution and dilated convolution are fused to form fused edge features to enhance erosion boundary features. The feature fusion unit is used to synthesize the density-adapted feature map, the spatial attention map output by the spatial attention branch, the channel weight vector output by the channel attention branch, and the fused edge features output by the local-global boundary enhancer to obtain the occlusion perception processed feature map.

[0061] In this embodiment, the OA module detects occluded areas through spatial attention branches, enhances feature representation through channel attention branches, and highlights corrosion edge features by combining local-global boundary enhancers. This can effectively address the detection difficulties caused by occlusions such as vegetation and dust, improve the detection robustness under partial occlusion conditions, and thus solve the problem of metal corrosion areas on building surfaces being occluded by vegetation, dust, etc.

[0062] In this embodiment, the spatial attention branch can generate a spatial attention map through two layers of n×n convolution and a sigmoid activation function, while the channel attention branch obtains channel statistical features through global average pooling and max pooling, respectively.

[0063] Taking a 3×3 convolution kernel as an example, such as Figure 3 As shown, the OA module mainly includes the following three components:

[0064] (1) Spatial Attention Branch: A spatial attention map is generated using two 3×3 convolutional layers and a sigmoid activation function. Each pixel location is assigned a weight between 0 and 1, representing the probability of occlusion at that location. Spatial Attention Map By analyzing the input feature map It is generated using two n×n convolutional layers and the Sigmoid activation function, and the calculation formula can be expressed as: ,in σ represents the input feature, and σ is the Sigmoid function.

[0065] (2) Channel Attention Branch: First, channel statistical features are obtained through global average pooling and max pooling, respectively. Then, a shared multilayer perceptron is used to generate channel weight vectors, thereby evaluating the importance of different channels. Channel Weight Vector By analyzing the input feature map Channel statistical features are obtained using global average pooling and max pooling, and generated using a shared multilayer perceptron. The calculation formula can be expressed as: ,in and These represent global average pooling and global max pooling, respectively. It is a multilayer perceptron.

[0066] (3) Local-Global Boundary Enhancer: This function uses depthwise separable convolution to extract local edge features and dilated convolution to obtain global contextual information. The two are then fused to enhance the erosion boundary features. The calculation formula can be expressed as:

[0067] Where DSConv represents depthwise separable convolution, and DilatedConv represents dilated convolution. , where r is the expansion rate.

[0068] The final output of the OA module can be calculated in the following way: ,in This is the feature map after occlusion perception processing output by the OA module. This is the input feature map of the OA module, i.e., the density-adapted feature map output by the DSConv module. The spatial attention map output by the spatial attention branch. The channel weight vector is the output of the channel attention branch. This represents the fused edge features output by the local-global boundary enhancer.

[0069] In this embodiment, the feature map processed by the OA module can effectively reduce the interference of occluded areas, while enhancing the expression of erosion edges, improving the robustness of the network in complex environments, solving problems such as complex textures on building surfaces, changes in lighting, and partial occlusion, and improving the robustness and anti-interference ability of detection.

[0070] In this embodiment, the multi-scale occlusion perception feature extraction module further includes an AFE module for adaptive feature enhancement of the feature map after occlusion perception processing. The AFE module includes a multi-scale feature extraction unit, a scale attention mechanism, and an adaptive feature fusion unit. The multi-scale feature extraction unit extracts feature information at different scales through parallel convolutional branches of different scales. The scale attention mechanism generates weight coefficients for each scale branch. The adaptive feature fusion unit performs weighted fusion of the multi-scale features based on the attention weights to obtain the feature-enhanced feature map. By employing the above-mentioned AFE module, the feature representation is improved through multi-scale feature extraction and adaptive weight fusion, which enhances the network's ability to detect eroded regions of different sizes, especially the ability to identify tiny eroded points.

[0071] In specific application embodiments, such as Figure 4 As shown, the AFE module can be configured to include the following components:

[0072] (1) Multi-scale feature extraction unit: extracts feature information of different scales from the feature map after occlusion perception processing output by the OA module through parallel 1×1, 3×3, and 5×5 convolution branches. Among them, the 3×3 and 5×5 convolutions adopt depth-separable convolution to reduce the amount of computation.

[0073] (2) Scale attention mechanism: Weight coefficients are generated for each scale branch, enabling the network to adaptively focus on the most relevant scale features. The attention weights are generated by a 1×1 convolution followed by a Softmax function, ensuring that the sum of the weights of the three branches is 1.

[0074] (3) Adaptive Feature Fusion Unit: This unit performs weighted fusion of multi-scale features based on attention weights. For example, the calculation formula is:

[0075] ,in This represents the attention weight for the corresponding branch. This indicates the output characteristics of each branch.

[0076] This embodiment uses the AFE module to adaptively fuse multi-scale features, which can significantly improve the network's ability to detect corrosion areas of different sizes, especially its sensitivity to small corrosion points. This enables the network to avoid missing tiny corrosion points and effectively cover the features of large-area corrosion.

[0077] In this embodiment, the Corrosion Feature Enhancement Attention (CRA) module in the target detection layer specifically includes a multi-level corrosion feature decomposer, a context fusion enhancer, and a cross-layer feature aggregation unit. The multi-level corrosion feature decomposer decomposes the feature map output by the multi-scale occlusion perception feature extraction module into multiple visual feature channels. These visual feature channels include various corrosion-related visual feature channels such as color anomaly channels, texture change channels, and structural defect channels, to achieve corrosion visual feature filtering, retaining only corrosion-related visual features (such as rust color, pitting texture, etc.) and removing useless features such as building background. The features of each visual feature channel are fused through a self-attention mechanism to generate a comprehensive corrosion feature representation. The context fusion enhancer calculates the correlation between each position in the feature map and all other positions using a self-attention mechanism to generate an attention map, and performs weighted aggregation based on the attention map to enhance the contrast between the corrosion area and the background. The cross-layer feature aggregation unit performs cross-layer aggregation on the feature map output by the context fusion enhancer using a feature pyramid network structure, and adjusts the weights of features at different levels according to the size characteristics of the corrosion area, finally obtaining the enhanced corrosion feature map output.

[0078] Unlike traditional general attention mechanisms, this embodiment optimizes the visual features of metal corrosion by setting up a corrosion feature enhancement attention module. It extracts corrosion-related features such as color anomalies, texture changes, and structural defects through a multi-level feature decomposer, improving the ability to identify multiple corrosion types. Combined with a context fusion enhancer, it captures the contrast information between the corroded area and the surrounding healthy area through a self-attention mechanism, enhancing the discriminativeness of corrosion features. Then, it achieves cross-layer feature aggregation by establishing feature connections between different detection levels and integrating multi-scale corrosion information, which can improve the detection accuracy of corrosion areas of different sizes. Thus, it can significantly improve the recognition accuracy of corrosion areas in complex backgrounds.

[0079] In specific application embodiments, such as Figure 5 As shown, the CRA module can be configured as follows:

[0080] (1) Multi-level corrosion feature decomposer: used to decompose the feature map output by the multi-scale occlusion perception feature extraction module into three visual feature channels, focusing on different visual features of corrosion respectively. The visual feature channels are as follows:

[0081] Color anomaly channel: Enhanced sensitivity to color changes caused by erosion through 1×1 convolution and channel attention mechanism;

[0082] Texture variation channel: Employs 3×3 depth separable convolution and Gabor filter bank to extract texture features of the corrosion area, effectively identifying different texture modes such as rust and pitting.

[0083] Structural Defect Channel: Uses 5×5 dilated convolution (dilation rate r=2) to capture a large range of structural information and identify structural changes and defects caused by corrosion.

[0084] The features of the three channels mentioned above are fused through a self-attention mechanism to generate a comprehensive erosion feature representation.

[0085] (2) Context fusion enhancer: used to capture the contrast information between the corroded area and the surrounding healthy area through a self-attention mechanism. Formula (3) directly reflects and enhances the contrast information between the two. It enhances the discriminativeness of corrosion features. The self-attention mechanism calculates the correlation between each position and all other positions in the comprehensive corrosion feature representation output by the multi-level corrosion feature decomposer, and generates an attention map. The calculation formula is:

[0086] (2)

[0087] Where Q and K represent the query and key-value transformation, respectively, and d is the feature dimension. The comprehensive corrosion characteristics are represented by the first... Feature vectors at each position The comprehensive corrosion characteristics are represented by the first... Feature vectors at each position The temperature parameter is used to adjust the smoothness of the correlation distribution and reduce the impact of extreme weights on the results.

[0088] Then, contextual information fusion is performed: weighted aggregation is performed based on the attention map to enhance the contrast between the eroded region and the background. The calculation expression is as follows:

[0089] (3)

[0090] in V For value transformation, The first output of the context fusion enhancer The feature map of each position, where j represents the index of all positions in the eroded region. N This represents the total number of locations in the feature map.

[0091] (3) Cross-layer feature aggregation unit: In order to make full use of multi-scale information, the CRA module sets up a cross-layer feature aggregation unit to realize feature connection between different detection levels. The cross-layer feature aggregation unit can be specifically configured as follows:

[0092] The Feature Pyramid Network (FPN) structure transmits semantically strong features via a top-down path and detailed features via a bottom-up path. This FPN structure aggregates the feature maps output by the context fusion enhancer across layers, allowing for complementary fusion of features selected by a multi-level erosion feature decomposer and enhanced by the context fusion enhancer, resulting in more accurate localization.

[0093] Scale-adaptive aggregation: Based on the size characteristics of the eroded area, the weights of features at different levels are dynamically adjusted. That is, in the cross-layer aggregation process, the features are fused according to scale (large erosion / small erosion). Small target erosion points are given higher weights to lower-level features, while large eroded areas are given more attention to higher-level features.

[0094] Residual connections: Introducing residual connections into feature fusion ensures stable gradient propagation and improves training efficiency.

[0095] Metal corrosion is characterized by "large size range and features easily confused with the background". This embodiment further sets up a CRA module, which can comprehensively capture multi-scale and cross-level basic features through multi-level feature decomposition and cross-layer aggregation, ensuring that feature information of different sizes and levels is not missed. At the same time, it accurately aggregates corrosion-specific features, focusing on corrosion-related details and semantics on the captured basic features, and enhances the discriminativeness of corrosion features. Furthermore, by fusing cross-level corrosion-specific features and dynamically matching size weights, corrosion features can be made more prominent and more recognizable, which can significantly improve the ability to identify different types of corrosion, especially the corrosion detection performance in complex backgrounds.

[0096] In this embodiment, an image preprocessing module is also provided in the detection model before the scale occlusion perception feature extraction module. The image preprocessing module includes an image quality assessment unit for screening images that meet the quality requirements and an illumination compensation (FM) unit for performing illumination compensation. The illumination compensation unit performs multi-level decomposition of the input image by using wavelet transform to suppress noise in high-frequency components and performs brightness equalization processing to adjust the image brightness distribution, thereby reducing the impact of outdoor ambient light changes on the detection results.

[0097] Specifically, such as Figure 6 As shown, the illumination compensation unit can be configured as follows:

[0098] High-frequency suppression filtering: Wavelet transform is used to decompose the input image into multiple levels to suppress noise in high-frequency components and retain effective structural information.

[0099] Brightness equalization processing: Adaptive histogram equalization technology is used to adjust the image brightness distribution and reduce the impact of uneven lighting. The processing procedure is as follows:

[0100] The image is divided into multiple 8×8 small regions;

[0101] Apply histogram balancing to each region;

[0102] The results are merged using bilinear interpolation to avoid boundary artifacts.

[0103] Contrast Adjustment: Adaptively adjusts contrast based on image statistical characteristics to enhance the distinction between eroded areas and the background.

[0104] The image processed by the illumination compensation unit in this embodiment has a more consistent brightness distribution and clearer structural information, providing a more stable input for subsequent feature extraction.

[0105] To address the insufficient sensitivity of traditional fixed-weight loss functions in metal corrosion detection, this embodiment employs a Dynamic Perceptual Loss Function (CSDL) during model training. This function integrates corrosion severity assessment into the loss calculation, dynamically evaluating the severity of each corrosion region based on its size, density, and texture complexity. An adaptive weight allocation mechanism is then implemented to dynamically adjust the relative weights of various loss terms (specifically, classification loss, localization loss, and confidence loss, etc.), allowing the network training process to give greater attention to severely corroded areas. Simultaneously, considering the blurred boundaries of corrosion regions, a boundary refinement loss is introduced to improve the localization accuracy of corrosion region boundaries. This dynamic perception mechanism enables the network to more effectively learn corrosion characteristics of different severity levels, significantly improving the detection accuracy and boundary localization capability of high-risk corrosion regions, and providing a more reliable quantitative basis for building safety assessment.

[0106] By employing multi-dimensional feature analysis and adaptive weight allocation, the network achieves higher detection sensitivity for high-risk corrosion areas. A corrosion severity evaluator is constructed to dynamically assess the severity of corrosion based on the size, density, and texture complexity of the corrosion area. Adaptive weight allocation is implemented to dynamically adjust the weights of classification loss, localization loss, and confidence loss according to the corrosion severity, enabling the network to pay more attention to severely corroded areas.

[0107] In specific application embodiments, such as Figure 7 As shown, the dynamic perception loss function can be implemented using the following steps:

[0108] (1) Corrosion Severity Assessor: Used to assess the severity of each corroded area. The steps include:

[0109] Region feature extraction: Three key indicators are extracted: size, density, and texture complexity of the eroded region. The size indicator is the normalized area of ​​the eroded region, the density indicator is the standard deviation of the pixel values ​​within the eroded region, and the texture complexity is calculated using Local Binary Pattern (LBP) features.

[0110] Severity Score: Based on the above characteristics, the corrosion severity score S is calculated using a weighted summation method.

[0111] (4)

[0112] in Assigning weights to each indicator. The scores for each indicator are normalized.

[0113] (2) Adaptive weight allocation: used to dynamically adjust the weights of each component of the loss function based on the corrosion severity score S. Specifically: classification loss weights: Location loss weights: Confidence loss weights: ,in , and To adjust the parameters and control the sensitivity of each loss term's weight to severity, the total loss function is calculated as follows:

[0114] (5)

[0115] in, This represents the total dynamic sensing loss. Represents classification loss. Indicates location loss. Indicates confidence loss. Indicates boundary loss, , as well as These are the weighting coefficients for each loss term. , as well as These are parameters adjusted to control the sensitivity of each loss term's weight to severity. The corrosion severity score is calculated based on the size, density, and texture complexity of the corroded area.

[0116] By using a dynamic weight allocation mechanism, the network pays more attention to severely corroded areas, which can improve the detection accuracy of high-risk areas.

[0117] (3) Boundary refinement loss

[0118] To address the issue of blurred boundaries in corroded regions, a boundary refinement loss is introduced, involving the following steps:

[0119] Boundary extraction: Use the Sobel operator to extract the boundary mask of the true annotation;

[0120] Distance Transformation: Calculates the distance map from the predicted region to the true boundary;

[0121] Boundary loss calculation:

[0122] (6)

[0123] in and These are the predicted and the actual masks, respectively. This is a distance map. This is the attenuation parameter.

[0124] This embodiment addresses the issue of ambiguous boundaries in corrosion regions by introducing a boundary refinement loss. The boundary is extracted using the Sobel operator and combined with distance transformation calculation, making the network more focused on the boundary accuracy of the corrosion region. This enables accurate definition of the corrosion range and effectively improves the positioning accuracy of the corrosion region boundary.

[0125] To meet the deployment requirements of drone platforms and improve the real-time performance of drone edge computing platforms, this embodiment further adopts the following lightweight strategy during the training process:

[0126] (1) Adaptive channel pruning is adopted to dynamically adjust the network structure according to the importance of channels, remove redundant channels, and reduce the amount of computation.

[0127] (2) A gradient flow control mechanism is adopted to control the learning rate of different channels through gradient modulation parameters in order to balance network performance and computational complexity.

[0128] (3) Adopt a cross-layer feature reuse strategy to reduce redundant calculations, reduce memory usage, and improve inference speed through feature reuse.

[0129] Specifically, the steps of adaptive channel pruning include:

[0130] Channel importance assessment: The importance of each channel is assessed based on the absolute value of the scaling factor γ of the batch normalization layer.

[0131] Importance threshold determination: K-means clustering is used to classify channels into two categories: important and unimportant.

[0132] Dynamic pruning: Pruning operations are performed periodically during training to remove channels with importance below a threshold, and then the network is fine-tuned to restore performance.

[0133] Pruning stop condition: Pruning stops when performance degrades by more than a preset threshold (1%).

[0134] The steps of the gradient flow control mechanism include:

[0135] Gradient modulation parameter: Introduce channel-level gradient modulation parameter β to control the learning rate of different channels.

[0136] Importance-driven adjustment: Important channels are assigned larger β values, while unimportant channels are assigned smaller β values.

[0137] Dynamic update strategy: During training, the β value is updated periodically based on the impact of channels on the loss function.

[0138] This mechanism ensures that network resources are concentrated on extracting the most valuable features, thus optimizing computational efficiency.

[0139] The steps for cross-layer feature reuse include:

[0140] Feature reuse path: Establish cross-layer connections from low layer to high layer, allowing high layer to directly reuse features processed by low layer.

[0141] Selective feature propagation: The amount of feature reuse is controlled through a gating mechanism to avoid the propagation of redundant information.

[0142] Shared convolution kernels: Convolution kernel parameters are shared between layers with similar functions, reducing the number of model parameters.

[0143] This embodiment uses an improved YOLOv11 network framework as the core detection architecture. The feature extraction network C3k2_ODAF consists of C3k2, a multi-scale occlusion-aware feature extraction module, and a corrosion feature enhancement attention module (CRA module). The multi-scale occlusion-aware feature extraction module includes the aforementioned DSConv module, OA module, and AFE module. The CRA module includes the aforementioned multi-level corrosion feature decomposer, context fusion enhancer, and cross-layer feature aggregation unit. Simultaneously, an illumination compensation unit (FM) is introduced during image preprocessing to construct the YOLO-ODC network. A dynamic perceptual loss function is used during the training phase. Using the trained modules, high-precision and robust detection of metal corrosion on building surfaces can be achieved. The overall network architecture is as follows: Figure 8 As shown.

[0144] In a specific application embodiment, a quadcopter drone can be used as the carrier, equipped with a high-definition RGB camera (4K resolution) and a thermal imager, with a three-axis gimbal ensuring image stability. The drone is equipped with an edge computing unit, supporting real-time preprocessing and preliminary analysis of some data, while simultaneously transmitting data to a ground workstation for in-depth processing via 4G / 5G networks. During the data acquisition process, the drone scans the building surface along a preset flight path, maintaining a distance of 2-5 meters from the surface, acquiring images with an 80% overlap rate. The acquired images can undergo the following preprocessing steps:

[0145] Image quality assessment: Automatically filters high-quality images based on image sharpness, brightness, and contrast scores;

[0146] Illumination compensation: High-frequency suppression filtering and brightness equalization processing are performed through the FM unit to reduce the illumination differences in images acquired at different times;

[0147] Image registration: Register the RGB image with the thermal image to establish a pixel-level correspondence;

[0148] Data augmentation: The training dataset is augmented using rotation, scaling, and flipping methods.

[0149] This embodiment also includes a multi-dimensional corrosion state discrimination standard in the output of detection results to ensure the reliability of the assessment. Specifically, the following key indicators are obtained based on the output indicators to classify the degree of corrosion: (1) Corrosion area ratio, calculated by the ratio of the corrosion area to the surface area of ​​the detection target; (2) Corrosion depth parameter, estimated using image texture features and light reflection characteristics; (3) Corrosion type identification result, distinguishing different forms such as pitting corrosion, crevice corrosion, and stress corrosion; (4) Corrosion area distribution density, assessing the concentration or dispersion of corrosion.

[0150] The aforementioned indicators are further evaluated using feature parameters extracted by the CRA module, classifying the corrosion status into four levels: Level I (mild, <10% area, shallow corrosion); Level II (moderate, 10%-30% area, surface peeling); Level III (severe, 30%-50% area, significant penetration); and Level IV (dangerous, >50% area, structural integrity compromised). The detection results can be marked with colored bounding boxes and accompanied by a corrosion degree heatmap to visually represent the spatial distribution, providing precise location and priority recommendations for building maintenance.

[0151] This embodiment further provides a UAV visual inspection device for metal corrosion areas on building surfaces, including a processor and a memory. The memory is used to store computer programs, and the processor is used to execute the computer programs to perform the methods described above.

[0152] Those skilled in the art will understand that the above embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce implementations of the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0153] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Therefore, any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention should fall within the protection scope of the present invention.

Claims

1. A UAV visual inspection method for metal corrosion areas on building surfaces, characterized by the following steps: include: Step S01. Construct a detection model for identifying metal corrosion regions in an image. The detection model includes at least a multi-scale occlusion perception feature extraction module and a corrosion feature enhancement attention module. The multi-scale occlusion perception feature extraction module obtains an initial feature map extracted by a feature extraction network and performs adaptive convolution processing based on the feature density of the initial feature map to obtain a density-adapted feature map. The density-adapted feature map is then processed using spatial attention and channel attention mechanisms to perceive occlusion regions, reduce the feature values ​​of occlusion regions, and initially enhance corrosion features and corrosion boundary features to obtain an occlusion perception processed feature map. The corrosion feature enhancement attention module performs multi-level corrosion visual feature decomposition on the feature map output by the multi-scale occlusion perception feature extraction module and fuses the decomposed corrosion visual features to obtain an enhanced corrosion feature map. Step S02. Train the constructed detection model. During the training process, a dynamic perceptual loss function is used for training. After the training is completed, the trained detection model is obtained. The dynamic perceptual loss function is to evaluate the severity of corrosion based on the state of each corrosion area and adjust the weight of the loss term according to the severity of corrosion. Step S03. Obtain images of the building surface to be tested using a drone equipped with an image acquisition device to obtain the image to be tested. Input the image to be tested into the trained detection model and output the detection results of the metal corrosion area on the building surface. The detection results include the location of the corrosion area, corrosion type and severity assessment information. In the multi-scale occlusion perception feature extraction module, the DSConv module performs adaptive convolution processing based on the feature density of the initial feature map to obtain a density-adapted feature map. The steps include: Calculate the local variance of the initial feature map to evaluate the feature density and generate a density map D; The convolution operation is selected based on the value of the density map D, where the density value is higher than a first threshold. For dense regions, perform convolution operations using standard n×n convolutions, where the density value is below the second threshold. For sparse regions, n×n dilated convolution is used for convolution operations, with density values ​​between the second threshold. With the first threshold For the region between, standard n×n convolution is used for convolution operations, and n×n dilated convolution is used for convolution operations, and then the results are fused to obtain the final convolution operation result, where n represents the size of the convolution kernel.

2. The UAV visual inspection method for metal corrosion areas on building surfaces according to claim 1, characterized in that, The expression for calculating the final convolution result obtained by fusing the convolution operation performed using standard n×n convolution and the convolution operation performed using n×n dilated convolution is as follows: in, Represents the initial feature map. This represents the output feature map of the DSConv module. Represents a standard n×n convolution. This represents an n×n dilated convolution. , Indicates the expansion rate. The fusion coefficient, calculated based on density D, is expressed as follows: .

3. The UAV visual inspection method for metal corrosion areas on building surfaces according to claim 1, characterized in that, In the multi-scale occlusion perception feature extraction module, the OA module uses spatial attention and channel attention to perceive occluded areas, reduce the feature values ​​of occluded areas, and initially enhance erosion features and erosion boundary features. The OA module includes a spatial attention branch, a channel attention branch, a local-global boundary enhancer, and a feature fusion unit. The spatial attention branch generates a spatial attention map based on the density-adapted feature map to assign weights to each pixel position to represent the probability of occlusion at the corresponding position. The channel attention branch obtains channel statistical features from the density-adapted feature map and generates channel weight vectors through a shared multilayer perceptron to evaluate the importance of different channels. The local-global boundary enhancer extracts local edge features from the density-adapted feature map using depthwise separable convolution and obtains global context information through dilated convolution. The results of depthwise separable convolution and dilated convolution are fused to form fused edge features to enhance erosion boundary features. The feature fusion unit is used to combine the density-adapted feature map, the spatial attention map output by the spatial attention branch, the channel weight vector output by the channel attention branch, and the fused edge features output by the local-global boundary enhancer to obtain the occlusion-aware feature map.

4. The UAV visual inspection method for metal corrosion areas on building surfaces according to claim 3, characterized in that, The calculation expression for the feature map after occlusion perception processing is as follows: in, This is the feature map after occlusion perception processing output by the OA module. This is the input feature map of the OA module, i.e., the density-adapted feature map output by the DSConv module. The spatial attention graph output by the spatial attention branch. By analyzing the input feature map It is generated using two n×n convolutional layers and the Sigmoid activation function. The channel weight vector is the output of the channel attention branch. By analyzing the input feature map Channel statistical features are obtained using global average pooling and max pooling, and generated using a shared multilayer perceptron. The fused edge features are output by the local-global boundary enhancer.

5. The UAV visual inspection method for metal corrosion areas on building surfaces according to claim 1, characterized in that, The multi-scale occlusion perception feature extraction module further includes an AFE module for adaptive feature enhancement of the feature map after occlusion perception processing. The AFE module includes a multi-scale feature extraction unit, a scale attention mechanism, and an adaptive feature fusion unit. The multi-scale feature extraction unit extracts feature information of different scales in the feature map after occlusion perception processing through parallel convolutional branches of different scales. The scale attention mechanism generates weight coefficients for each scale branch. The adaptive feature fusion unit performs weighted fusion of the multi-scale features based on the attention weights to obtain the feature map after feature enhancement.

6. The UAV visual inspection method for metal corrosion areas on building surfaces according to any one of claims 1 to 5, characterized in that, The corrosion feature enhancement attention module includes a multi-level corrosion feature decomposer, a context fusion enhancer, and a cross-layer feature aggregation unit. The multi-level corrosion feature decomposer decomposes the feature map output by the multi-scale occlusion perception feature extraction module into multiple visual feature channels to achieve corrosion visual feature filtering. The visual feature channels include color anomaly channels, texture change channels, and structural defect channels. The features of each visual feature channel are fused through a self-attention mechanism to generate a comprehensive corrosion feature representation. The context fusion enhancer calculates the correlation between each position and all other positions in the comprehensive corrosion feature representation output by the multi-level corrosion feature decomposer using a self-attention mechanism, generates an attention map, and performs weighted aggregation based on the attention map to enhance the contrast between the corrosion region and the background. The cross-layer feature aggregation unit performs cross-layer aggregation on the feature map output by the context fusion enhancer using a feature pyramid network structure and adjusts the weights of features at different levels according to the size characteristics of the corrosion region, finally obtaining the enhanced corrosion feature map output.

7. The UAV visual inspection method for metal corrosion areas on building surfaces according to any one of claims 1 to 5, characterized in that, The calculation expression for the dynamic sensing loss function is as follows: in, This represents the total dynamic sensing loss. Represents classification loss. Indicates location loss. Indicates confidence loss. Indicates boundary loss. , as well as These are the weighting coefficients for each loss term. , as well as These are parameters adjusted to control the sensitivity of each loss term's weight to severity. The corrosion severity score is calculated based on the size, density, and texture complexity of the corroded area.

8. The UAV visual inspection method for metal corrosion areas on building surfaces according to any one of claims 1 to 5, characterized in that, The detection model includes an image preprocessing module before the scale occlusion perception feature extraction module. The image preprocessing module includes an image quality assessment unit for screening images that meet quality requirements and an illumination compensation unit for performing illumination compensation. The illumination compensation unit performs multi-level decomposition of the input image by using wavelet transform to suppress noise in high-frequency components and performs brightness equalization processing to adjust the image brightness distribution.

9. A UAV visual inspection device for metal corrosion areas on building surfaces, comprising a processor and a memory, wherein the memory is used to store computer programs, characterized in that, The processor is used to execute the computer program to perform the method as described in any one of claims 1 to 8.

Images