Building damage identification and marking method based on UAV images

Abstract

The present invention discloses a method for identifying and marking building damage based on drone imagery, belonging to the field of image data processing technology. The method comprises the following steps: constructing a dataset D containing damaged buildings; constructing an improved target detection network; selecting a loss function as Width Over (WIoU) and using dataset D to train the improved target detection network until convergence by minimizing Width Over (WIoU), thereby obtaining an improved target recognition model; and using a drone equipped with a gimbal camera to capture an aerial image to be identified, inputting the image into the improved target recognition model to obtain a recognition map, and locating the image on a map. The method can accurately identify damaged buildings from aerial images, utilize the drone's position information and gimbal camera information to directly mark the world coordinates of the damaged areas, and directly return a marked map to facilitate real-time damage assessments in disaster-stricken areas by management personnel, enabling rapid results in areas with limited communication.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a method for identifying and marking building damage based on drone images. Background Art

[0002] Building damage assessment is very important in post-disaster reconstruction and post-disaster assessment. In building damage assessment, the first step is to identify the damaged buildings. Identification methods include remote sensing satellite-based identification methods, drone-based identification methods, and field survey-based identification methods.

[0003] Identification methods based on remote sensing satellites include: (1) Jiawei Tang from Xi'an University of Technology used deep learning to design a network model, using the principles of machine learning related to computer vision. Buildings in remote sensing satellite images were segmented using convolutional neural networks, and damage classification and assessment networks were used to detect damage to buildings in sub-temporal remote sensing images. This method can meet the needs of post-disaster assessment in most situations today. (2) NV Ivanova of Russia used remote sensing satellites to use height models and regression models to assess forest damage after a hurricane. Abhilash Dutta Roy of the United States used remote sensing satellites, mainly using lidar and drones, to assess damage after a hurricane. This provides local managers with reasonable construction and management methods. However, this method is mainly limited to two-dimensional planes and is affected by weather. The marking accuracy is low, the cost is high, and the real-time performance is insufficient.

[0004] Drone-based recognition methods: For example, Daniel Jozi of Canada used drone imagery to automatically assess damaged buildings after a disaster. The proposed method integrates texture-based features, including variability and uniformity, with edge-based features, employing edge detection to identify and assess damage to buildings after a disaster. However, this method tends to overlook the limited computing resources in disaster-stricken areas, and communications may be impaired, making it difficult to quickly obtain recognition results on the disaster front lines. Field survey-based recognition methods typically involve experts visiting the disaster area for on-site inspection, identification, and assessment. This method is slow, significantly affected by the environment and terrain, and can be quite dangerous.

[0005] Currently, after a damaged building is identified, the object detection model typically returns an identification image—an image labeled with the target category and detection box. This image cannot be directly mapped to a map. Furthermore, a series of coordinate transformations are performed during mapping, such as converting image coordinates to camera coordinates, then to drone coordinates, and finally to world coordinates. The conversion from camera coordinates to drone coordinates relies on a rotation matrix. Currently, drones require a gimbal to stabilize the camera, but existing technologies only consider the drone's body information during this conversion, ignoring gimbal deflection information. This results in inaccurate coordinate conversions and network-dependent positioning.

[0006] Glossary:

[0007] YOLOv11 (You Only Look Once version 11) is an object detection algorithm. Its network architecture consists of a backbone network, a neck network, and a head network. The C3K2 module is a feature extraction component in the YOLOv11 backbone network. It is an improved design based on the traditional C3 module. The backbone network generally consists of four C3K2 modules from top to bottom, with the SPPF layer as the last layer. The backbone network outputs a feature map at the output of the second C3K2 module, the third C3K2 module, and the SPPF layer, respectively, which are fed into the neck network. A total of three feature maps are input into the neck network.

[0008] HS-FPN (Hierarchical Scale-based Feature Pyramid Network or High-level Screening-feature Pyramid Network) is a network architecture for multi-scale feature fusion, primarily used to address the multi-scale challenges in object detection. HS-FPN consists of a feature selection module and a feature fusion module. The feature selection module consists of multiple branches, each of which performs channel attention (CA) on feature maps of different scales. A 1×1 convolutional layer is then used to align the dimensions of feature maps of different scales before fusion, ensuring that they have the same dimensions before fusion, enabling effective feature fusion. Selective Feature Fusion (SFF): By combining filtered low-level features with high-level features, HS-FPN captures more comprehensive target feature information by fusing features at different levels, thereby improving the model's ability to detect objects of different scales.

[0009] C3K2 module: It is a key module in the YOLO11 model. It is an improved design based on the traditional C3 module. Its module structure features are: (1) Compared with the standard C3 module, the C3K2 module introduces a variable convolution kernel C3K module and a channel separation strategy, providing more powerful feature extraction capabilities, especially suitable for more complex scenes and deep feature extraction tasks. The C3K2 structure includes a CBS layer, a segmentation layer, a multi-layer stacked C3K module, a splicing layer, and a CBS layer. Assume that the input feature F in First, feature extraction is performed through the CBS layer, and then it is divided into two parts F1 and F2. F1 is extracted through the multi-layer stacked C3K module feature extraction to obtain F3, which is then spliced ​​with F2 through the splicing layer and then passed through the CBS module to obtain the output feature F of the C3K2 module. out。

[0010] CBS module: Convolution-BatchNorm-SiLU module. It is a standard convolutional module consisting of a convolution layer, a batch normalization layer, and a SiLU activation function. This module extracts relevant information, normalizes the data stream, and uses the SiLU activation function to refine the feature map.

[0011] The C3K module is part of the C3K2 module. Its structure consists of a convolutional layer, a multi-layered bottleneck layer, a concatenation layer, and a CBS layer. Similarly, after the input features are extracted by the convolutional layer, they are split into two paths: one path is passed through the multi-layered bottleneck layer to obtain an output, and the other path is concatenated with the output through the concatenation layer before passing through the CBS layer to obtain the output features.

[0012] MLLA (Mamba-Like Linear Attention) is a model that combines the Mamba model with the linear attention mechanism. MLLA mainly integrates the two key factors of Mamba: the "forget gate" and block design. At the same time, MLLA uses positional encoding (ROPE) to replace the forget gate, thereby providing the necessary position information while maintaining parallel computing and fast inference speed.

[0013] Aggregated Attention is an aggregated attention layer, an enhanced version of TransXNet, and was first used in YOLOv8.

[0014] mAP (mean Average Precision) is the average precision value of the category.

[0015] GFlOPs (Giga Floating-point Operations Per second) is the floating-point operation amount. Summary of the Invention

[0016] The purpose of the present invention is to provide a method for identifying and marking building damage based on drone images that solves the above problems, can quickly obtain identification and positioning results in communication-restricted areas, and has clear identification, accurate positioning, and is not restricted by communication.

[0017] In order to achieve the above-mentioned object, the technical solution adopted by the present invention is as follows: a method for identifying and marking building damage based on drone images, comprising the following steps;

[0018] S1, construct a dataset D containing damaged buildings;

[0019] Obtain multiple images containing damaged buildings in different scenes. For each image, mark the damaged building as the target to obtain samples. All samples constitute the dataset D.

[0020] S2, construct an improved target detection network, including S21~S24;

[0021] S21, obtaining a YOLOv11 object detection network, including a backbone network, a neck network, and a head network, wherein the backbone network includes four C3K2 modules from top to bottom, the last layer of the backbone network is an SPPF layer, and the backbone network outputs the first feature map to the third feature map from the output end of the second C3K2 module, the third C3K2 module, and the SPPF layer in sequence;

[0022] The neck network includes 3 input terminals and 3 output terminals, wherein the i-th input terminal is marked as the i-th input terminal, corresponding to the input i-th feature map, 1≤i≤3, and the three output terminals are respectively connected to the three detection heads of the head network, and the detection head includes a classification branch and a regression branch;

[0023] S22, copy a backbone network, mark the two backbone networks as B1 and B2 respectively, and set a channel splicing layer between the two i-th feature maps to splice the two i-th feature maps into the i-th splicing map, which is input to the i-th input end of the neck network, 1≤i≤3;

[0024] S23, introduce the MLLA model into the C3K2 module to obtain the C3K2-MB module, and introduce the aggregated attention layer into the C3K2 module to obtain the C3K2-AggregatedAtt module;

[0025] S24, replace each C3K2 module in B1 with a C3K2-MB module, replace several C3K2 modules in B1 with a C3K2-AggregatedAtt module, add HS-FPN to the classification branch and the regression branch, and obtain an improved object detection network;

[0026] S3, select the loss function as WIoU, use the dataset D to minimize WIoU to train the improved target detection network until convergence, and obtain the improved target recognition model;

[0027] In S4, the drone is equipped with a gimbal camera to take aerial photos of the target object, which are input into the improved target recognition model to obtain the recognition image and locate it on the map.

[0028] Preferably, in S1, the image is obtained by extracting frames from a video containing a damaged building.

[0029] Preferably, in S22, the i-th spliced ​​graph is obtained as follows:

[0030] Input the samples into B1 and B2 respectively, and mark the i-th feature map of B1 as , the i-th feature map of B2 is marked as , the channel splicing layer obtains the i-th splicing graph F according to the following formula i ;

[0031] ,

[0032] Where Concat(∙) is the channel concatenation operation.

[0033] Preferably, in S23, the method for constructing the C3K2-MB module is:

[0034] A C3K2 module is obtained, which includes multiple stacked C3K modules, and the C3K module contains multiple stacked bottleneck layers. The bottleneck layer in the C3K2 module is replaced with the MLLA model to obtain a C3K2-MB module.

[0035] Preferably, in S23, the method for constructing the C3k2-AggregatedAtt module is:

[0036] Sa1, construct the C3k-AggregatedAtt module:

[0037] Get the C3k module, including convolutional layers, N bottleneck layers, channel concatenation layers, and CBS layers;

[0038] The bottleneck layer is replaced by an aggregated attention layer. N aggregated attention layers are connected sequentially from top to bottom, and the output of each aggregated attention layer is connected to the channel splicing layer. The output of the channel splicing layer is connected to the CBS layer to obtain the C3k-AggregatedAtt module.

[0039] Sa2, replace the bottleneck layer in the C3K2 module with the C3k-AggregatedAtt module to obtain the C3k2-AggregatedAtt module.

[0040] Preferably, in S24, HS-FPN is added to the classification branch and the regression branch, specifically:

[0041] The classification branch includes a CBS layer, a depth-wise separable convolution layer, a CBS layer, a depth-wise separable convolution layer, a CBS layer, a HS-PFN layer, a two-dimensional convolution layer, and a HS-PFN layer connected in sequence, and the classification loss is calculated through the output of the last HS-PFN layer;

[0042] The regression branch includes two CBS layers, a HS-PFN layer, a two-dimensional convolutional layer, and a HS-PFN layer connected in sequence, and the bounding box loss is calculated through the output of the last HS-PFN layer.

[0043] Preferably, the positioning in S4 specifically includes the following steps:

[0044] S41, obtaining gimbal camera information and drone POS information, wherein the gimbal camera information includes camera information and gimbal deflection information;

[0045] S42, performing distortion correction on the recognition image based on the camera information to obtain a correction image P1;

[0046] S43, convert P1 from image coordinates to camera coordinates P according to camera information c ;

[0047] S44, construct the camera coordinate P according to the gimbal deflection information and the drone POS information c The rotation matrix R to the drone coordinates c2b , the camera coordinate P c Convert to drone coordinates P b ;

[0048] S45, converts the drone coordinates to world coordinates through rigid body transformation;

[0049] S46, marking the area of ​​the bounding box in the aerial image on the map.

[0050] Preferably, the drone POS information includes longitude, latitude, altitude, heading, roll, pitch, and yaw information.

[0051] Preferably, the distortion correction includes radial distortion correction and tangential distortion correction.

[0052] When constructing an improved target detection network, the present invention designs a C3K2-MB module and a C3k2_AggregatedAtt module, and adds HS-FPN to the classification branch and regression branch of the detection head.

[0053] Regarding the C3K2-MB module: The bottleneck layer of the C3k module is replaced with the Mamba-Like Linear Attention (MLLA) module. MLLA integrates Mamba's selective state-space mechanism and uses RoPE positional encoding instead of the traditional forget gate, enhancing positional awareness while retaining parallel computing capabilities. The standard residual block in C3K2 is replaced with the Mamba-2 layer, leveraging its linear time complexity and long sequence modeling advantages. Mamba's selective scanning mechanism dynamically adjusts the receptive field of the convolution kernel in C3K2 to adapt to inputs of varying resolutions. This change preserves the cross-stage connection structure and only modifies the internal feature processing units.

[0054] The C3k2_AggregatedAtt module integrates the aggregated attention mechanism directly into the C3k2 module. This integration is achieved through feature reweighting, embedding an attention submodule within the convolutional reparameterization structure to form the new C3k2_AggregatedAtt module. Unlike solutions that insert a separate attention layer, this design achieves a deep fusion of feature extraction and attention. Compared to the original version without attention, it enhances multi-scale feature interaction, dynamically adjusts feature responses, and improves sensitivity to small objects, while maintaining the original residual design and preventing gradient vanishing.

[0055] The HS-FPN module is introduced into the detection head. Because it processes multi-scale features, the CBS module connected to it is used in the preceding stage to unify the number of channels of each layer's feature map to 256 using 1×1 convolution before feeding it into the HS-FPN module. The HS-FPN module uses multi-level feature fusion and a channel-wise attention mechanism to gradually integrate features from different layers, improving the model's segmentation performance. The HSFPN effectively integrates multi-scale features from the backbone through channel adjustment, upsampling, attention, and feature fusion. After processing, it outputs features with a unified number of channels, facilitating subsequent processing in the segmentation head.

[0056] The idea of ​​the present invention is: first construct an improved target detection network and train it into an improved target recognition model, where the target is a damaged building; then use a drone equipped with a gimbal camera to take aerial photos to be identified; then use the target recognition model to identify the target in the aerial photos to obtain an identification map, and finally locate the target in the identification map on the map.

[0057] Compared with the prior art, the advantages of the present invention are:

[0058] (1) Improved target detection network. First, two backbone networks are used. In one backbone network, the C3K2 module is replaced by the C3K2-MB module, and in the other backbone network, the C3K2 module is replaced by the C3k2_AggregatedAtt module. The C3K2-MB module can dynamically adjust the receptive field of the convolution kernel in C3K2 based on the selective scanning mechanism of Mamba to adapt to inputs of different resolutions. The C3k2_AggregatedAtt module can enhance the multi-scale feature interaction capability, dynamically adjust the feature response, improve the sensitivity to small targets, maintain the original residual design, and avoid gradient disappearance. The features extracted by the two backbone networks are further channel-spliced ​​and then sent to the neck network and the head network. The improvement of the backbone network has strong robustness, saves computing resources, focuses on spatial information, and can also retain more detailed information. Second: Based on the improvement of the backbone network, the present invention also introduces the HS-FPN module in the head network, which has the advantages of reducing training parameters and focusing on more small and detailed features.

[0059] Through these improvements, the improved object recognition model ultimately trained by the present invention can effectively identify occluded or overlapping objects, thereby improving segmentation performance. Its design enables the model to better handle these difficult samples, thereby enhancing detection reliability. At the same time, its high computational efficiency reduces computational costs while maintaining high detection accuracy.

[0060] (2) The present invention performs a series of coordinate transformations during positioning. The present invention overcomes the defect of the prior art that only the rotation matrix from the drone to the camera is considered while the gimbal deflection information is ignored. The rotation matrix is ​​constructed by considering the gimbal deflection information and the drone POS information, thereby ensuring the accuracy of the coordinate transformation. According to the method of step S4 of the present invention, the present invention can use the drone's POS information and the gimbal camera information to directly mark the world coordinates of the damaged area and directly return a marked map to facilitate management personnel to conduct real-time assessment of the damage in the disaster area, and the results can be quickly obtained in areas with limited communication. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 To improve the target recognition model structure diagram;

[0062] Figure 2 This is a connection diagram of two backbone networks;

[0063] Figure 3 This is a schematic diagram of the C3K2-MB module structure;

[0064] Figure 4 Schematic diagram of the C3K2-AggregatedAtt module structure;

[0065] Figure 5Schematic diagram of the detection head structure. DETAILED DESCRIPTION

[0066] The present invention will be further described below with reference to the embodiments and accompanying drawings.

[0067] Example 1: See Figures 1 to 5 ,A building damage identification and marking method based on UAV ,imagery includes the following steps;

[0068] S1, construct a dataset D containing damaged buildings;

[0069] Obtain multiple images containing damaged buildings in different scenes. For each image, mark the damaged building as the target to obtain samples. All samples constitute the dataset D.

[0070] S2, construct an improved target detection network, including S21~S24;

[0071] S21, obtaining a YOLOv11 object detection network, including a backbone network, a neck network, and a head network, wherein the backbone network includes four C3K2 modules from top to bottom, the last layer of the backbone network is an SPPF layer, and the backbone network outputs the first feature map to the third feature map from the output end of the second C3K2 module, the third C3K2 module, and the SPPF layer in sequence;

[0072] The neck network includes 3 input terminals and 3 output terminals, wherein the i-th input terminal is marked as the i-th input terminal, corresponding to the input i-th feature map, 1≤i≤3, and the three output terminals are respectively connected to the three detection heads of the head network, and the detection head includes a classification branch and a regression branch;

[0073] S22, copy a backbone network, mark the two backbone networks as B1 and B2 respectively, and set a channel splicing layer between the two i-th feature maps to splice the two i-th feature maps into the i-th splicing map, which is input to the i-th input end of the neck network, 1≤i≤3;

[0074] S23, introduce the MLLA model into the C3K2 module to obtain the C3K2-MB module, and introduce the aggregated attention layer into the C3K2 module to obtain the C3K2-AggregatedAtt module;

[0075] S24, replace each C3K2 module in B1 with a C3K2-MB module, replace several C3K2 modules in B1 with a C3K2-AggregatedAtt module, add HS-FPN to the classification branch and the regression branch, and obtain an improved object detection network;

[0076] S3, select the loss function as WIoU, use the dataset D to minimize WIoU to train the improved target detection network until convergence, and obtain the improved target recognition model;

[0077] In S4, the drone is equipped with a gimbal camera to take aerial photos of the target object, which are input into the improved target recognition model to obtain the recognition image and locate it on the map.

[0078] In step S1, the image is obtained by extracting frames from a video containing a damaged building.

[0079] In step S22, the i-th spliced ​​graph is obtained by inputting samples into B1 and B2 respectively, and marking the i-th feature graph of B1 as , the i-th feature map of B2 is marked as , the channel splicing layer is based on Get the i-th splicing graph F i , where Concat(∙) is the channel concatenation operation.

[0080] In step S24, for B1, all four C3K2 modules are replaced by C3K2-MB modules, but for B2, several C3K2 modules can be replaced by C3K2-AggregatedAtt modules. Figure 2 In the example shown in B2, the second and fourth C3K2 modules are replaced. Similarly, all four C3K2 modules in B2 may be replaced.

[0081] The positioning in S4 specifically includes the following steps S41 to S46;

[0082] S41, obtaining gimbal camera information and drone POS information, wherein the gimbal camera information includes camera information and gimbal deflection information;

[0083] S42, performing distortion correction on the recognition image based on the camera information to obtain a correction image P1;

[0084] S43, convert P1 from image coordinates to camera coordinates P according to camera information c ;

[0085] S44, construct the camera coordinate P according to the gimbal deflection information and the drone POS information c The rotation matrix R to the drone coordinates c2b , the camera coordinate P c Convert to drone coordinates P b ;

[0086] S45, converts the drone coordinates to world coordinates through rigid body transformation;

[0087] S46, marking the area of ​​the bounding box in the aerial image on the map.

[0088] The drone POS information includes longitude, latitude, altitude, heading, roll, pitch, and yaw information. The distortion correction includes radial distortion correction and tangential distortion correction.

[0089] Example 2: See Figures 1 to 5 ,Based on Example 1, we give more specific construction methods and structures of ,C3K2-MB module, C3k2-AggregatedAtt module, and detection head.

[0090] The method of constructing the C3K2-MB module is as follows: obtain a C3K2 module, which includes multiple stacked C3K modules, and the C3K module contains multiple stacked bottleneck layers. Replace the bottleneck layer in the C3K2 module with the MLLA model to obtain the C3K2-MB module. For more detailed structure, see Figure 3 : The structure of the C3K2 module includes a segmentation layer, a convolutional layer A1, a convolutional layer A2, N stacked C3K-MB modules, a channel splicing layer C1, and a convolutional layer A3. The features of the C3K2-MB module are input into two paths after the segmentation layer. One path passes through the convolutional layer A1 and is directly sent to the channel splicing layer C1. The other path passes through the convolutional layer A2 and N stacked C3K-MB modules and then is sent to the channel splicing layer C1. The structure of the C3K-MB module includes a convolutional layer A4, N stacked MLLA modules, a channel splicing layer C2, and a convolutional layer A5, which are arranged in sequence. The features sent to the C3K-MB module are first extracted by the convolutional layer A4 and then divided into two paths. One path is directly sent to the channel splicing layer C2, and the other path is sent to the channel splicing layer C2 after passing through N stacked C3K-MB modules. The channel splicing layer C2 performs channel splicing on the input features, and then performs a convolution operation on the convolutional layer A5 before sending them to the next level.

[0091] In S23, the method for constructing the C3k2-AggregatedAtt module is:

[0092] Sa1, construct C3k-AggregatedAtt module: obtain C3k module, including convolution layer, N bottleneck layers, channel splicing layer, CBS layer; replace the bottleneck layer with aggregate attention layer, N aggregate attention layers are connected from top to bottom, and the output of each aggregate attention layer is connected to the channel splicing layer, and the output of the channel splicing layer is connected to the CBS layer to obtain C3k-AggregatedAtt module;

[0093] Sa2, replace the bottleneck layer in the C3K2 module with the C3k-AggregatedAtt module to obtain the C3k2-AggregatedAtt module.

[0094] The structure of the C3k-AggregatedAtt module can be found in Figure 4On the right, we also improve the bottleneck layer of the C3k module by replacing N stacked bottleneck layers with N stacked aggregate attention layers. In addition to stacking, the output of each aggregate attention layer is introduced into the channel concatenation layer. In the C3k-AggregatedAtt module, assuming that the feature F in , extract the feature F through the convolution layer conv It is then divided into two paths, one is sent to the channel splicing layer, and the other is extracted through N aggregation attention layers in sequence. The aggregation attention feature output by the nth aggregation attention layer is , 1≤n≤N, the channel splicing layer will F conv and N Perform channel splicing and then pass the convolution layer to obtain the output feature F of the C3k-AggregatedAtt module out C3k2-AggregatedAtt module structure see Figure 4 On the left is the improvement of the C3k2 module. The C3k2 module in the C3k2 module can be directly replaced with the C3k-AggregatedAtt module.

[0095] Add HS-FPN to the classification branch and regression branch of the detection head, specifically;

[0096] The classification branch includes a CBS layer, a depth-wise separable convolution layer, a CBS layer, a depth-wise separable convolution layer, a CBS layer, a HS-PFN layer, a two-dimensional convolution layer, and a HS-PFN layer connected in sequence, and the classification loss is calculated through the output of the last HS-PFN layer;

[0097] The regression branch includes two CBS layers, a HS-PFN layer, a two-dimensional convolutional layer, and a HS-PFN layer connected in sequence, and the bounding box loss is calculated through the output of the last HS-PFN layer.

[0098] Example 3: Based on the design of the present invention, the parameters in the network can be greatly reduced. By comparing the improved target detection network constructed by S21 of the present invention with the existing YOLOv11 target detection network, the present invention can reduce the parameters in each layer of the network by approximately 40.3%. In order to illustrate the effect of the present invention, this embodiment uses YOLOv8, YOLOv11 and the improved target detection network constructed by S21 of the present invention to conduct a detection performance comparison experiment under the same experimental conditions. The experimental results use the performance indicators of precision Precision, recall rate Recall, average precision value mAP of all categories, and floating-point operation amount GFlOPs to evaluate the performance of the algorithm proposed in this study, and obtain Table 1. Table 1 Experimental conditions are: using model segmentation task, training environment GPU: NVIDIA 4080; system: WINDOWS11.

[0099] Table 1. Comparison of detection performance of different detection models

[0100] Model Precision Recall mAP50 mAP50-90 GFLOPs YOLOv8 0.595 0.44 0.509 0.315 1328.4 YOLOv11 0.644 0.581 0.614 0.354 318.5 The present invention 0.694 0.61 0.659 0.43 9.9

[0101] In Table 1, mAP50 represents the average precision when the IoU is set to 0.5, and mAP50-90 represents the average precision when the IoU ranges from 0.5 to 0.95 with a step size of 0.05. IoU is the intersection over union ratio between the predicted box and the ground-truth box.

[0102] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for identifying and marking building damage based on drone images, characterized in that: The following steps are included: S1, construct a dataset D containing damaged buildings; Obtain multiple images containing damaged buildings in different scenes. For each image, mark the damaged building as the target to obtain samples. All samples constitute the dataset D. S2, construct an improved target detection network, including S21~S24; S21, obtaining a YOLOv11 object detection network, including a backbone network, a neck network, and a head network, wherein the backbone network includes four C3K2 modules from top to bottom, the last layer of the backbone network is an SPPF layer, and the backbone network outputs the first feature map to the third feature map from the output end of the second C3K2 module, the third C3K2 module, and the SPPF layer in sequence; The neck network includes 3 input terminals and 3 output terminals, wherein the i-th input terminal is marked as the i-th input terminal, corresponding to the input i-th feature map, 1≤i≤3, and the three output terminals are respectively connected to the three detection heads of the head network, and the detection head includes a classification branch and a regression branch; S22, copy a backbone network, mark the two backbone networks as B1 and B2 respectively, and set a channel splicing layer between the two i-th feature maps to splice the two i-th feature maps into the i-th splicing map, which is input to the i-th input end of the neck network, 1≤i≤3; S23, introduce the MLLA model into the C3K2 module to obtain the C3K2-MB module, and introduce the aggregated attention layer into the C3K2 module to obtain the C3K2-AggregatedAtt module; S24, replace each C3K2 module in B1 with a C3K2-MB module, replace several C3K2 modules in B1 with a C3K2-AggregatedAtt module, add HS-FPN to the classification branch and the regression branch, and obtain an improved object detection network; S3, select the loss function as WIoU, use the dataset D to minimize WIoU to train the improved target detection network until convergence, and obtain the improved target recognition model; S4, the drone carries a gimbal camera to take the aerial image to be identified, inputs it into the improved target recognition model, obtains the identification image, and locates it on the map; In S23, the method for constructing the C3K2-MB module is: Obtain a C3K2 module, including multiple stacked C3K modules, which in turn contain multiple stacked bottleneck layers. Replace the bottleneck layer in the C3K2 module with the MLLA model to obtain a C3K2-MB module. The method to construct the C3k2-AggregatedAtt module is: Sa1, construct the C3k-AggregatedAtt module: Get the C3k module, including convolutional layers, N bottleneck layers, channel concatenation layers, and CBS layers; The bottleneck layer is replaced by an aggregated attention layer. N aggregated attention layers are connected sequentially from top to bottom, and the output of each aggregated attention layer is connected to the channel splicing layer. The output of the channel splicing layer is connected to the CBS layer to obtain the C3k-AggregatedAtt module. Sa2, replace the bottleneck layer in the C3K2 module with the C3k-AggregatedAtt module to obtain the C3k2-AggregatedAtt module; In S24, HS-FPN is added to the classification branch and regression branch, specifically: The classification branch includes a CBS layer, a depth-wise separable convolution layer, a CBS layer, a depth-wise separable convolution layer, a CBS layer, a HS-FPN layer, a two-dimensional convolution layer, and a HS-FPN layer connected in sequence, and the classification loss is calculated through the output of the last HS-FPN layer; The regression branch includes two CBS layers, a HS-FPN layer, a two-dimensional convolution layer, and a HS-FPN layer connected in sequence, and the bounding box loss is calculated through the output of the last HS-FPN layer.

2. The method for identifying and marking building damage based on drone images according to claim 1 is characterized in that: In S1, the image is obtained by extracting frames from a video containing a damaged building.

3. The method for identifying and marking building damage based on drone images according to claim 1 is characterized in that: In S22, the i-th spliced ​​graph is obtained by inputting the samples into B1 and B2 respectively, and marking the i-th feature graph of B1 as , the i-th feature map of B2 is marked as , the channel splicing layer obtains the i-th splicing graph F according to the following formula i ; ， Where Concat(∙) is the channel concatenation operation.

4. The method for identifying and marking building damage based on drone images according to claim 1, characterized in that: The positioning in S4 specifically includes the following steps: S41, obtaining gimbal camera information and drone POS information, wherein the gimbal camera information includes camera information and gimbal deflection information; S42, performing distortion correction on the recognition image based on the camera information to obtain a correction image P1; S43, convert P1 from image coordinates to camera coordinates P according to camera information c ; S44, construct the camera coordinate P according to the gimbal deflection information and the drone POS information c The rotation matrix R to the drone coordinates c2b , the camera coordinate P c Convert to drone coordinates P b ; S45, converts the drone coordinates to world coordinates through rigid body transformation; S46, marking the area of ​​the bounding box in the aerial image on the map.

5. The method for identifying and marking building damage based on drone images according to claim 4 is characterized in that: The drone POS information includes longitude, latitude, altitude, heading, roll, pitch, and yaw information.

6. The method for identifying and marking building damage based on drone images according to claim 4 is characterized in that: The distortion correction includes radial distortion correction and tangential distortion correction.

Images