Remote sensing image building detection method
By introducing perceived coordinate attention, multi-scale residual and densely connected feature pyramid modules into the YOLOv7 network, the detection problems of complex background and scale diversity in remote sensing images are solved, and the accuracy and robustness of building detection are significantly improved.
Patent Information
- Application Number
- CN202411757292.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-03
- Publication Date
- 2025-03-11
AI Technical Summary
The existing remote sensing image building detection methods are prone to missed and missed detection problems under complex backgrounds and diversified scales, and the characteristics of small buildings do not significantly affect the accuracy of the algorithm.
An improved YOLOv7 network architecture is designed, combining the perceived coordinate attention (PCA) module, a multi-scale residual (MSR) module and a densely connected feature pyramid (DFP) module to enhance the robustness and accuracy of the model.
Through the combination of these modules, interference from complex backgrounds is reduced, the problems of building scale changes and insignificant characteristics of small buildings are alleviated, and the accuracy and robustness of target detection are improved.
Smart Images
Figure CN119600453A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a remote sensing image building detection method, and belongs to the technical field of deep learning. Background Art
[0002] In the past few decades, traditional remote sensing image building detection methods have mainly relied on image processing and machine learning techniques. However, these traditional methods have gradually revealed some limitations when dealing with the rapid changes and large-scale demands in the urbanization process. Manually designed features may be difficult to cover complex urban environments and different types of buildings, and the performance of traditional classifiers may be limited by the quality and diversity of features, all of which will affect the accuracy and generalization ability of the model.
[0003] With the rise of deep learning technology, convolutional neural networks such as R-CNN, Fast R-CNN, Faster R-CNN and YOLO series have achieved remarkable achievements in remote sensing images, and can locate and identify building targets at different scales, angles and complex backgrounds. However, there are still some problems in the application of remote sensing image building detection methods based on deep learning. For example, common deep learning algorithms may miss detection and misdetection when dealing with complex backgrounds; the diversity of building scales and the lack of obvious features of small buildings will affect the accuracy of the algorithm. Summary of the invention
[0004] In view of the above problems, the present invention provides a remote sensing image building detection method, collects remote sensing images of different regions through the Internet and performs a series of processing to create a remote sensing image building detection data set, and designs an improved YOLOv7 network architecture based on the YOLOv7 algorithm, combined with the Perceptual Coordinate Attention (PCA) module, the Multi-Scale Residuals (MSR) module and the Dense Feature Pyramid Module (DFP) module. Through this innovative design, the interference of complex backgrounds is further reduced, and the problems caused by the change of building scale and the unclear features of small buildings are effectively alleviated, thereby improving the robustness and accuracy of target detection.
[0005] As the background of buildings in remote sensing images is complex, the YOLOv7 network does not introduce an attention mechanism and cannot fully capture the associations and dependencies between different locations and distinguish the information at different locations. To this end, the present invention designs a Perceptual Coordinate Attention (PCA) module and inserts it into the backbone network of YOLOv7 to fully capture the associations and dependencies between different locations, thereby improving the feature extraction and target positioning capabilities of the model in remote sensing image building detection under complex backgrounds. The advantage of the PCA module is that it can make full use of the input information and improve the model's perception and positioning capabilities for remote sensing image buildings. By introducing coordinate attention and a lightweight multi-layer perceptron, the PCA module can adaptively pay attention to and process the position and scale features of buildings in different remote sensing images, thereby improving the accuracy and robustness of remote sensing image building detection. The ELAN module of the YOLOv7 network backbone is used multiple times at different levels to extract multi-scale features of remote sensing buildings, but the size of the receptive field of the ELAN module is 3×3, and the multi-scale features of remote sensing image buildings cannot be fully extracted. The present invention designs a multi-scale residual (Multi-Scale Residuals, MSR) module with a stronger ability to extract multi-scale features to replace the ELAN module in YOLOv7, further enhancing the network's ability to extract multi-scale remote sensing building features. The MSR module is intended to be designed as a series of parallel convolution branches with different receptive field sizes to capture features of different scales. In the MSR module, the present invention also intends to introduce a residual connection mechanism to further improve the performance of the module. Residual connections can enable features in different branches to be transferred to each other, which enables the model to better understand the subtle differences brought about by changes in the size of remote sensing buildings. Embedding the MSR module in YOLOv7 enables the model to learn more robust remote sensing building feature extraction capabilities at multiple scales and pay attention to the feature differences of remote sensing buildings at different scales, thereby providing a more comprehensive feature representation for remote sensing buildings of various scales, thereby improving the performance of the model in detecting remote sensing buildings at multiple scales.
[0006] In view of the fact that the features of small remote sensing buildings in remote sensing images are tiny and not obvious, the present invention improves the PAFPN structure in the YOLOv7 network head and proposes a densely connected feature pyramid (Dense Feature Pyramid Module, DFP) module to improve the detection accuracy of small remote sensing buildings. Based on the original PAFPN structure, the densely connected feature pyramid (DFP) module uses a skip connection strategy to densely connect each feature layer, so that the fused feature map has rich semantic information and location information. This can strengthen the combination of the remote sensing building location information of shallow features and the remote sensing building semantic information of deep features, and solve the problem of missing semantic information of small remote sensing buildings in shallow feature maps. BRIEF DESCRIPTION OF THE DRAWINGS
[0007] Figure 1 A schematic diagram of a perception coordinate attention module provided by the present invention; Figure 2 A schematic diagram of a coordinate attention module; Figure 3 A schematic diagram of a lightweight multi-layer perceptron module; Figure 4 A schematic diagram of a multi-scale residual module provided by the present invention; Figure 5 A schematic diagram of a densely connected feature pyramid module provided by the present invention; Figure 6 A remote sensing image building detection algorithm network is provided by the present invention. DETAILED DESCRIPTION
[0008] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0009] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present application.
[0010] In one embodiment, a remote sensing image building detection method is provided. The YOLOv7 algorithm is used as the network framework for remote sensing image building detection, and an improved YOLOv7 network architecture is designed in combination with a Perceptual Coordinate Attention (PCA) module, a Multi-Scale Residuals (MSR) module, and a Dense Feature Pyramid Module (DFP) module, thereby enhancing the model's detection performance for remote sensing image buildings.
[0011] First, we download remote sensing images of multiple regions from the Internet, divide the remote sensing images into 1,000 images of 600×600 size, annotate the buildings in the images through annotation software, create an initial data set, and enhance the diversity of samples in the data set by performing brightness transformation, random cropping, rotation, scaling and other operations on the remote sensing images in the initial data set, and form 2,000 images as the final remote sensing image building data set. 70% of the data set is used for training, 20% for verification, and 10% for testing.
[0012] The present invention uses a perceptual coordinate attention (PCA) module to improve the backbone network of the YOLOv7 network and improve the model's feature extraction and target positioning capabilities under the complex background of remote sensing buildings. Specifically, the input of the PCA module is divided into three branches. The first branch is processed by a coordinate attention (CA) module, the second branch is processed by a lightweight multilayer perceptron (LMLP) module, and the third branch retains the original image features. The outputs of the LMLP module and the CA module are spliced together, and then the number of channels is reduced to the same as the input through a 1x1 convolution, and then pixel-level point addition operations are performed with the third branch, and finally used as the output of the PCA module. The innovation of the perceptual coordinate attention (PCA) module lies in the fusion of the coordinate attention module (CA) and the lightweight multilayer perceptron module (LMLP), giving play to the advantages of both. Through the perceptual coordinate attention (PCA) module, the improved network can better handle targets under the complex background of remote sensing buildings and improve the capabilities of feature extraction and target positioning.
[0013] Among them, the coordinate attention (CA) module adaptively focuses on remote sensing buildings at different locations by embedding location awareness, thereby improving the accuracy and robustness of remote sensing image building detection. The CA module decomposes channel attention into two feature encoding processes that aggregate features along different directions, capturing long-range dependencies along one spatial direction and retaining precise location information along another spatial direction. Then, the generated feature maps are encoded separately to form a pair of direction-aware and position-sensitive feature maps, which are finally complementarily applied to the input feature map to enhance the representation of remote sensing buildings of interest. The CA module structure is as follows: Figure 2 shown.
[0014] Specifically, for the input , first use the size and The pooling kernel encodes each channel along the horizontal and vertical coordinate directions, so the height is No. The output of each channel is expressed as follows: = (h,i) Similarly, the width is No. The output of each channel is expressed as follows: = (j,w) The above two transformations aggregate features along two spatial directions and return a pair of direction-aware attention maps. These two transformations also allow the attention module to capture long-range dependencies along one spatial direction and save precise location information along another spatial direction, which helps the network locate remote sensing buildings of interest more accurately. This coordinate information embedding operation corresponds to the X Avg Pool and Y Avg Pool parts in the above figure.
[0015] In order to better utilize the coordinate information embedding module above to generate a representation with a global receptive field and precise location information, a coordinate attention generation operation is designed. This operation can make full use of the location information of the captured remote sensing buildings, accurately locate the area of interest, and effectively capture the relationship between channels.
[0016] Specifically, the coordinated attention generation operation first concatenates the two feature maps generated by the previous module, and then transforms them using a shared 1x1 convolution, as shown in the following formula: It is the intermediate feature map of spatial information in the horizontal and vertical directions, where r represents the downsampling ratio.
[0017] Next, along the spatial dimension Split into two separate tensors and , and then use two convolution and The feature map and Transform to the same channel as the input and get the following result.
[0018] ( ( )) ( ( )) Then and Expanding on this, as the attention weight, the final output of the CA module can be expressed as follows: The specific structure of LMLP is as follows Figure 3 The LMLP module is an optimized and improved multi-layer perceptron (MLP). Its function is to capture the global long-term dependency relationship. Using this module, a more comprehensive and accurate feature representation of remote sensing buildings with large scale differences in remote sensing images can be obtained, thereby improving the accuracy of remote sensing building recognition and classification and more accurately locating remote sensing buildings.
[0019] In the backbone of the original YOLOv7 network, the ELAN module is used multiple times at different levels to extract multi-scale features of remote sensing buildings, and the size of the receptive field of the module is 3×3. The present invention believes that the ELAN module does not fully mine the multi-scale semantic information of the remote sensing building feature map. Inspired by the bottleneck layer and ELAN, a parallel multi-branch residual cascade MSR module with different receptive field sizes is designed to replace the ELAN module to enhance the model's ability to extract multi-scale features. The specific design of the MSR module is as follows: Figure 4As shown. Assume that X∈R^(H×W×C) is the input feature. After the transition of 1×1 convolution, the channel dimension of X increases to (n+1)×C. Then, we divide X into n+1 different groups, denoted as {X_i}, where i∈0,1,2,3,… In order to reduce the computational cost, n = 3 is selected. It should be noted that, except for X_0, each group passes through a Bottleneck layer and is cascaded through a residual structure. For X_0, it is allowed to retain information from the previous layers, and then it is added element-wise with the features after the bottleneck layer and the 1×1 convolution layer adjust the number of channels and connect the operation. Among them, c is the number of channels, k_i×k_i is the size of the convolution kernel, k_1 is 3, k_n>...>k_1, and n is the number of Bottleneck modules used.
[0020] The neck part of the original YOLOv7 network adopts the PAFPN structure, which fuses the feature information of different levels of the backbone network through top-down and bottom-up bidirectional feature propagation, thereby enhancing the fusion of semantic information and detail information of building features, providing richer and more accurate feature representations at different scales, and improving the expressive power of the feature pyramid. However, some small or even tiny buildings themselves have less feature information. In addition, although the shallow feature map contains rich building location information, the semantic information of the building is relatively small, that is, the feature propagation is still not sufficient. For this reason, the present invention designs a densely connected feature pyramid (DFP) module based on the PAFPN structure to further strengthen the combination of the location information of shallow features and the semantic information of deep features, and make up for the problem of missing semantic information of small remote sensing buildings in the shallow feature map.
[0021] The densely connected feature pyramid (DFP) module is as follows Figure 5 As shown in the figure, multiple information fusion paths are added on the basis of the PAFPN structure to make the feature propagation more sufficient and comprehensive, and the fused feature map has rich building semantic information and building location information. The specific steps of its construction are as follows: (1) The feature map output by SPPCSPC is transformed into a channel by the CBS module, and then upsampled by 4 times by the UPSAMPLE module and input into the Concat_2 module; (2) The feature map output by ELAN-W_3 is transformed through the CBS module, then upsampled by 2 times through the UPSAMPLE module and input into the Concat_5 module, and then transformed through the CBS module and output; (3) The feature map output by ELAN-W_3 is transformed into a channel by the CBS module, and then upsampled by 4 times by the UPSAMPLE module and input into the Concat_6 module; (4) The feature map output by ELAN-W_4 is transformed into a channel by the CBS module, and then upsampled by 2 times by the UPSAMPLE module and input into the Concat_6 module; (5) The feature map output by the Concat_6 module is output after the number of channels is transformed by the CBS module.
[0022] like Figure 6 As shown in the figure, the input module (Input) of the improved YOLOv7 network pre-processes the input image and scales the image to the default scale (640×640×3), and then inputs it to the backbone network (backbone). The scaled image is input to the backbone network, first passes through the CBS module for channel transformation and feature extraction, then passes through the PCA module to make the location information and features of the remote sensing buildings more prominent, and then passes through the MSR module to further mine the deep semantic information of the buildings in the feature map. The feature maps output by the backbone network at different levels will have richer location information and semantic information than the backbone network before the improvement. After these feature maps of different scales are input to the DFP module, more fusion paths will be used to make the features more fully fused, further strengthening the combination of the location information of shallow features and the semantic information of deep features, thereby greatly reducing the missing semantic information of small and medium-sized buildings in the shallow feature map.
[0023] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0024] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present application, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be based on the attached claims.
Claims
1. A remote sensing image building detection method, characterized in that: include: Step 1: Download remote sensing images of multiple regions from the Internet, divide the remote sensing images into separate images, annotate the buildings in the images using annotation software, create an initial data set, and enhance the diversity of samples in the data set by performing brightness transformation, random cropping, rotation, and scaling operations on the remote sensing images in the initial data set, and form several images as the final remote sensing image building data set; Step 2: Build a remote sensing image building detection algorithm network, which specifically includes inserting the perception coordinate attention module and the multi-scale residual module into the backbone network of YOLOv7, and improving the PAFPN structure of the YOLOv7 neck into a densely connected feature pyramid module; Step 3: Train the constructed remote sensing image building detection algorithm network on the training set and validation set in the remote sensing image building dataset, and optimize the parameters; Step 4: Input the test set in the remote sensing image building dataset into the remote sensing image building detection algorithm network that has been trained in step 3, and output the final detection result.
2. The remote sensing image building detection algorithm as claimed in claim 1, characterized in that: The perception coordinate attention module includes the following: The input of the Perceptual Coordinate Attention module is divided into three branches. The first branch is processed by the Coordinate Attention module, the second branch is processed by the Lightweight Multilayer Perceptron module, and the third branch retains the original image features. The outputs of the LMLP module and the CA module are concatenated together, and then the number of channels is reduced to the same as the input through 1x1 convolution, and then pixel-level point addition is performed with the third branch, and finally used as the output of the PCA module. The Perceptual Coordinate Attention (PCA) module is inserted into the backbone network of YOLOv7.
3. The remote sensing image building detection algorithm as claimed in claim 1, characterized in that: The multi-scale residual module is characterized by: The multi-scale residual module fully extracts multi-scale features by designing a parallel multi-branch residual cascade structure with different receptive field sizes. The multi-scale residual (MSR) module is inserted into the backbone network of YOLOv7.
4. The remote sensing image building detection algorithm as claimed in claim 1, characterized in that ,,The densely connected feature pyramid module is: (1) The feature map output by SPPCSPC is converted into a channel by the CBS module, and then upsampled by 4 times by the UPSAMPLE module and input into the Concat_2 module; (2) The feature map output by ELAN-W_3 is converted to a certain number of channels by the CBS module, then upsampled by a factor of 2 by the UPSAMPLE module and input to the Concat_5 module, and then converted to a certain number of channels by the CBS module before being output; (3) The feature map output by ELAN-W_3 is converted into a channel by the CBS module, and then upsampled by 4 times by the UPSAMPLE module and input into the Concat_6 module; (4) The feature map output by ELAN-W_4 is converted into a channel by the CBS module, and then upsampled by 2 times by the UPSAMPLE module and input into the Concat_6 module; (5) The feature map output by the Concat_6 module is output after the number of channels is transformed by the CBS module.