Wide-range remote sensing image rural residential area extraction method and system fusing target detection and visual attention mechanism

Abstract

The present application belongs to the technical field of residential area extraction, and discloses a large-range remote sensing image rural street-type residential area extraction method and system fusing target detection and visual attention mechanism, which comprises the following steps: generating a residential area candidate area detection frame on a large-range remote sensing image through a YOLOv7 model detection; and performing accurate extraction of the residential area based on the candidate area detection frame by using a PgNet model to obtain a rural residential area on the large-range remote sensing image. The present application combines the candidate mechanism of the target detection technology and the visual attention mechanism method, and roughly removes the vegetation and terrain, which is conducive to the rapid and automatic extraction of the rural residential area and the buildings thereof, and improves the search efficiency. After obtaining the candidate frame, the PgNet efficient extraction mechanism is used, so that the rural street-type residential area can be rapidly and automatically extracted in remote sensing images of different sizes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of residential area extraction, and particularly relates to a large-scale remote sensing image rural street residential area extraction method and system fusing target detection and visual attention mechanism. BACKGROUND

[0002] Rural street residential area is one of the most important artificial ground objects in rural areas, and street residential area is an important part of rural residential area. Rapid and accurate extraction and mapping of rural residential area is of great significance to rural development and planning, rural land management and monitoring, rural population estimation and rural modernization process. The rural residential area on remote sensing image is a mixed settlement composed of houses, vegetation, roads and soil land, etc. Due to the differences in material structure and form of ground objects, the buildings of rural residential area produce different differences on remote sensing image. And the details of residential area in different sizes and different resolution images are different. When the ground object distribution is complex, a large amount of interference information is mixed into the detection network, and the extraction effect is poor. Traditional residential area extraction methods such as manual visual interpretation are time-consuming and laborious, and consume more manpower and material resources.

[0003] The human visual system has the ability to quickly and accurately locate the object or region of interest from a complex scene, which is called visual attention mechanism. Salient object detection (SOD) is a simulation of this mechanism, aiming to segment the most visually attractive object or region in a given image. In recent years, methods based on visual attention mechanism have been applied to remote sensing image residential area detection (many examples are given below). Related research summarizes the strong extraction ability of saliency detection method in road network extraction, house detection and crop classification, etc. PicaNet network identifies relevant context regions and constructs informative context features for each pixel, achieving better extraction effect at pixel level. The edge loss attention module proposed by BasNet network improves the perception ability of building edges. CtdNet designs a complementary three-edge decoding network to improve feature mining ability and promote the strengthening of structural feature information. F3Net network proposes cross feature module (CFM) and minimizes new pixel position perception loss, so as to realize region segmentation and extraction of accurate local details. ScfNet proposes spatial context feature (SCF) module to focus on solving the problem of adaptive feature learning in large-scale point cloud. PgNet proposes a staggered grafting structure, which uses Transformer and CNN backbone network to independently extract features from different resolution images, so as to solve the contradiction between the sampling depth and the receptive field range of the network.

[0004] Existing research shows that the widely used deep learning technology provides a new direction and possibility for remote sensing image residential area extraction, but there are still some problems. First, the research on residential area extraction mainly focuses on urban areas, and the research on rural areas is relatively less. Unlike urban blocks, rural block-style residential areas are relatively small and scattered settlements, and are distributed with vegetation and farmland. Therefore, directly using the extraction method of urban building area for feature extraction of rural residential area is not strong in pertinence and weak in applicability, and its effect still needs to be improved. Second, the residential area extraction method based on visual attention mechanism has obtained good results on small-scale images, but on large-scale images, there are too many interference factors, and the results are not ideal. Therefore, it is still necessary to further explore the efficient extraction method suitable for rural block-style residential areas. Third, the residential area recognition algorithm based on deep learning has relatively good detection effect and algorithm performance when extracting residential areas on small-scale images, but it also introduces complex model structure and large number of parameters. When detecting blocks on large-scale images, especially when extracting blocks on large-scale remote sensing images, the detection speed and the accuracy of single block extraction effect still need to be improved, and the processing efficiency and convenient deployment ability of the algorithm still need to be further optimized. SUMMARY

[0005] In view of the problems of poor adaptability and poor precision effect of rural block extraction in multi-scale remote sensing images, the present application provides a large-scale remote sensing image rural block-style residential area extraction method and system combining target detection and visual attention mechanism. The method adopts a two-step strategy of rough positioning and fine extraction. First, the YOLOv7 model in the deep learning method is used to detect and generate the residential area candidate area detection frame on the large-scale remote sensing image, and then the PgNet model is used to accurately extract the residential area based on the candidate area detection frame to obtain the rural residential area on the large-scale remote sensing image. Through the combination of the candidate mechanism of the target detection technology and the visual attention mechanism method, the vegetation and terrain are roughly removed, which is conducive to the rapid and automatic extraction of rural residential areas and buildings, and improves the retrieval efficiency. After obtaining the candidate frame, the PgNet efficient extraction mechanism is used, so as to realize the rapid and automatic extraction of rural block-style residential areas in different size remote sensing images.

[0006] In order to achieve the above purpose, the present application adopts the following technical scheme:

[0007] The present application provides a large-scale remote sensing image rural block-style residential area extraction method combining target detection and visual attention mechanism, which comprises:

[0008] The YOLOv7 model is used to detect and generate the residential area candidate area detection frame on the large-scale remote sensing image;

[0009] The PgNet model is used for accurate extraction of the residential area based on the candidate area detection frame, so as to obtain the rural residential area on a large range of remote sensing images.

[0010] Further, the detection of the candidate area detection frame of the residential area on the large range of remote sensing images by the YOLOv7 model comprises the following steps.

[0011] The YOLOv7 model parameter weight is pre-trained by using the open source data set CBDV1.0.

[0012] The YOLOv7 model is used for detection of a test area, so as to obtain the rough positioning of the candidate residential area boundary frame on the large range of remote sensing images, and record the possibility of the street block, and the other parts of the image except the street block are non-street block backgrounds.

[0013] Through the rough positioning, the candidate frame area is sliced, and the vegetation and non-residential area information are removed.

[0014] Further, the accurate extraction of the residential area based on the candidate area detection frame by the PgNet model comprises the following steps.

[0015] The PgNet is used for detection of the generated candidate detection area.

[0016] The PgNet detection generates a saliency map, and the binary image corresponding to the saliency map is obtained by best threshold segmentation.

[0017] In combination with the original input image, the binary mask processing is performed based on the above binary image, so as to obtain the final rural street block residential area.

[0018] Another aspect of the present application provides a large range of remote sensing image rural street block residential area extraction system fusing target detection and visual attention mechanism, comprising:

[0019] The coarse extraction unit is used for detecting the candidate area detection frame of the residential area on the large range of remote sensing images by the YOLOv7 model.

[0020] The PgNet model is used for accurate extraction of the residential area based on the candidate area detection frame, so as to obtain the rural residential area on a large range of remote sensing images.

[0021] Further, the coarse extraction unit is specifically used for:

[0022] The YOLOv7 model parameter weight is pre-trained by using the open source data set CBDV1.0.

[0023] The pre-trained YOLOv7 model is used for detecting the test area, so as to obtain the rough positioning of the candidate residential area boundary frame on a large range of remote sensing images, and record the possibility of the frame being a block, and other parts of the image except the block are non-block backgrounds;

[0024] Through rough positioning, the slice candidate frame area is cut off, and the vegetation and non-residential area information is removed.

[0025] Further, the fine extraction unit is specifically used for:

[0026] PgNet is used to detect the generated candidate detection area;

[0027] The PgNet detection result is a saliency map, and the corresponding binary image is obtained by best threshold segmentation of the saliency map;

[0028] In combination with the original input image, the binary mask processing is performed based on the above binary image, so as to obtain the final rural block residential area.

[0029] Compared with the prior art, the present application has the beneficial effects:

[0030] The visual attention mechanism algorithm PgNet and the target detection algorithm YOLOv7 are combined, and a large range of remote sensing image rural block residential area extraction method and system are proposed, which combines target detection and visual attention mechanism. YOLOv7 is used as a rural block rough positioning detector, and the pain point of residential area positioning is solved through pre-retrieval mechanism; PgNet saliency detection algorithm is used for accurate extraction of the candidate area, and the problem of precision loss caused by scale change is further solved. Experiments are carried out on the CBDV1.0 group building data set and the self-built data set, and the feasibility and effectiveness of the method are verified. The experimental results show that the method can realize multi-scale remote sensing image block residential area extraction, and provide support for large range of remote sensing image block residential area surveying and mapping. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 It is a flowchart of a large range of remote sensing image rural block residential area extraction method combining target detection and visual attention mechanism according to an embodiment of the present application;

[0032] Figure 2 It is a structural schematic diagram of YOLOv7;

[0033] Figure 3 It is a rough positioning flowchart according to an embodiment of the present application;

[0034] Figure 4 It is a network structure schematic diagram of PgNet;

[0035] Figure 5A flowchart for accurate extraction of an embodiment of the present application;

[0036] Figure 6 An example graph for the CBDV1.0 dataset;

[0037] Figure 7 An example graph for the self-built dataset;

[0038] Figure 8 An example graph of the prediction results of different algorithm experiments;

[0039] Figure 9 A curve of the changes of Precision, F1 and Recall of the YOLOv7 algorithm in rough positioning;

[0040] Figure 10 A segmentation effect graph of the self-built dataset;

[0041] Figure 11 CBDV1.0 block resident extraction effect comparison;

[0042] Figure 12 An architecture schematic diagram of a system for extracting rural block residential areas from large-scale remote sensing images by fusing target detection and visual attention mechanisms according to an embodiment of the present application. DETAILED DESCRIPTION

[0043] The present application will be further explained and described below in conjunction with the accompanying drawings and specific embodiments:

[0044] As shown in Figure 1 A method for extracting rural block residential areas from large-scale remote sensing images by fusing target detection and visual attention mechanisms, which combines saliency detection and target detection to extract residential areas on large-scale remote sensing images, comprising:

[0045] First, a YOLOv7 model in a deep learning method is used to detect and generate candidate residential area bounding boxes on large-scale remote sensing images, and then a PgNet model is used to accurately extract residential areas based on the candidate residential area bounding boxes to obtain rural residential areas on large-scale remote sensing images.

[0046] Specifically, the method of embedding PgNet into YOLOv7 model is used. The study area is mainly concentrated in rural areas, and the research method is planned to be divided into two parts, rough positioning and fine extraction. First, the target detection technology is used to quickly obtain the rough positioning of the street block building boundary frame on a large range of remote sensing images, and record its possibility as a street block building. The candidate region detection frame is used as a preset sample for subsequent street block accurate extraction. Then, the method of saliency detection is used to detect the sample generated based on the candidate detection frame as the boundary, so as to obtain the accurate boundary of each street block building. Through rough positioning, the vegetation and most of the non-residential information are removed, which is conducive to the rapid automatic extraction of street blocks and improves the processing efficiency of detection. Through fine extraction, the problem of precision loss caused by scale change is further solved, the strong edge perception ability of street block building is maintained, the clear street block building contour boundary is obtained, and the pain point of rough positioning of residential area boundary is solved. The technical route is shown in Figure 1 .

[0047] 1 Rough extraction

[0048] 1.1 Overview of YOLOv7 algorithm

[0049] YOLOv7 is a new single-stage target detection network proposed by Alexey Bochkovskiy in July 2022. YOLO algorithm is the most typical representative of one-stage target detection algorithm, which is based on deep neural network for object recognition and positioning, and has higher running efficiency and higher detection accuracy.

[0050] The main features of YOLOv7 network( Figure 2 ) are as follows:

[0051] ① Model reparameterization: ensures that the parameter quantity in the network after parameter fusion is greatly reduced.

[0052] ② Self-adaptive multi-positive sample matching label assignment strategy simOTA: accelerates the training efficiency of the model, increases the number of positive samples, and is conducive to enhancing the expression of street block features and suppressing negative information.

[0053] ③ ELAN efficient network architecture: multi-input stacking structure for feature extraction, so that the network can learn more features in detection, and has stronger robustness and stability

[0054] ④ Use auxiliary branch to assist convergence: increase training cost and improve accuracy without affecting inference time, improve training efficiency without increasing time cost.

[0055] YOLOV7 is the most advanced algorithm in the YOLO series at present, surpassing the previous YOLO series in accuracy and speed. It is selected as the basis for rough positioning of rural street residential areas on a large range of remote sensing images.

[0056] 1.2 Rough positioning principle

[0057] Considering the balance between efficiency improvement and precision loss in algorithm design, the first step of algorithm design is to realize the rapid search of street candidate areas. A lightweight, fast and high-precision YOLOv7 detection algorithm is used as a candidate area detector. The process is as follows Figure 3 .

[0058] ① Use the pre-trained model parameter weight of the open source dataset CBDV1.0.

[0059] ② Use YOLOv7 to detect the test area to obtain the rough positioning of the candidate residential area boundary box on a large range of remote sensing images, and record its possibility as a street. Other parts of the image except the street are non-street backgrounds.

[0060] ③ Through rough positioning, slice the candidate box area, and remove vegetation and most non-residential information to facilitate rapid automatic extraction of streets and improve the processing efficiency of detection.

[0061] 2 Fine extraction

[0062] 2.1 PgNet overview

[0063] PgNet (as shown in Figure 4 ) is a single-stage framework jointly developed by Beijing University of Aeronautics and Astronautics and Pengcheng Laboratory in 2022, called pyramid grafting network (PgNet), which has the following characteristics:

[0064] ① Design a cross-model grafting module, use a double-branch encoder staggered connection to capture continuous semantics and rich details, and solve the precision loss problem of multi-scale changes.

[0065] ② Propose attention-guided loss supervision of CMGM-generated attention matrix to further promote feature grafting.

[0066] 2.1.1 Cross-model grafting module

[0067] The cross-model grafting module is used to fuse the features extracted by different backbone networks. Resnet and Transformer are used as double-branch encoders, and staggered connection is used to capture continuous semantics and rich details, fully inherit global information, realize the complementary of broken information of the two, and balance the contradiction of image features of different resolutions. Specifically, the cross-model grafting module flattens the features extracted by ResNet to f′ R∈v 1×C×HW Features extracted by the Swing Transformer Similarly, layer normalization and linear mapping are then applied to obtain three new features. Z is obtained through matrix multiplication, as shown in the following formula:

[0068]

[0069]

[0070] Then Z is linearly mapped and restored to v. H×W×C Then it goes through a convolutional layer.

[0071] 2.1.2 Attention-guided loss

[0072] Within the same image, neighborhood-based residential information features possess inherently stable semantic similarity and relatively uniform texture detail features. Furthermore, in small-scale images, neighborhood residential areas stand out more significantly than other background areas, meaning that neighborhood residential information has a higher weight in cross-attention. Specifically, for a salient mapping M of size H×W, it is flattened into a 1×HW map M′, and then matrix multiplication is applied to it to obtain the corresponding attention matrix M. a This process can be recorded as:

[0073] M a =F(M)

[0074]

[0075] in M represents a The value of the coordinates (x, y) in the middle. Let M' represent the value at coordinate (x, 1) in the transpose of the matrix M'. y Let G represent the value at coordinate (1, y) in M′. G is obtained using the transformation F(·). a SP a ,RP a Here, represents the truth mapping, and SP and RP are the significant intermediate predictions generated by features S2 and R5, respectively. We construct an attention-guided loss based on the weighted binary cross-entropy loss to supervise the CAM generated by CMGM. Therefore, L AG Can write

[0076]

[0077] Where β is a hyperparameter used to adjust the weight of ω.

[0078] 2.2 Principle of Fine Extraction

[0079] The human visual attention mechanism can quickly and accurately extract the region of interest from a complex scene, making it have good efficiency and robustness in small-scale image detection. Rural residential areas can be regarded as saliency targets in remote sensing images relative to large-scale vegetation and other information, so the method of saliency detection can be used in the study of rural residential area extraction. PgNet does not consider the basic attributes and local features of objects that have outstanding performance on the image, and effectively balances the contradictions between different resolution image inputs through the structure of staggered grafting, which has strong applicability in small-scale residential area detection.

[0080] ①After obtaining the block candidate frame, the essence of block detection is to distinguish block information and non-block information, PgNet network based on visual attention mechanism is used to detect the generated candidate detection area, and the problem of precision loss caused by scale change is further solved through fine extraction.

[0081] ②The PgNet accurate detection generates a saliency map, and the corresponding binary image is obtained by threshold segmentation of the saliency map, that is, the background is represented by a black background, and the target is represented by a white area.

[0082] ③Combined with the original input image, the binary mask processing based on the above binary image is carried out to obtain the final rural residential area.

[0083] The PgNet accurate extraction network structure process is as shown in Figure 5 .

[0084] In order to verify the effect of the present application, the following experiments are carried out:

[0085] 3. Experiment

[0086] 3.1 Experimental data set:

[0087] 3.1.1 CBDV1.0

[0088] The present application applies the CBDV1.0 clustered building detection dataset (Clustered Building Detection Dataset, CBDV1.0) as the target detection rough extraction dataset, and the example diagram is as shown in Figure 6 . The dataset contains 1564 samples labeled by hand, and the labeled content is rural block residential area. The image comes from the multispectral sensor of "Gaofen No. 2" satellite, and the spatial resolution is 4m. The size of the CBDV1.0 single sample is 680x720, which completely covers a geographical area of 506km2. In addition, these images are sampled from no less than 60 Chinese cities, and have high diversity.

[0089] 3.1.2 Self-built data set

[0090] The present application intends to use the self-built block data set as the pre-training data set of the accurate extraction algorithm model, and then use the trained pre-training weight as the initial weight to accurately detect the roughly framed rural block residential land. As a pre-training data set for accurate extraction, the quantity and accuracy of the data set required for rural residential land training are very high. The data set contains 1758 manually labeled samples, and the data source is aerial photographs. The annotation content is block, road, water system, and vegetation. As an implementable way, only the block label is extracted for experiment in the present embodiment. The size of a single sample is 600*600, and the spatial resolution is 3m. These images are sampled from more than 10 provinces such as Hebei, Henan, Shandong, and Shanxi, and an example is shown in Figure 7

[0091] 3.2 Experimental environment and parameter setting

[0092] The experimental operating system is Windows 11, the CPU version is 12th Gen Intel(R) Core(TM) i9-12900H, the GPU is NVIDIA GeForce RTX3080TI, and the deep learning framework is torch1.11.0+cu115. The experiment adopts the transfer learning strategy, uses the pre-training weight of the self-built rural block data set for training, and the network model improves the accuracy of the detection result by relying on its unsaturated continuous learning ability through transfer learning. The rough positioning training process is divided into a freezing stage and a thawing stage. The rough positioning training configuration information is shown in Table 1. The accurate extraction training configuration is shown in Table 2.

[0093] Table 1: Rough positioning training configuration

[0094]

[0095] Table 2: Accurate extraction training configuration

[0096]

[0097] 3.3 Experiment

[0098] 3.3.1 Comparison experiment of rough positioning algorithm

[0099] In order to verify the practicability and effectiveness of the selected YOLOv7 algorithm for rough positioning, several widely used deep learning target detection algorithms such as YOLOv3, YOLOv4, YOLOv5, and YOLOX are selected for comparison.

[0100] Figure 8 ​YOLOv3, YOLOv4, YOLOv5, original YOLOX algorithm and YOLOv7 algorithm experimental prediction results are displayed, original dataset label information is visualized on the image, and the performance of different algorithms is evaluated:

[0101] (1) Classification accuracy is improved: from the analysis of the prediction information, for the street residential information, the determination probability of the algorithm is improved compared with other algorithms, which ensures the accuracy of target detection, and the classification accuracy of the model network is higher.

[0102] (2) The network is more suitable for binary information classification and detection: the performance of YOLOv7 algorithm is balanced, which can identify and detect different scale residential areas, and can accurately classify and regress. The classification and detection performance is more comprehensive than other algorithms, and the applicability is stronger and the effect is better.

[0103] (3) The detection performance is stronger: on the basis of calibrating residential areas in the original dataset, the improved network additionally identifies scattered residential areas in the image. Compared with other algorithms, the number of labels predicted by YOLOv7 algorithm increases significantly, which means that the scale of residential areas detected by YOLOv7 algorithm is larger, the classification accuracy of residential and non-residential information is higher, and the performance of positive samples in classification and detection is stronger.

[0104] (4) The detection ability in complex background is stronger: when the image tone tends to be consistent and the vegetation types are mixed, YOLOv7 network can identify more scattered residential information in the complex background area indicated by the arrow compared with other networks, and can focus on the information of interest. The effect is better in the image with complex background information.

[0105] The screening effect of YOLOv7 on rural residential areas in remote sensing images is explored, the CBDV1.0 dataset is tested on the YOLOv7 network, and F1 score, Precision accuracy and Recall (recall rate) are used for evaluation, as shown in Table 3, Figure 9 The F1 score considers the influence of accuracy and recall rate; the accuracy value refers to the percentage of the number of targets that the model can correctly predict the street and the real street; the recall rate refers to the percentage of the real street that is correctly predicted. The experimental results prove that YOLOv7 network has strong screening ability in rough positioning, and in comparison with other mainstream detection networks, the F1 score is improved by 3.53%-9.96%, the Precision accuracy is improved by 39.68%-84.81%, and the Recall (recall rate) is improved by 8%-69%. Through data measurement and verification, YOLOv7 detection algorithm can better frame the most suitable prediction area in the street positioning of CBDV1.0 dataset than other mainstream detection algorithms, and can realize the positioning of most street residential targets, providing a good candidate frame for subsequent accurate extraction.

[0106] Table 3: Comparison of experimental accuracy (%) of different algorithms

[0107]

[0108] 3.3.2 Exploring the Effects of Attention-Guided Loss

[0109] To enhance the representation of residential area feature information and guide the stacked network to better achieve interaction of feature information from different sources, giving residential area information a higher weight in cross-attention, attention-guided loss was used to assist this process. In the experiments, the effectiveness of the attention loss mechanism was evaluated using several metrics: mIoU (mean Intersection over Union), Accuracy, F1 score, Precision, and Recall. mIoU refers to the average percentage of the intersection and union of predicted block pixels with real block pixels; Accuracy represents the percentage of pixels that correctly predict houses out of all pixels; F1 score considers both precision and recall; Precision refers to the percentage of pixels that the model can correctly predict in both blocks and real blocks; and Recall refers to the percentage of real blocks that are correctly predicted. The results are shown in Table 4.

[0110] Table 4: Comparison of Experimental Precision in Attention-Guided Loss

[0111]

[0112] 3.3.3 Comparison Experiment of Precise Extraction Algorithms

[0113] In the street block semantic segmentation dataset, analysis is performed based on high-precision pixel-level labels to accurately represent the extent, distribution, and geometric contours of buildings on the images. In the experiments of this invention, the label information of the original dataset is visualized on the images and compared with the experimental results. To verify the effectiveness of this improved algorithm, several classic mainstream visual attention mechanism algorithms such as BasNet, F3Net, PicaNet, CtdNet, and ScfNet were selected for comparison with the algorithm of this invention. The comparison results are as follows:

[0114] Table 5: Comparison of Experiments with Different Algorithms (%)

[0115]

[0116] The segmentation results on the self-built dataset are as follows: Figure 10 As shown, from Figure 10It can be known that the method has stronger feature mining capability compared with other visual attention mechanism algorithms, the probability of correct classification and identification is higher, the efficiency of filtering semantic information of block residential land is improved, and the integrity of the block residential land is further strengthened.

[0117] The block residential land accurate extraction effect on the CBDV1.0 large-scale remote sensing image dataset is as shown in the following table: Figure 11 As can be seen from the first row, the method can realize fine segmentation of block part and non-block information; as can be seen from the second row, the information detection rate is increased in the block extraction application of the algorithm; as can be seen from the third row and the fourth row, compared with other networks, the algorithm effectively suppresses the expression of negative information in the large-scale acquisition of non-block information; as can be seen from the fifth row, the algorithm can not only realize the detection of large-area blocks, but also has the identification ability of scattered houses in the block; as can be seen from the sixth row, the algorithm has strong contour regularization ability, realizes fine extraction of edge information, and has strong broken information capture ability.

[0118] In general, the algorithm has better edge integrity detection, and the boundary transition is more regular and smooth; effectively strengthens the construction of house feature information, effectively suppresses the missed detection and misdetected situation caused by the similarity between ground objects, has strong multi-scale generalization ability, and improves the detection rate of small block residential land; the retained detail feature information is relatively rich and complete, and the algorithm has strong distinguishing ability for block residential land and its surrounding background. The algorithm has stronger applicability in large-scale scene images, and performs better in large-scale and large-scale data, and shows stronger robustness in the face of complex buildings and variable environment, which verifies the effectiveness of the method.

[0119] Based on the above embodiment, as shown in the following table: Figure 12 The application further provides a large-scale remote sensing image rural block residential land extraction system fusing target detection and visual attention mechanism, comprising:

[0120] A coarse extraction unit is configured to generate a residential land candidate region detection frame on a large-scale remote sensing image by using a YOLOv7 model for detection;

[0121] A fine extraction unit is configured to use a PgNet model to perform accurate extraction of residential land based on the candidate region detection frame, so as to obtain rural residential land on the large-scale remote sensing image.

[0122] Further, the coarse extraction unit is specifically configured to:

[0123] Pre-train the YOLOv7 model parameter weight using the open source dataset CBDV1.0;

[0124] The pre-trained YOLOv7 model is used for detecting the test area, so as to obtain the rough positioning of the candidate residential area boundary frame on a large range of remote sensing images, and record the possibility of being a block, and other parts of the image except the block are non-block backgrounds;

[0125] Through rough positioning, the slice candidate frame area is cut off, and the vegetation and non-residential area information is removed.

[0126] Further, the fine extraction unit is specifically used for:

[0127] PgNet is used to detect the generated candidate detection area;

[0128] The PgNet detection result is a saliency map, and the corresponding binary image is obtained by best threshold segmentation of the saliency map;

[0129] In combination with the original input image, the binary mask processing is performed based on the above binary image, so as to obtain the final rural block residential area.

[0130] In summary, in order to realize the fast and accurate extraction of rural block residential areas in large range remote sensing images, the visual attention mechanism algorithm PgNet and the target detection algorithm YOLOv7 are combined, a large range remote sensing image rural block residential area extraction method and system are proposed, which are combined with target detection and visual attention mechanism, and the problems of poor adaptability and poor precision effect of multi-scale remote sensing image rural block extraction are solved. YOLOv7 is used as a rural block rough positioning detector, and the pain points of residential area positioning are solved through pre-retrieval mechanism, and the processing efficiency of the algorithm is improved; PgNet saliency detection algorithm is used for accurate extraction of the candidate area, and the problem of precision loss caused by scale change is further solved, and the strong edge perception ability of block type building group is maintained. Through testing on the CBDV1.0 block data set and the self-built data set, the experimental results show that the method has strong positioning perception ability and edge integrity for block type residential area compared with other traditional methods, can effectively capture clear block contour boundary, and solves the difficult problem of residential area accurate extraction. It can be applied to large range scene detection and intelligent judgment, and provides strong support for surveying and mapping production.

[0131] The above only shows the preferred embodiments of the present application, and it should be noted that for ordinary skilled persons in the technical field, some improvements and refinements can be made without departing from the principles of the present application, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A method of extracting rural street-based residential areas from large-scale remote sensing images by fusing object detection and visual attention mechanisms, characterized in that, The method comprises the following steps: generating a candidate area bounding box of a residential area on a large-scale remote sensing image through a YOLOv7 model detection; precisely extracting the residential area based on the candidate area bounding box by using a PgNet model to obtain a rural residential area on the large-scale remote sensing image; the step of generating the candidate area bounding box of the residential area on the large-scale remote sensing image through the YOLOv7 model detection comprises the following steps: pre-training the YOLOv7 model parameter weight by using an open-source data set CBDV1.0; detecting the test area by using the pre-trained YOLOv7 model to obtain a rough positioning of a candidate residential area bounding box on the large-scale remote sensing image, and record the possibility of the candidate residential area being a street block, and the other parts of the image except the street block being a non-street block background; through the rough positioning, slicing the candidate box area, and removing the vegetation and non-residential area information; the step of precisely extracting the residential area based on the candidate area bounding box by using the PgNet model to obtain the rural residential area on the large-scale remote sensing image comprises the following steps: detecting the generated candidate detection area by using the PgNet; generating a saliency map after the PgNet detection, and obtaining a corresponding binary graph by segmenting the saliency map by using an optimal threshold value; combining the original input image, performing binary mask processing based on the above binary graph, and obtaining a final rural street block residential area.

2. A system for extracting rural street pattern settlements from large-scale remote sensing images by fusing object detection and visual attention mechanism, characterized in that, The method comprises the following steps: a rough extraction unit is configured to generate a candidate area bounding box of a residential area on a large-scale remote sensing image through a YOLOv7 model detection; a precise extraction unit is configured to precisely extract the residential area based on the candidate area bounding box by using a PgNet model to obtain a rural residential area on the large-scale remote sensing image; the rough extraction unit is specifically configured to: pre-train the YOLOv7 model parameter weight by using an open-source data set CBDV1.0; detect the test area by using the pre-trained YOLOv7 model to obtain a rough positioning of a candidate residential area bounding box on the large-scale remote sensing image, and record the possibility of the candidate residential area being a street block, and the other parts of the image except the street block being a non-street block background; through the rough positioning, slicing the candidate box area, and removing the vegetation and non-residential area information; the precise extraction unit is specifically configured to: detect the generated candidate detection area by using the PgNet; generate a saliency map after the PgNet detection, and obtain a corresponding binary graph by segmenting the saliency map by using an optimal threshold value; combine the original input image, perform binary mask processing based on the above binary graph, and obtain a final rural street block residential area.

Images