A method, system, device, and medium for SAR image road extraction based on DeepLabv3+ and adaptive connected component labeling.
By using DeepLabv3+ and adaptive connected component labeling, the problem of road breakage in SAR image road segmentation was solved, achieving efficient and accurate road extraction results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XIDIAN UNIV
- Filing Date
- 2024-02-26
- Publication Date
- 2026-05-26
Smart Images
Figure CN118135396B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image road extraction technology, specifically relating to a SAR image road extraction method, system, device and medium based on DeepLabv3+ and adaptive connected component labeling. Background Technology
[0002] Synthetic Aperture Radar (SAR) is a crucial technology in remote sensing. It utilizes the synthetic aperture principle of radar for remote sensing observation, offering all-weather, all-day Earth observation capabilities. Since its inception, its unique operational characteristics have compensated for the susceptibility of optical images to weather and illumination variations, overcoming the limitations of optical remote sensing influenced by external conditions such as weather. This has led to its irreplaceable role in remote sensing. Furthermore, as a novel type of microwave imaging radar, SAR possesses rich characteristic signals, including amplitude, phase, and polarization information, making it an indispensable remote sensing tool for Earth observation.
[0003] As a crucial ground feature in remote sensing, road extraction is a highly popular research topic. Because SAR image road segmentation is closely related to important areas such as urban planning, residential life, and GIS database updates, it has always been a research hotspot in satellite image interpretation. Its applications are also extensive; timely and comprehensive road traffic information systems can play a vital role in many fields, including traffic navigation, urban planning, agriculture and forestry, and autonomous flight. With continuous development, human methods of observing the Earth have become increasingly sophisticated. The application of high-resolution satellites, aerial drones, imaging radar, and other equipment has led to a massive increase in available data. In this context, how to utilize this massive amount of data to quickly and comprehensively extract road information to update geographic information systems has become a research hotspot in this field.
[0004] In SAR image road extraction, high-quality labels are typically required for supervised model training. However, due to the large size and high noise levels of SAR images, obtaining high-quality labels is extremely resource-intensive. When training a network using a limited existing labeled SAR road dataset, the road segmentation results often show broken sections. Therefore, there is an urgent need to utilize efficient post-processing algorithms to connect these broken sections.
[0005] Patent application CN116452987A discloses a road information extraction method based on a self-attention model and a convolutional neural network. This method uses an encoder based on a road segmentation model to compress the spatial resolution of the original data image and extract features. It then uses a decoder to generate pixel connectivity structure prediction results and road segmentation results. Based on these results, it reverse-engineers an output result with the same resolution as the input image, and combines the road segmentation results to obtain the final output, which can improve the fragmented road segmentation problem. However, this method has drawbacks: it requires a large amount of manually labeled high-resolution remote sensing data, meaning it needs a large amount of labeled data to achieve this, and road fragmentation still exists in the network result image.
[0006] Patent application CN117078943B discloses a remote sensing image road segmentation method that integrates multi-scale features and a dual attention mechanism. This method simultaneously applies attention weighting to feature maps output from multiple stages of the encoder module from both spatial and channel perspectives. A context extraction module is set between the encoder module and the multi-scale decoder to capture global and local contextual information from high-level feature maps, thereby improving the performance of semantic segmentation of remote sensing images. The drawback of this method is that it only applies to road targets in optical remote sensing images. Since SAR remote sensing image features are significantly different from optical images, this method has low robustness and poor segmentation accuracy when dealing with SAR images.
[0007] In summary, existing technologies using supervised learning methods often require a significant investment of manpower and resources for road target labeling. When using limited labeled data and training and testing with only CNN networks, the resulting images often contain numerous breaks for roads, a specific type of ground feature. This results in poor segmentation accuracy for road targets, failing to reflect the actual characteristics of the ground features. Summary of the Invention
[0008] To overcome the shortcomings of the prior art, the present invention aims to provide a method, system, device and medium for SAR image road extraction based on DeepLabv3+ and adaptive connected component labeling. By applying a connected component labeling module, a single-pixel dynamic planning connection module and an adaptive road widening module, the broken parts of the CNN prediction result map are connected, thereby improving the accuracy of SAR image road segmentation.
[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0010] A SAR image road extraction method based on DeepLabv3+ and adaptive connected component labeling includes the following steps:
[0011] Step 1: Acquire SAR remote sensing images, preprocess the SAR remote sensing images to form a SAR image road dataset, divide the SAR image road dataset into a training set, a validation set and a test set, and perform data augmentation on the training set.
[0012] Step 2: Input the training set augmented in Step 1 into the DeepLabv3+ semantic segmentation network for training. The DeepLabv3+ semantic segmentation network consists of a feature encoding module and a feature decoding module. The DeepLabv3+ semantic segmentation network uses the ResNet101 network as the backbone network. Input the test set into the trained DeepLabv3+ semantic segmentation network to obtain the test result image.
[0013] Step 3: Build a road post-processing algorithm framework based on the adaptive connected component labeling algorithm. The road post-processing algorithm framework consists of a connected component labeling module, a dynamic programming connection module, and an adaptive road widening module.
[0014] Step 4: Using the road post-processing algorithm framework built in Step 3, connect the road breaks that appear in the test result image obtained in Step 2 to obtain the final road segmentation result.
[0015] The specific method for step 1 is as follows:
[0016] Step 1.1: Acquire SAR remote sensing images;
[0017] Step 1.2: Perform multi-view processing, orthorectification and depth transformation on the SAR remote sensing images obtained in Step 1.1. Manually label the SAR remote sensing images after depth transformation with road tags to obtain the SAR image road dataset.
[0018] Step 1.3: Divide the SAR image road dataset obtained in Step 1.2 into a training set, a validation set, and a test set;
[0019] Step 1.4: Perform data rotation augmentation on the training set obtained in Step 1.3, including rotations of 90°, 180°, and 270°, to obtain the data-augmented training set.
[0020] The specific method for step 2 is as follows:
[0021] In the DeepLabv3+ semantic segmentation network, in the feature encoding module, the input training set after data augmentation in step 1.4 is processed by the ResNet101 backbone network to extract features, generating two effective feature layers: a shallow feature layer and a deep feature layer. This deep feature layer is then transmitted to the ASPP module, which includes a 1×1 convolution, three 3×3 dilated convolutions, and a global pooling operation. The convolution, dilated convolution, and global pooling operation in the ASPP module process the deep features output by the ResNet101 backbone network to capture contextual information at different scales.
[0022] In the feature decoding module, the shallow features output by the Resnet101 backbone network and the output features of the ASPP module are transformed into the same shape, connected together, and then convolved to obtain the output.
[0023] The augmented training set from step 1.4 is then input into the DeepLabv3+ semantic segmentation network for training. During training, the cross-entropy loss function L is used. cro The details are as follows:
[0024] Cross-entropy loss L cro :
[0025]
[0026] Where C represents the number of categories, y i p represents the true label set for the i-th class. i This represents the prediction result for class i. After training, the trained model parameters are saved. The saved trained model parameters are validated on the validation set obtained in step 1.3. The model parameter with the highest index on the validation set is selected and loaded into the DeepLabv3+ semantic segmentation network. The test set obtained in step 1.3 is input into the trained DeepLabv3+ semantic segmentation network to obtain the test result image.
[0027] The specific method for step 3 is as follows:
[0028] Step 3.1, Connected Component Labeling Module: This module distinguishes roads in different regions. The SAR image road dataset obtained in Step 1.2 contains labels. Figure 1 There are two categories: "Other" with a label value of 0 and "Road" with a label value of 1. The image with pixel values of 0 and 1 is called a binary road element image. Combining the labels, connected component labeling can be performed by scanning each image block only twice. The specific method is as follows:
[0029] First scan: Traverse the binary road element graph, access the label value of each pixel. If the label value of the current pixel is not 0, meaning that the point is a road element, then:
[0030] (1). If all the label values in the neighborhood of a pixel are 0, then assign a new label count_label to the current pixel, and count_label = count_label + 1;
[0031] (2). If there are non-zero label values in the neighborhood, the minimum value among them is assigned to the pixel. Then, using a disjoint-set data structure, all the values in the neighborhood of the pixel are connected and identified as the same connected component.
[0032] The second scan: Traverse each non-zero pixel and use the find method of the established disjoint-set data structure to set the value of each non-zero pixel to the value of the corresponding root node; after the scan is completed, pixels with the same label value in the image form the same connected component.
[0033] Step 3.2, Dynamic Programming Connection Module: Connect different connected components using single pixels, as follows:
[0034] Given two connected regions A and B, calculate the shortest distance between them, the coordinates of the two pixels to be connected, and the width of the connection. In connected region A, arbitrarily select a pixel P1. Calculate the shortest distance from P1 to B, and denote the pixel with the shortest distance to P1 as Q1. Q1 belongs to connected region B. Then calculate the shortest distance from Q1 to connected region A, and denote the pixel with the shortest distance to Q1 as P2. P2 belongs to connected region A. Repeat this process until two pixels P1 and P2 are finally found. * and Q * , where P * Belonging to the connected region A, Q * If a region is connected (B), let the distance between two points be denoted as . when When the distance between two connected regions is less than the threshold, it is considered that the two regions can be connected. The above L is the threshold value set. A threshold value is set so that when the distance between two connected regions is less than the threshold value, the two regions can be considered to be connected.
[0035] Calculate the required width for subsequent road connection widening. The specific steps are as follows:
[0036] The coordinates of all points in each connected region were recorded, and n points were selected at equal intervals from these points: P1, P2, ..., Pn. nFor each point, a search is performed in four directions: up, down, left, and right. Four values L1, L2, L3, and L4 are initialized to 1 for each direction. For example, if the search proceeds in the up direction, if the next pixel is not 0, then L1 = L1 + 1; if it is 0, the search in this direction terminates. The final search distance values L1, L2, L3, and L4 for each point are obtained. Taking point P1 as an example, the final road width searched for this point is... That is, the minimum distance searched in four directions is multiplied by 2; this operation is performed on all n points, and a road width can be obtained for each point. Then let That is, the maximum value of the road width obtained from n points is taken as the road width of the current connected component;
[0037] Finally, the road width of each connected region is obtained. For example, if there are two connected regions A and B, then the road widths of the two connected regions are W and W respectively. A W B W AB =min(W A W B The width of the road that needs to connect these two connected areas;
[0038] Ultimately, four values are obtained between each pair of connected regions: the shortest distance between them, the coordinates of the two pixels, and the width of the road to be connected. When the shortest distance is less than the set threshold, the two connected regions are connected by a single pixel, and the connected "segment" is marked. During the connection process, the coordinates of the two pixels are used to connect them using the formula y = ax + b. At the break, a single pixel is connected.
[0039] Step 3.3, Adaptive Road Widening Module: Widens the connection points of single pixels;
[0040] Road widening is performed using templates. In step 3.2, the required road width for connecting every two connected regions was recorded. The road width values are in the form of 2, 4, 6, 8, 10, etc. For each width, a template is automatically matched for widening. Road width 2 corresponds to a 2×2 template, road width 4 corresponds to a 3×3 template, road width 6 corresponds to a 5×5 template, road width 8 corresponds to a 7×7 template, and road width greater than or equal to 10 corresponds to a 9×9 template. The single-pixel connection "segments" marked between every two connected regions are traversed. Based on the required road width between these two connected regions, the corresponding template is automatically matched, and all other points within the pixel template are set as road elements.
[0041] This invention also provides a SAR image road extraction system based on DeepLabv3+ and adaptive connected component labeling, comprising:
[0042] SAR image road dataset processing module: used to acquire SAR remote sensing images, preprocess SAR remote sensing images to form SAR image road dataset, divide SAR image road dataset into training set, validation set and test set, and perform data augmentation on training set.
[0043] Test result image acquisition module: This module is used to input the augmented training set into the DeepLabv3+ semantic segmentation network for training. The DeepLabv3+ semantic segmentation network consists of a feature encoding module and a feature decoding module. The DeepLabv3+ semantic segmentation network uses the ResNet101 network as the backbone network. The test set is input into the trained DeepLabv3+ semantic segmentation network to obtain the test result image.
[0044] Road post-processing algorithm framework building module: This module is used to build a road post-processing algorithm framework based on the adaptive connected component labeling algorithm. The road post-processing algorithm framework consists of a connected component labeling module, a dynamic programming connection module, and an adaptive road widening module.
[0045] Final road segmentation result acquisition module: This module is used to connect the road breaks that appear in the test result image using the established road post-processing algorithm framework to obtain the final road segmentation result.
[0046] This invention also provides a SAR image road extraction device based on DeepLabv3+ and adaptive connected component labeling, comprising:
[0047] Memory: A computer-readable device that stores the above-mentioned SAR image road extraction method based on DeepLabv3+ and adaptive connected component labeling;
[0048] Processor: Used to implement the SAR image road extraction method based on DeepLabv3+ and adaptive connected component labeling when executing the computer program.
[0049] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, enables the implementation of the SAR image road extraction method based on DeepLabv3+ and adaptive connected component labeling.
[0050] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0051] 1. This invention targets roads in SAR imagery as a special ground feature. By creating a dedicated dataset and using data rotation to expand a small amount of labeled data (training set), the model can fully extract the spatial and semantic features of SAR roads. It also makes full use of the high-quality road dataset and addresses road breaks in the CNN network prediction results by using a post-processing algorithm based on adaptive connected component labeling to adaptively connect the road breaks, thereby improving the accuracy of road extraction.
[0052] 2. This invention, through its designed connected component labeling module and dynamic programming connection module, can distinguish different broken regions as different connected components for road breaks in CNN network prediction results, and connect single pixels.
[0053] 3. The present invention, through its designed adaptive road widening module, can calculate the required road width between different connected domains at road junctions, and adaptively match different templates to widen the road at the junctions of single pixels, so that the final road segmentation result is more consistent with the actual road features.
[0054] In summary, this invention utilizes a high-quality dataset to address the issues of poor SAR image road target segmentation accuracy, misclassification, and road fragmentation when using CNN networks for training and testing with only a limited amount of labeled data. By applying connected component labeling, dynamic programming connections for single pixels, and adaptive road widening to the images, broken sections are connected, and the connections are adaptively widened, resulting in smooth road edges and extracted roads that more closely resemble reality. This invention solves the technical problems of incomplete road extraction and obvious road fragmentation in existing CNN-based SAR road segmentation algorithms. Attached Figure Description
[0055] Figure 1 This is a flowchart illustrating the implementation of the present invention.
[0056] Figure 2 This involves rotating SAR images and their corresponding labels from 3m resolution training data, including rotations of 90°, 180°, and 270°. Figure 2 (a) is a schematic diagram of the unrotated raw SAR data and its corresponding labels. Figure 2 (b) is a schematic diagram of a 90° rotation. Figure 2 (c) is a schematic diagram of a 180° rotation. Figure 2 (d) is a schematic diagram of a 270° rotation.
[0057] Figure 3 This is a neighborhood diagram used by the connected component labeling module; where, Figure 3 (a) is a schematic diagram of a 2-neighborhood. Figure 3 (b) is a schematic diagram of a 3-neighborhood. Figure 3 (c) is a schematic diagram of a 4-neighborhood.
[0058] Figure 4 These are schematic diagrams illustrating different templates used in the adaptive road widening module; among them, Figure 4 (a) is a schematic diagram of a 2×2 template. Figure 4 (b) is a schematic diagram of a 3×3 template. Figure 4 (c) is a schematic diagram of a 5×5 template.
[0059] Figure 5 This is a comparison image showing the connection of road breaks in CNN result images using the method of this invention; wherein, Figure 5 (a) is the CNN result image. Figure 5 (b) is a graph showing connected components. Figure 5 (c) is a single-pixel connectivity graph. Figure 5 (d) shows the result of widening the connection. Figure 5 (e) is a diagram showing the results of the method of the present invention. Detailed Implementation
[0060] The following description, in conjunction with the accompanying drawings, further illustrates examples and effects of the present invention.
[0061] like Figure 1 As shown, a SAR image road extraction method based on DeepLabv3+ and adaptive connected component labeling includes the following steps:
[0062] Step 1: Acquire SAR remote sensing images, preprocess the SAR remote sensing images to form a SAR image road dataset, divide the SAR image road dataset into a training set, a validation set and a test set, and perform data augmentation on the training set.
[0063] The specific method for step 1 is as follows:
[0064] Step 1.1: Acquire 3m resolution SAR remote sensing images. The SAR images were acquired using the Gaofen-3 remote sensing satellite.
[0065] Step 1.2: The 3m resolution SAR remote sensing image obtained in Step 1.1 is processed by multi-view processing, orthorectification and depth conversion using professional remote sensing processing software. The Image Labeler tool in MATLAB software is used to manually label the SAR remote sensing image after depth conversion to obtain the SAR image road dataset, which includes 1107 image blocks of 512×512 size.
[0066] Step 1.3: Divide the SAR image road dataset obtained in Step 1.2 into a training set, a validation set, and a test set in a ratio of 8:1:1. The dataset is still insufficient and needs to be expanded.
[0067] Step 1.4: Due to the special nature of SAR data, general data augmentation methods are no longer applicable. For the special target of roads in SAR images, the training set obtained in Step 1.3 is augmented by data rotation, including rotations of 90°, 180° and 270°, to obtain the augmented training set.
[0068] like Figure 2 The diagram shown is a schematic diagram of the rotation of the training set data. The first column corresponds to the original SAR image and its corresponding label, and the following three columns correspond to the schematic diagrams of three rotation angles of 90°, 180° and 270° respectively.
[0069] Step 2: Input the data augmentation training set from Step 1.4 into the DeepLabv3+ semantic segmentation network for training. The DeepLabv3+ semantic segmentation network consists of a feature encoding module and a feature decoding module. In this invention, the DeepLabv3+ semantic segmentation network uses the ResNet101 network as the backbone network. Input the test set into the trained DeepLabv3+ semantic segmentation network to obtain the test result image.
[0070] The specific method for step 2 is as follows:
[0071] In the DeepLabv3+ semantic segmentation network, in the feature encoding module, the input training set after data augmentation in step 1.4 is processed by the ResNet101 backbone network to extract features, generating two effective feature layers: a shallow feature layer and a deep feature layer. This deep feature layer is then transmitted to the ASPP module, which includes a 1×1 convolution, three 3×3 dilated convolutions, and a global pooling operation. The convolution, dilated convolution, and global pooling operation in the ASPP module process the deep features output by the ResNet101 backbone network to capture contextual information at different scales.
[0072] In the feature decoding module, the shallow features output by the Resnet101 backbone network and the output features of the ASPP module are transformed into the same shape, connected together, and then convolved to obtain the output.
[0073] The augmented training set is input into the DeepLabv3+ semantic segmentation network for training. The cross-entropy loss function L is used during training. cro The details are as follows:
[0074] Cross-entropy loss L cro :
[0075]
[0076] Where C represents the number of categories, y i p represents the true label set for the i-th class. i This represents the prediction result for class i. After training, the trained model parameters are saved. The saved trained model parameters are validated on the validation set obtained in step 1.3. The model parameter with the highest index on the validation set is selected and loaded into the DeepLabv3+ semantic segmentation network. The test set obtained in step 1.3 is input into the trained DeepLabv3+ semantic segmentation network to obtain the test result image.
[0077] For roads, a special type of geographical feature, relying solely on network testing of test data results in incomplete extraction of road targets, leading to gaps in the data. To address this issue, this invention establishes a road post-processing algorithm framework based on an adaptive connected component labeling algorithm, which connects the gaps in the road network.
[0078] Step 3: Build a road post-processing algorithm framework based on the adaptive connected component labeling algorithm. The road post-processing algorithm framework consists of a connected component labeling module, a dynamic programming connection module, and an adaptive road widening module.
[0079] The specific method for step 3 is as follows:
[0080] Step 3.1, Connectivity Labeling Module: To connect road breaks, the first step is to distinguish roads in different areas. In the SAR image road dataset obtained in Step 1.2, the labels... Figure 1 There are two categories: "Other" with a label value of 0 and "Road" with a label value of 1. In this invention, an image where the pixel values of each pixel are only 0 and 1 is called a binary road element image. Combined with the labels, connected component labeling can be performed by scanning each image block only twice. The specific method is as follows:
[0081] First scan: Traverse the binary road element graph, access the label value of each pixel. If the label value of the current pixel is not 0, meaning that the point is a road element, then:
[0082] (1). If all the label values in the neighborhood of a pixel are 0, then assign a new label count_label to the current pixel, and count_label = count_label + 1;
[0083] (2). If there are non-zero label values in the neighborhood, the minimum value among them is assigned to the pixel. Then, using a disjoint-set data structure, all the values in the neighborhood of the pixel are connected and identified as the same connected component.
[0084] The second scan: Traverse each non-zero pixel and use the find method of the established disjoint-set data structure to set the value of each non-zero pixel to the value of the corresponding root node; after the scan is completed, pixels with the same label value in the image form the same connected component.
[0085] like Figure 3 As shown, there are three options for selecting the neighborhood: 2-neighborhood, 3-neighborhood, and 4-neighborhood. This invention uses 4-neighborhood, which can effectively distinguish different road connectivity regions in the CNN test result graph.
[0086] like Figure 5 As shown, Figure 5 (a) shows the output image of the CNN, where the road target shows a broken pattern. Figure 5 (b) marks the road as a connected region, and different connected regions are distinguished by different colors.
[0087] Step 3.2, Dynamic Programming Connection Module: After distinguishing connected components in the above steps, it is necessary to selectively perform single-pixel connections on different connected components. The specific steps are as follows:
[0088] Given two connected regions A and B, we need to calculate the shortest distance between them, the coordinates of the two pixels to be connected, and the width of the connection. In connected region A, arbitrarily select a pixel P1. Calculate the shortest distance from P1 to B, and denote the pixel with the shortest distance to P1 as Q1 (Q1 belongs to connected region B). Then calculate the shortest distance from Q1 to connected region A, and denote the pixel with the shortest distance to Q1 as P2 (P2 belongs to connected region A). Repeat this process until the two points P1 and P2 are finally found. * (belonging to connected region A) and Q * (Belonging to connected region B), let the distance between the two points be denoted as . when When the distance between two connected regions is less than the threshold, it is considered that connected region A and connected region B should be connected. The above L is the threshold value set. Not all connected regions need to be connected in pairs. A threshold value needs to be set. When the distance between two connected regions is less than the threshold value, it is considered that the two can be connected. In this embodiment of the invention, the threshold value L = 15 is set.
[0089] In addition to calculating the shortest distance between two connected components and the corresponding coordinates of the two points, it is also necessary to calculate the required width for widening at the connection points with subsequent roads. The specific steps are as follows:
[0090] The coordinates of all points in each connected region were recorded, and then 10 points were selected at equal intervals from these points: P1, P2, ... P 10For each point, a search is performed in four directions: up, down, left, and right. Four values L1, L2, L3, and L4 are initialized to 1 for each direction. For example, if the search proceeds in the up direction, if the next pixel is not 0, then L1 = L1 + 1; if it is 0, the search in this direction terminates. The final search distance values L1, L2, L3, and L4 for each point are obtained. Taking point P1 as an example, the final road width searched for this point is... That is, the minimum distance searched in four directions is taken and then multiplied by 2; this operation is performed on all 10 points, and a road width can be obtained for each point. Then let That is, the maximum value of the road width obtained from the 10 points is taken as the road width of the current connected domain.
[0091] The above steps yield the road width for each connected region. For example, if there are two connected regions A and B, then the road widths for the two connected regions are W and W respectively. A W B W AB =min(W A W B The width of the road that needs to connect these two connected areas.
[0092] Ultimately, four values are obtained between each pair of connected regions: the shortest distance between them, the coordinates of the two pixels, and the width of the road to be connected. When the shortest distance is less than the set threshold of 15, the two connected regions are connected by a single pixel, and the connected "segment" is marked. During the connection process, the coordinates of the two pixels are used to connect them using the formula y = ax + b.
[0093] like Figure 5 As shown, Figure 5 (b) marks the road as a connected region. Figure 5 (c) then selectively connects single pixels in different connected components, with the connection effect as follows: Figure 5 As shown in (c), individual pixels were connected at the break point.
[0094] Step 3.3, Adaptive Road Widening Module: The above operation only connected single pixels. However, when the road is very wide, the connection of single pixels does not conform to the actual situation. Therefore, it is necessary to widen the connection of single pixels.
[0095] The road widening is done using a template. In the process described above, the road width that needs to be connected between every two connected domains has been recorded. The road width values are in the form of 2, 4, 6, 8, 10, etc. For each width, a template is automatically matched for widening.
[0096] like Figure 4 As shown, these are different template diagrams: road width 2 corresponds to a 2×2 template, road width 4 corresponds to a 3×3 template, road width 6 corresponds to a 5×5 template, road width 8 corresponds to a 7×7 template, and road width greater than or equal to 10 corresponds to a 9×9 template. The single-pixel connecting "segments" marked between each pair of connected regions are traversed, and the corresponding template is automatically matched according to the road width that needs to be connected between the two connected regions. All other points in the pixel template are set as road elements.
[0097] like Figure 5 As shown, the specific widening result is as follows: Figure 5 (d) When the road is narrow, a smaller template is used to widen it, and when the road is wide, a larger template is used to widen it, thus realizing the adaptive road widening function.
[0098] Step 4: Using the road post-processing algorithm framework built in Step 3, connect the road breaks that appear in the test result image obtained in Step 2 to obtain the final road segmentation result.
[0099] The test result map output by CNN is processed by the connected component labeling module, the dynamic programming connection module, and the adaptive road widening module. Ultimately, it can connect the road breaks well, and the connection is smooth, which is consistent with the actual road conditions. Figure 5 (e) compared to Figure 5 (d) then the colors of the roads marked with other colors were unified. Figure 5 (e) compared to Figure 5 (a) The part selected by the red box shows that the width of the road break was adaptively widened to obtain the final segmentation result.
[0100] like Figure 5 As shown, Figure 5 (a) shows the output image of the CNN, which exhibits a fragmentation phenomenon when dealing with the road target. Figure 5 (b) involves marking the connected components of the road target, using different colors to distinguish different connected components, and then selectively connecting these components, resulting in the following connection effect: Figure 5 As shown in (c), the break was connected by individual pixels. For each pixel connection, a template was automatically matched to widen the connection. The specific widening result is shown below. Figure 5 (d) When the road is narrow, a smaller template is used to widen it; when the road is wide, a larger template is used to widen it, thus achieving adaptive road widening functionality. Figure 5 (e) is the final result diagram of the present invention. In response to the road breakage phenomenon in the original result diagram, the broken parts can be connected adaptively, making the result more in line with the visual effect and significantly improving the segmentation effect.
[0101] The effects of the present invention will be further explained below with reference to simulation experiments.
[0102] 1. Simulation experimental conditions:
[0103] The hardware testing platform for the simulation experiment of this invention is: CPU 48 cores 192G; graphics card Ascend 910 Premium A 32G.
[0104] The software platform for the simulation experiment of this invention is as follows: Operating system: EulerOS 2.0 (SP8); Online platform: ModelArts; Notebook integrated development environment, Python 3.7.6; Hardware platform: Ascend 910
[0105] Experimental Methods: A SAR image road extraction method based on DeepLabv3+ and adaptive connected component labeling.
[0106] 2. Simulation content and result analysis:
[0107] Figure 2 It involves rotating SAR images and their corresponding labels from 3m resolution training data, including rotations of 90°, 180°, and 270°.
[0108] Figure 3 This is a neighborhood diagram used by the connected region labeling module;
[0109] Figure 4 These are schematic diagrams illustrating the different templates used in the adaptive road widening module;
[0110] Figure 5 This is a comparison image showing the connection of road breaks in the CNN result image using the method of this invention.
[0111] The dataset used in the simulation experiment of this invention is a dataset covering the Xi'an and Baoji areas taken by the Gaofen-3 remote sensing satellite. It is randomly divided into training set, validation set and test set. The dataset contains 1107 image patches of 512×512 size, with an image resolution of 3m, polarization mode of HH and striping mode of UFS.
[0112] The overall quantitative evaluation results of the simulation experiment of this invention on the test set are shown in Table 1, and the quantitative evaluation results for the road category are shown in Table 2. From the tables, it can be seen that the Precision, Recall, and F1 score indicators for the road category are generally high, indicating a good segmentation effect for the road category. The overall qualitative evaluation results are shown in... Figure 5 , Figure 5 (a) shows the original CNN output. Figure 5(e) is the final result diagram of the method of the present invention. In response to the road breakage phenomenon in the original result diagram, the broken parts can be connected adaptively, making the result more in line with the visual effect and significantly improving the segmentation effect.
[0113] Table 1 Overall quantitative evaluation indicators of the present invention on the test set.
[0114] Evaluation indicators Method of the present invention OA 96.85％ Kappa 69.99％ MIoU 76.28％ FWIoU 94.37％
[0115] Table 2. Quantitative evaluation indicators of the road category on the test set for this invention.
[0116] Evaluation indicators Method of the present invention Precision 74.38％ Recall 69.13％ F1 score 71.66％
[0117] The simulation results above show that, compared with supervised CNN methods, the method of the present invention, through the post-processing method based on adaptive connected component labeling algorithm, significantly improves the road segmentation effect of SAR images; from Figure 5 As can be seen, the method of this invention connects the road breaks in the CNN output and adaptively widens the connections, so that the visual effect conforms to the actual ground features and ultimately achieves better segmentation performance.
[0118] This invention also provides a SAR image road extraction system based on DeepLabv3+ and adaptive connected component labeling, comprising:
[0119] SAR image road dataset processing module: used to collect SAR remote sensing images in step 1, preprocess the SAR remote sensing images to form a SAR image road dataset, divide the SAR image road dataset into training set, validation set and test set, and perform data augmentation on the training set.
[0120] Test result image acquisition module: This module is used to input the training set after data augmentation in step 1 into the DeepLabv3+ semantic segmentation network for training in step 2. The DeepLabv3+ semantic segmentation network consists of a feature encoding module and a feature decoding module. The DeepLabv3+ semantic segmentation network uses the ResNet101 network as the backbone network. The test set is input into the trained DeepLabv3+ semantic segmentation network to obtain the test result image.
[0121] Road post-processing algorithm framework building module: This module is used to build the road post-processing algorithm framework based on the adaptive connected component labeling algorithm in step 3. The road post-processing algorithm framework consists of a connected component labeling module, a dynamic programming connection module, and an adaptive road widening module.
[0122] Final road segmentation result acquisition module: This module is used to connect the road breaks that appear in the test result image obtained in step 2 using the road post-processing algorithm framework built in step 3 in step 4, and obtain the final road segmentation result.
[0123] This invention also provides a SAR image road extraction device based on DeepLabv3+ and adaptive connected component labeling, comprising:
[0124] Memory: A computer-readable device that stores the above-mentioned SAR image road extraction method based on DeepLabv3+ and adaptive connected component labeling;
[0125] Processor: Used to implement the SAR image road extraction method based on DeepLabv3+ and adaptive connected component labeling when executing the computer program.
[0126] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, enables the implementation of the SAR image road extraction method based on DeepLabv3+ and adaptive connected component labeling.
Claims
1. A SAR image road extraction method based on DeepLabv3+ and adaptive connected component labeling, characterized in that: Includes the following steps: Step 1: Acquire SAR remote sensing images, preprocess the SAR remote sensing images to form a SAR image road dataset, divide the SAR image road dataset into a training set, a validation set and a test set, and perform data augmentation on the training set. Step 2: Input the training set augmented in Step 1 into the DeepLabv3+ semantic segmentation network for training. The DeepLabv3+ semantic segmentation network consists of a feature encoding module and a feature decoding module. The DeepLabv3+ semantic segmentation network uses the ResNet101 network as the backbone network. Input the test set into the trained DeepLabv3+ semantic segmentation network to obtain the test result image. Step 3: Build a road post-processing algorithm framework based on the adaptive connected component labeling algorithm. The road post-processing algorithm framework consists of a connected component labeling module, a dynamic programming connection module, and an adaptive road widening module. Step 4: Using the road post-processing algorithm framework built in Step 3, connect the road breaks that appear in the test result image obtained in Step 2 to obtain the final road segmentation result. The specific method for step 3 is as follows: Step 3.1, Connected Component Labeling Module: This module distinguishes roads in different regions. In the SAR image road dataset, the label map has two categories: "Other" with a label value of 0 and "Road" with a label value of 1. The image with pixel values of 0 and 1 is called a binary road element map. Combined with the labels, connected component labeling can be performed by scanning each image block twice. The specific steps are as follows: First scan: Traverse the binary road element graph, access the label value of each pixel. If the label value of the current pixel is not 0, meaning that the point is a road element, then: (1) If all the label values in the neighborhood of a pixel are 0, then assign a new label to the current pixel. ,at the same time ; (2). If there are non-zero label values in the neighborhood, the minimum value among them is assigned to the pixel. Then, using a disjoint-set data structure, all the values in the neighborhood of the pixel are connected and identified as the same connected component. The second scan: Traverse each non-zero pixel and use the find method of the established disjoint-set data structure to set the value of each non-zero pixel to the value of the corresponding root node; after the scan is completed, pixels with the same label value in the image form the same connected component. Step 3.2, Dynamic Programming Connection Module: Connect different connected components using single pixels, as follows: Given two connected regions and connected regions Calculate the shortest distance between the two, the coordinates of the two pixels to be connected, and the width to be connected in the connected region. Select any pixel ,calculate arrive The shortest distance between them is denoted as . The shortest distance pixel is , Belongs to a connected region ; then calculate To connected regions The shortest distance, denoted by The shortest distance pixel is , Belongs to a connected region This process is repeated until two points are finally found. and ,in, Belongs to a connected region , Belongs to a connected region Let the distance between the two points be denoted as . ,when When, then it is considered a connected region. Connected regions The above should be connected. The threshold value set; Calculate the required width for subsequent road connection widening. The specific steps are as follows: The coordinates of all points in each connected region were recorded, and n points were selected at equal intervals from these points: For each point, perform a search in four directions: up, down, left, and right, and initialize four values for each direction. , , , All are initialized to 1. If the search proceeds upwards, and the next pixel is not 0, then... If the value is 0, the search in this direction ends, and the search distance values in the four directions of the point are finally obtained. , , , ; With point For example, the final road width searched at this point is , , , This involves taking the minimum distance from each of the four directions and multiplying it by 2; this operation is performed on all n points, resulting in a road width for each point. Then let That is, the maximum value of the road width obtained from n points is taken as the road width of the current connected domain; Finally, the road width of each connected component is obtained, if there is , Given two connected regions, the road widths of the two connected regions are respectively ,make The required road width to connect these two connected areas; Ultimately, each pair of connected regions will yield four values: the shortest distance between them, the coordinates of the two pixels, and the width of the road to be connected. When the shortest distance is less than a set threshold, the two connected regions are connected by a single pixel, and the connected "segment" is marked. The connection process utilizes the coordinates of the two pixels, expressed as a formula. The connection was made by linking individual pixels at the break point; Step 3.3, Adaptive Road Widening Module: Widens the connection points of single pixels; Road widening is performed using templates. In step 3.2, the required road width for connecting every two connected regions has been recorded. The road width values are in the form of 2, 4, 6, 8, 10, etc. For each width, a template is automatically matched for widening. Road width 2 corresponds to a 2×2 template, road width 4 corresponds to a 3×3 template, road width 6 corresponds to a 5×5 template, road width 8 corresponds to a 7×7 template, and road width greater than or equal to 10 corresponds to a 9×9 template. The single-pixel connection "segments" marked between every two connected regions are traversed. Based on the required road width between these two connected regions, the corresponding template is automatically matched, and all other points in the pixel template are set as road elements.
2. The SAR image road extraction method based on DeepLabv3+ and adaptive connected component labeling according to claim 1, characterized in that: The specific method for step 1 is as follows: Step 1.1: Acquire SAR remote sensing images; Step 1.2: Perform multi-view processing, orthorectification and depth transformation on the SAR remote sensing images obtained in Step 1.
1. Manually label the SAR remote sensing images after depth transformation with road tags to obtain the SAR image road dataset. Step 1.3: Divide the SAR image road dataset obtained in Step 1.2 into a training set, a validation set, and a test set; Step 1.4: Perform data rotation augmentation on the training set obtained in Step 1.3, including rotations of 90°, 180°, and 270°, to obtain the data-augmented training set.
3. A SAR image road extraction method based on DeepLabv3+ and adaptive connected component labeling as described in claim 1 or 2, characterized in that: The specific method for step 2 is as follows: In the DeepLabv3+ semantic segmentation network, in the feature encoding module, the augmented training set of the input data is processed by the ResNet101 backbone network to extract features, generating two effective feature layers: a shallow feature layer and a deep feature layer. This deep feature layer is then transmitted to the ASPP module, which includes a 1×1 convolution, three 3×3 dilated convolutions, and a global pooling operation. The convolution, dilated convolution, and global pooling operations in the ASPP module process the deep features output by the ResNet101 backbone network to capture contextual information at different scales. In the feature decoding module, the shallow features output by the Resnet101 backbone network and the output features of the ASPP module are transformed into the same shape, connected together, and then convolved to obtain the output. The augmented training set is input into the DeepLabv3+ semantic segmentation network for training, and the cross-entropy loss function is used during the training process. The details are as follows: Cross-entropy loss : in, Indicates the number of categories. Indicates the first The actual label set by the class Indicates the first The prediction results of the class are obtained; after training, the trained model parameters are saved, and the saved trained model parameters are validated on the validation set. The model parameter with the highest index on the validation set is selected and loaded into the DeepLabv3+ semantic segmentation network. The test set is input into the trained DeepLabv3+ semantic segmentation network to obtain the test result image.
4. A SAR image road extraction system based on DeepLabv3+ and adaptive connected component labeling, used to implement the method of claim 1, characterized in that: include: SAR image road dataset processing module: used to acquire SAR remote sensing images, preprocess SAR remote sensing images to form SAR image road dataset, divide SAR image road dataset into training set, validation set and test set, and perform data augmentation on training set. Test result image acquisition module: This module is used to input the augmented training set into the DeepLabv3+ semantic segmentation network for training. The DeepLabv3+ semantic segmentation network consists of a feature encoding module and a feature decoding module. The DeepLabv3+ semantic segmentation network uses the ResNet101 network as the backbone network. The test set is input into the trained DeepLabv3+ semantic segmentation network to obtain the test result image. Road post-processing algorithm framework building module: This module is used to build a road post-processing algorithm framework based on the adaptive connected component labeling algorithm. The road post-processing algorithm framework consists of a connected component labeling module, a dynamic programming connection module, and an adaptive road widening module. Final road segmentation result acquisition module: This module is used to connect the road breaks that appear in the test result image using the established road post-processing algorithm framework to obtain the final road segmentation result.
5. A SAR image road extraction device based on DeepLabv3+ and adaptive connected component labeling, characterized in that: include: Memory: A device that stores computer programs and is readable by a computer; Processor: Used to implement the SAR image road extraction method based on DeepLabv3+ and adaptive connected component labeling as described in any one of claims 1-3 when executing the computer program.
6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that, when executed by a processor, enables the implementation of the SAR image road extraction method based on DeepLabv3+ and adaptive connected component labeling as described in any one of claims 1-3.