A road extraction network construction method, a road extraction method, a device, and a medium

Abstract

The application discloses a road extraction network construction method, a road extraction method, equipment and a medium, and relates to the technical field of image analysis. The method comprises the following steps: acquiring optical RGB images, optical panchromatic images and SAR images containing roads to form multi-source sample data; performing pixel-level labeling on road regions in the multi-source sample data to obtain first sample labels; labeling connected regions of each road to obtain second sample labels; inputting the multi-source sample data into a pre-constructed road extraction network to obtain road extraction results and road connectivity prediction results; calculating prediction losses of model output results and the first sample labels and the second sample labels, and multi-source feature alignment losses; and training the road extraction network according to the joint losses to minimize the weighted total loss. The method of the application realizes accurate fusion of multi-source image features and improves the completeness and clarity of road extraction.

Description

Technical Field

[0001] This invention relates to the field of image analysis technology, and in particular to a method for constructing a road extraction network, a road extraction method, equipment, and medium. Background Technology

[0002] Road extraction is a crucial task in remote sensing image analysis, playing a key role in urban planning, traffic monitoring, disaster assessment, and map updating. However, roads exhibit significant complexity and diversity in different geographical environments, including intricate urban road networks, winding rural roads, and narrow, hidden mountain trails. Traditional road extraction methods face numerous challenges, such as low accuracy, poor adaptability, and significant interference from complex backgrounds. Currently, mainstream road extraction schemes can be categorized into three main types.

[0003] (1) Road extraction schemes based on optical images: For example, Chinese patent CN120070911A proposes a detail enhancement and scale selection road extraction network and method, which uses multi-scale features and attention mechanism to enhance boundary feature extraction; Chinese patent CN119399637A proposes an improved D-LinkNet, which uses dilated convolution to enhance the receptive field during feature learning, improve feature representation ability, and solve the road extraction problem in complex scenes. However, when optimizing the breakpoints of road extraction under image occlusion, label guidance is adopted, and in actual inference, the connectivity needs to be evaluated by the model's confidence in occlusion and shadow.

[0004] (2) Road extraction scheme based on SAR imagery: Chinese patent CN117746051A proposes a SAR image road extraction method based on SegNet, which enhances road extraction by using the difference weight of positive and negative sample information entropy; Chinese patent CN115393712A proposes a SAR image road extraction method based on dynamic hybrid pooling strategy, in which strip pooling in dynamic hybrid pooling strategy can capture long-distance relationships and local context in road SAR images, and is suitable for road SAR images with long strip structure features. However, the imaging principle based on SAR imagery results in relatively low image resolution and speckle noise, which makes road details blurry and makes it difficult to accurately distinguish between roads and similar surrounding features, especially in areas with complex road structures and similar feature features.

[0005] (3) Road extraction scheme based on SAR and optical fusion: Chinese patent CN118038274A proposes a multi-branch, multi-task, multimodal remote sensing image road extraction method. It adopts SAR image and optical image fusion, and uses channel-strip spatial attention to fuse the shallow and deep features of SAR and optical images to mine the advantageous information after fusion. The road edge information is extracted from SAR image to further enhance road extraction. Although this scheme makes up for the shortcomings of a single data source to a certain extent, the existing fusion methods still have defects in feature alignment and fusion strategy, which leads to the fused features not being able to fully utilize the advantages of multi-source data.

[0006] In view of the above, this application is hereby submitted. Summary of the Invention

[0007] This invention provides a method for constructing a road extraction network, a road extraction method, equipment, and medium to solve the above-mentioned technical problems.

[0008] This invention is achieved through the following technical solution:

[0009] A first aspect of the present invention provides a method for constructing a road extraction network, comprising:

[0010] Acquire optical RGB imagery, optical panchromatic imagery, and SAR imagery containing roads to form multi-source sample data;

[0011] The road regions in the multi-source sample data are labeled at the pixel level to obtain the first sample label; the connected regions of each road are labeled to obtain the second sample label.

[0012] A road extraction network is constructed, comprising a multi-level feature extraction module, a multi-source feature fusion module, a multi-path attention enhancement module, a deconvolution module, and a dual-task classification module; wherein, the dual-task classification module is used to perform road pixel classification task and road connectivity prediction task;

[0013] The multi-source sample data is input into the road extraction network, and the multi-level features of the optical RGB image, optical panchromatic image and SAR image are extracted sequentially by the multi-level feature extraction module. The multi-source feature fusion module fuses the multi-level features, the multi-path attention enhancement module enhances the features, the deconvolution module upsamples the data, and the dual-task classification module obtains the road pixel classification result and the road connectivity prediction result.

[0014] Calculate the first loss between the road pixel classification result and the first sample label, calculate the second loss between the road connectivity prediction result and the second sample label, and calculate the feature alignment loss between each of the multi-level features;

[0015] The road extraction network is jointly trained based on the first loss, the second loss, and the feature alignment loss, so as to minimize the weighted total loss of the first loss, the second loss, and the feature alignment loss.

[0016] This invention addresses the shortcomings of single-source extraction schemes and the deficiencies in feature alignment and fusion strategies in existing multi-source fusion schemes, which prevent the fused features from fully leveraging the advantages of the two data sources. By effectively fusing optical RGB images, optical panchromatic images, and SAR images, this invention compensates for the feature representation deficiencies of various data sources, forming a complementary relationship and providing sufficient information for feature extraction.

[0017] Simultaneously, by utilizing multi-level feature extraction and fusion of multi-source data, the structural information of roads at different scales is fully captured. By embedding a multi-path attention enhancement module, the importance weights of different source data at different feature levels are learned, and multi-source features are adaptively fused to fully leverage the advantages of texture details in optical images and all-weather information in SAR images.

[0018] By using feature alignment loss, the features of optical RGB, optical panchromatic, and SAR images can be accurately aligned, driving the feature fusion module to learn accurate feature mapping relationships. This enables precise matching of multi-source image features at the spatial and semantic levels, providing high-quality fused features for subsequent road extraction.

[0019] By introducing connectivity prediction tasks and connectivity loss calculations, road connectivity is constrained, which effectively enhances road connectivity when it is occluded by shadows or other obstructions, avoids problems such as road breaks or loss, and improves the completeness of road extraction.

[0020] Furthermore, the multi-level feature extraction module includes three ConvNeXt encoders, which are used to extract multi-level features of the optical RGB image, optical panchromatic image and SAR image respectively;

[0021] Each ConvNeXt encoder includes N stacked deep convolutional blocks. The N-1 deep convolutional blocks, excluding the first one, are used to extract N-1 hierarchical features respectively, where N≥4.

[0022] Furthermore, the multi-source feature fusion module includes three CCFF modules and a multi-source feature splicing module;

[0023] Each CCFF module obtains a feature fusion representation of N-1 hierarchical features through deconvolution upsampling and convolution downsampling operations, resulting in fused multi-level features.

[0024] The multi-source feature stitching module is used to stitch together the multi-level features of the fused optical RGB image, optical panchromatic image and SAR image to obtain multi-source fused features.

[0025] Furthermore, the multi-path attention enhancement module includes a spatial attention module and a channel attention module;

[0026] In the spatial attention module, the spatial attention weights of the multi-source fusion features are calculated, and the spatial attention weights are applied to the multi-source fusion features to obtain spatial attention-enhanced features;

[0027] In the channel attention module, the channel attention weights of the spatial attention enhancement feature are calculated, and the channel attention weights are applied to the spatial attention enhancement feature to obtain the multi-path attention enhancement feature.

[0028] Furthermore, the first loss is calculated using a binary cross-entropy loss function weighted by the number of pixels, expressed as:

[0029]

[0030] in, Indicates the first loss. The total number of pixels in the image. For pixels The first sample label, This indicates that pixel i represents the road. This indicates that pixel i is not a road;

[0031] Pixels predicted by the model Let be the probability of the road. For predicted pixels The probability of being a non-road;

[0032] , These are the weights for road pixels and non-road pixels, respectively. , This represents the number of pixels in the image whose first sample label is "road".

[0033] Furthermore, the second loss is calculated using the multi-class cross-entropy loss function, expressed as:

[0034]

[0035] in, Indicates the second loss. This represents the total number of connected regions in the image. Connected region Category labels for actual connected regions To predict connected regions The probability vector of each category label.

[0036] Furthermore, the feature alignment loss is calculated based on the MSE loss function, and is expressed as:

[0037]

[0038] in, Indicates feature alignment loss, , , These represent the multi-level features of optical RGB imagery, optical panchromatic imagery, and SAR imagery, respectively. This indicates the mean square error.

[0039] A second aspect of the present invention provides a road extraction method, comprising:

[0040] Acquire optical RGB imagery, optical panchromatic imagery, and SAR imagery of the area to be processed;

[0041] The optical RGB image, optical panchromatic image, and SAR image are preprocessed; the preprocessing includes unifying spatial resolution and georegistration.

[0042] The preprocessed optical RGB image, optical panchromatic image, and SAR image are input into the road extraction network to obtain road pixel classification results and road connectivity prediction results; the road extraction network is obtained by the road extraction network construction method described in any one of the first aspects of the present invention.

[0043] A third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the road extraction network construction method according to any one of the first aspects of the present invention or the road extraction method according to the second aspect.

[0044] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the road extraction network construction method according to any one of the first aspects of the present invention or the road extraction method according to the second aspect.

[0045] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0046] We innovatively propose a complete multi-source data fusion processing workflow for panchromatic optical images, RGB and SAR images, giving full play to the advantages of different data sources and providing a high-quality data foundation for road extraction;

[0047] By introducing pixel-level classification loss, road connectivity loss, and multi-source fusion feature alignment loss to jointly train the extraction network, the problem of feature mismatch between optical imagery and SAR imagery is effectively solved, achieving accurate fusion of multi-source image features and improving the completeness and clarity of road extraction. Attached Figure Description

[0048] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0049] Figure 1 This is a flowchart illustrating the construction method of the road extraction network;

[0050] Figure 2 This is the overall architecture diagram of the road extraction network;

[0051] Figure 3 This is a schematic diagram of the network structure of the CCFF module;

[0052] Figure 4 This is a schematic diagram of the network structure of the multi-path attention enhancement module. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0054] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification, claims, and accompanying drawings of this invention are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to other steps or units inherent in the device.

[0055] The terminology used in the various embodiments of the invention is for the purpose of describing particular embodiments only and is not intended to limit the various embodiments of the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of the invention pertain. The terms (such as those defined in a generally used dictionary) are to be interpreted as having the same meaning as in the context of the relevant technical field and are not to be interpreted as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of the invention.

[0056] Embodiments of the present invention provide a method for constructing a road extraction network, which aims to improve the accuracy, robustness and adaptability of road extraction through innovative data fusion strategies, feature alignment methods and loss calculation methods, especially to achieve more accurate and efficient road extraction in complex geographical environments and harsh imaging conditions.

[0057] See Figure 1 As shown, the present invention constructs a road extraction network based on multi-source data fusion using the following method.

[0058] Step 1: Construct a sample set: acquire optical RGB images, optical panchromatic images, and SAR images containing roads to form multi-source sample data.

[0059] SAR imagery, or Synthetic Aperture Radar imagery, generates images by actively emitting microwaves and receiving echoes from ground objects. Its unique imaging method enables it to perform all-weather, all-day imaging, penetrating clouds, fog, and vegetation to record the amplitude and phase information of the Earth's surface, making it particularly suitable for remote sensing monitoring under adverse weather conditions.

[0060] Optical RGB imagery: This type of image is created by a sensor capturing information from the three visible light bands (red, green, and blue) respectively, and then fusing them to form a true-color image. This image can intuitively reflect the natural colors and textures of ground features and has high visual recognition, but it is greatly affected by factors such as lighting and weather.

[0061] Optical panchromatic imagery: Single-band black-and-white images acquired by sensors in the visible light band. These images offer high spatial resolution and clearly reveal the details and outlines of ground features, but lack color information. Fusion with multispectral data can enhance image clarity and information content, playing a crucial role in applications such as road detection.

[0062] This invention acquires multi-source image data containing images of the same road by separately collecting optical RGB images, optical panchromatic images, and SAR images of the same geographical area. Building upon the all-weather, all-day imaging advantages of SAR images, optical images are introduced to supplement road details and texture information that are difficult to distinguish in SAR images, especially in areas where the road and surrounding features have similar gray levels, effectively improving the accuracy and clarity of road extraction.

[0063] Before utilizing multi-source data, the original acquired images need to be preprocessed to provide a spatially consistent data foundation for subsequent fusion and analysis. Georegistration is a fundamental step in multi-source data fusion, aiming to eliminate spatial positional discrepancies and geometric distortions between optical RGB, optical panchromatic, and SAR images. For optical images, ground control points (GCPs) can be used, employing geometric correction algorithms such as affine and polynomial transformations to uniformly map the images to a geographic coordinate system, ensuring the accurate geographical location of roads within the images. For SAR images, due to geometric distortions such as perspective contraction and overlay caused by their imaging mechanism, geometric correction methods based on orbital parameters and radar equations can be used, combined with terrain data for terrain undulation correction, and ground control points for error compensation, ensuring precise spatial alignment between SAR and optical images. After registration, the multi-source images are resampled to unify spatial resolution, providing a consistent data foundation for subsequent multi-source data fusion.

[0064] To increase data diversity and improve the model's generalization ability, preprocessing also includes various data augmentation operations on multi-source data. For optical images, geometric transformations such as rotation, translation, scaling, and flipping are used to simulate image changes under different viewpoints and shooting positions; image enhancement algorithms such as adjusting brightness, contrast, and saturation are used to simulate image characteristics under different lighting conditions and weather conditions; simultaneously, random cropping and noise addition are used to further enrich the data diversity. For SAR images, in addition to geometric transformations, speckle noise simulation and texture enhancement are used to simulate SAR image characteristics under different imaging conditions. During data augmentation, the spatial correspondence between multi-source data is maintained to ensure that the augmented data can still be used for fusion analysis.

[0065] Step 2, data annotation: perform pixel-level annotation on the road areas in the multi-source sample data to obtain the first sample label; annotate the connected regions of each road to obtain the second sample label.

[0066] The annotation data primarily relies on optical RGB imagery, as its intuitive color information facilitates manual identification of roads and backgrounds. Using professional remote sensing image annotation software, experienced annotators are organized to perform pixel-level annotations on roads in the optical RGB imagery according to unified annotation standards, such as labeling road pixels as 1 and non-road pixels as 0. During annotation, not only is the overall outline of the road marked, but different types of roads (such as urban arterial roads and rural paths) are further distinguished, and corresponding road type labels are assigned to different road types. Simultaneously, the boundaries between roads and surrounding features (such as building shadows and vegetation cover areas) are precisely delineated.

[0067] The sample image is processed using a connected component labeling algorithm to extract connected components. Different road connected components are regarded as different categories, and a unique category label is assigned to each component, such as using one-hot encoding.

[0068] After annotation is completed, multiple annotators review the same batch of data through cross-validation and quality control mechanisms. Disputed annotated areas are discussed and corrected to ensure the accuracy and consistency of the annotated data. To improve annotation efficiency and quality, semi-supervised learning-assisted annotation technology is introduced. This involves training a model using a small amount of already labeled data to predict large amounts of unlabeled data, assisting annotators in quickly completing the annotation work.

[0069] To ensure the scientific rigor of model training, validation, and testing, the labeled multi-source dataset was divided into training, validation, and test sets in an 8:1:1 ratio. The training set was used for learning and optimizing model parameters, enabling the model to learn road features and patterns from a large dataset. The validation set was used during model training to evaluate performance, adjust hyperparameters, and prevent overfitting. The test set was used after training to perform a final performance evaluation, verifying the model's generalization ability on unknown data. Stratified sampling was employed to ensure that the proportion of different road types and scenarios in each dataset remained consistent with the original dataset, guaranteeing a balanced data distribution and making the model training and evaluation results more representative.

[0070] Step 3, Model Construction: Construct a path extraction network that includes a multi-level feature extraction module, a multi-source feature fusion module, a multi-path attention enhancement module, and a dual-task classification module.

[0071] See Figure 2The diagram shows the overall architecture of the road extraction network. Based on the classic UNet network architecture, improvements have been made to both the encoder and decoder parts to achieve effective fusion of multi-source data and multi-task classification. At the network's input layer, multiple baseline networks are used as encoders to extract features from different data sources (preprocessed optical RGB, optical panchromatic, and SAR images as multi-source inputs). Each encoder extracts multi-level features from optical RGB, optical panchromatic, and SAR images respectively. Each encoder can consist of multiple deep convolutional blocks (ConvBlocks), forming a multi-level feature extraction network. Downsampling employs simple convolution operations, progressively extracting different levels of feature information for each data source. As the convolutional layers deepen, the spatial resolution of the feature map gradually decreases, while the semantic information gradually increases. The multi-level feature extraction module consists of three such multi-level feature extraction networks, used to extract multi-level features from optical RGB, optical panchromatic, and SAR images respectively. Simultaneously, a multi-source feature fusion module in the middle layer of the network fuses the multi-level features from different data sources to obtain multi-source fused features.

[0072] The decoder consists of a multi-channel attention enhancement module and a deconvolution module. The multi-source feature maps extracted by the encoder are input into the multi-channel attention enhancement module, which learns the importance weights of different source data at different feature levels. This adaptively fuses multi-source features, enhancing the difference between road features and the background, fully leveraging the texture detail advantages of optical imagery and the all-weather information advantages of SAR imagery, and compensating for the feature differences learned from single-source data. The deconvolution module upsamples the enhanced features and inputs them into the decoder head, mapping the optimized multi-source fused features to the output layers of the two classification tasks.

[0073] To improve the accuracy and completeness of road extraction, a multi-task semantic segmentation strategy is adopted. In the output layer of UNet, two task branches are set up for road pixel classification and road connectivity prediction, respectively. The road pixel classification task uses a binary classification method to divide each pixel in the image into road and non-road categories, thereby extracting road regions. The road connectivity prediction task, on the other hand, classifies roads into connected regions of 10 degrees and uses the classification of different regions to predict the overall road connectivity.

[0074] Step 4: Train model parameters based on prediction loss.

[0075] (1) Input the multi-source sample data prepared in steps one and two into the road extraction network constructed in step three. The data will pass through the multi-level feature extraction module, the multi-source feature fusion module, the multi-path attention enhancement module, the deconvolution module and the dual-task classification module in sequence to obtain the road extraction results and the road connectivity prediction results.

[0076] (2) For each input sample data and its corresponding model prediction result, calculate the first loss between the road extraction result and the first sample label, calculate the second loss between the road type recognition result and the second sample label, and calculate the feature alignment loss between each multi-level feature. Weight the first loss, the second loss and the feature alignment loss to obtain the total loss. Use each loss to jointly train the road extraction network to minimize the weighted total loss.

[0077] The embodiments of this invention innovatively propose a complete processing flow for extracting roads by fusing panchromatic optical images, optical RGB images, and SAR images. This includes multi-source data preprocessing, feature alignment, and data fusion, which fully leverages the advantages of different data sources to provide a high-quality data foundation for road extraction.

[0078] Deep learning-based feature alignment effectively solves the problem of feature mismatch between optical and SAR images, enabling accurate fusion of multi-source image features and improving the accuracy of road extraction.

[0079] In a further improved implementation, the multi-level feature extraction module uses three ConvNeXt encoders to extract multi-level features from optical RGB images, optical panchromatic images, and SAR images respectively. The shallow layer extracts detailed features such as road edges and textures (high resolution, low semantics), while the deep layer extracts semantic features such as the overall shape of the road and scene associations (low resolution, high semantics).

[0080] Each ConvNeXt encoder includes N stacked deep convolutional blocks. The N-1 deep convolutional blocks, excluding the first one, are used to extract N-1 hierarchical features respectively, where N≥4.

[0081] Furthermore, the multi-source feature fusion module includes three CCFF modules and a multi-source feature stitching module. Each CCFF module obtains a feature fusion representation of N-1 levels of features through deconvolution upsampling and convolution downsampling operations, resulting in fused multi-level features. The multi-source feature stitching module uses a Concat operation to stitch together the fused multi-level features of optical RGB imagery, optical panchromatic imagery, and SAR imagery, obtaining multi-source fused features.

[0082] See Figure 3 As shown, for the three levels of features, the CCFF module improves the resolution through deconvolution upsampling (U), preserves the semantics through convolution downsampling (D), and then obtains the feature fusion representation at different scales through Fusion fusion and Cat stacking, finally generating a fused multi-level feature map that covers the structural information of the road from details to the whole.

[0083] In a preferred embodiment, the multi-path attention enhancement module includes a spatial attention module and a channel attention module. In the spatial attention module, spatial attention weights are calculated for the multi-source fusion features and applied to the multi-source fusion features to obtain spatial attention-enhanced features. In the channel attention module, channel attention weights are calculated for the spatial attention-enhanced features and applied to the spatial attention-enhanced features to obtain multi-path attention-enhanced features.

[0084] See Figure 4 As shown, in the spatial attention module, the multi-source fusion features are first reshaped to adjust their dimensions, and then linearly projected to generate the query (Q) and key (Q) respectively. Q and its value (V). Then, Q is compared with... Matrix multiplication is performed, and the result is input into the Softmax function to generate spatial attention weights, which reflect the spatial correlation importance between different channels. Next, these spatial attention weights are multiplied with V to weight V, and then the dimensions are adjusted through projection and reshape operations. Finally, the features are fused with the original multi-source fusion features through residual connections to obtain spatial attention enhancement features, which strengthen the road's response in terms of spatial location and shape.

[0085] In the channel attention module, the spatial attention enhancement features also first undergo Reshape and Projection operations to generate query (Q) and key (Q) parameters. Q is first summed to the sum of the channel attention values ​​(V). Then, Q is multiplied by K², and the result is processed by Softmax to obtain the channel attention weights, which reflect the importance of different spatial locations to the channel features. These channel attention weights are then multiplied by V to weight V, and then the dimensions are adjusted by Projection and Reshape. Finally, the residual connections are added element-wise to the input spatial attention enhancement features to obtain the multi-path attention enhancement features. These features further highlight the channel information related to road features while suppressing the interference of background noise.

[0086] Furthermore, multiple stacked multi-channel attention enhancement modules (Dual-Ateention×N) and multiple deconvolution modules (UpSample×N) are used to perform multiple feature enhancements and upsampling, enabling the network to more accurately identify and extract roads, thus solving the problems of blurred road edges and discontinuous roads in traditional methods.

[0087] Furthermore, in the road pixel classification task, to address the severe imbalance in the number of road and non-road pixels, a binary cross-entropy loss function based on pixel count weights is used to calculate the first loss. Traditional binary cross-entropy loss functions treat all pixels equally, which can easily lead to the model biasing towards predicting the more prevalent non-road category during training, resulting in missed road pixels. This embodiment balances the impact of category differences on loss calculation by assigning weights to pixels of different categories.

[0088] The specific calculation formula is as follows:

[0089]

[0090] in, Indicates the first loss. The total number of pixels in the image. For pixels The first sample label, This indicates that pixel i represents the road. This indicates that pixel i is not a road;

[0091] Pixels predicted by the model Let be the probability of the road. For predicted pixels The probability of being a non-road;

[0092] , These are the weights for road pixels and non-road pixels, respectively. , This represents the number of pixels in the image whose first sample label is "road".

[0093] For example, if the road pixel ratio is 10% and the non-road pixel ratio is 90%, it can be set to... , By increasing the weight of road pixels in the loss calculation, the model is forced to focus on the prediction accuracy of road areas, thereby improving the recall and precision of road extraction.

[0094] This implementation addresses the problem of imbalanced sample classes by employing a weighted background suppression mechanism. By assigning different weights to the background class and the road class, the proportion of the road class in the loss function calculation is increased, effectively suppressing misclassification of the background class and improving the accuracy of road extraction.

[0095] Furthermore, to ensure good connectivity of the extracted roads and avoid issues such as breaks and fragmentation, a connectivity loss function based on multi-class cross-entropy is adopted. This method treats road connected regions as different categories, and guides the model to learn a continuous and complete road structure by measuring the category difference between the predicted connected regions and the actual connected regions.

[0096] First, the predicted road binary image and the ground truth road label image need to be processed using a connected component labeling algorithm to extract the predicted and ground truth connected components, and a unique class label (i.e., the second sample label) is assigned to each component. Then, the loss is calculated using multi-class cross-entropy, as shown in the following formula:

[0097]

[0098] in, Indicates the second loss. This represents the total number of connected regions in the image. Connected region Category labels for actual connected regions To predict connected regions The probability vector of each category label.

[0099] This loss function penalizes situations where the predicted connected regions are inconsistent with the actual connected regions, prompting the model to generate coherent and complete road connected regions. This effectively enhances the connectivity of roads when they are occluded by shadows or other obstructions, avoids problems such as road breaks or loss, and improves the completeness of road extraction.

[0100] Furthermore, during the multi-source data fusion process, to ensure accurate alignment of features from optical RGB, optical panchromatic, and SAR images, the mean squared error (MSE) loss function is used to measure feature differences. Feature alignment is crucial for leveraging the advantages of multi-source data; misalignment can lead to information confusion and reduce the accuracy of road extraction.

[0101] Let the features of the optical RGB image after feature extraction be... The characteristics of optical panchromatic images are SAR image features are All three dimensions are the same: C×H×W (C is the number of channels, H is the height, and W is the width). The formula for calculating the feature alignment loss function based on MSE is:

[0102]

[0103] in, Indicates feature alignment loss, , , These represent the multi-level features of optical RGB imagery, optical panchromatic imagery, and SAR imagery, respectively. This indicates the mean square error.

[0104] This function quantifies the difference between optical and SAR image features by calculating the squared error of the corresponding location feature elements and then averaging the results. During model training, minimizing... The driveable feature alignment module learns accurate feature mapping relationships, enabling precise matching of multi-source image features at the spatial and semantic levels, providing high-quality fused features for subsequent road extraction.

[0105] Finally, the total loss function is obtained by weighted summation of the three types of losses mentioned above, and is expressed as:

[0106]

[0107] in, , and The optimal values ​​for the weighting coefficients were determined through extensive experiments and cross-validation to balance the training objectives of pixel classification, connectivity optimization, and feature alignment, thereby achieving high-precision and high-completeness road extraction results.

[0108] Embodiments of the present invention also provide a road extraction method, comprising the following steps.

[0109] S2-1: Acquire the optical RGB image, optical panchromatic image, and SAR image of the area to be processed.

[0110] S2-2 preprocesses optical RGB imagery, optical panchromatic imagery, and SAR imagery; the preprocessing includes unifying spatial resolution and georegistration.

[0111] S2-3. The preprocessed optical RGB image, optical panchromatic image and SAR image are input into the road extraction network to obtain road pixel classification results and road connectivity prediction results.

[0112] The road extraction network is obtained by the road extraction network construction method of any of the above embodiments of the present invention.

[0113] Embodiments of the present invention also provide a road extraction network construction device, including a multi-source sample set construction module, a road extraction network construction module, and a model training module.

[0114] The multi-source sample set construction module includes sample data preprocessing and sample data labeling, which is used to preprocess and label the acquired multi-source sample data (optical RGB image, optical panchromatic image and SAR image) with first sample label and second sample label.

[0115] The road extraction network building module is used to construct a road extraction network based on the UNet architecture, including a multi-level feature extraction module, a multi-source feature fusion module, a multi-path attention enhancement module, a deconvolution module, and a dual-task classification module.

[0116] The model training module is used to input multi-source sample data into the constructed road extraction network, calculate the first loss, the second loss and the feature alignment loss based on the model prediction results and sample labels, and adjust the model parameters to minimize the weighted total loss.

[0117] Embodiments of the present invention also provide an electronic device, which includes a processor and a memory, wherein the number of processors may be one or more. The memory, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules. The processor executes various functional applications and data processing of the electronic device by running the software programs, instructions, and modules stored in the memory, thereby implementing the road extraction network construction method or road extraction method of any of the above embodiments of the present invention.

[0118] The memory primarily comprises a program storage area and a data storage area. The program storage area stores the operating system and at least one application program required for a given function; the data storage area stores data created based on terminal usage. Furthermore, the memory may include high-speed random access memory (RAM) and non-volatile memory, such as at least one disk storage device, flash memory, or other non-volatile solid-state storage device. In some instances, the memory may further include memory remotely located relative to the processor, which can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks (LANs), mobile communication networks, and combinations thereof.

[0119] Embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the road extraction network construction method or road extraction method of any embodiment of the present invention.

[0120] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0121] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0122] Embodiments of the present invention also provide a computer program product that, when run on a computer, causes the computer to execute the road extraction network construction method or road extraction method of any of the above embodiments of the present invention.

[0123] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A road extraction network construction method characterized by comprising: The method comprises the following steps: obtaining optical RGB images, optical panchromatic images and SAR images containing roads to form multi-source sample data; performing pixel-level labeling on the road regions in the multi-source sample data to obtain first sample labels; and performing labeling on the connected regions of each road to obtain second sample labels; constructing a road extraction network comprising a multi-level feature extraction module, a multi-source feature fusion module, a multi-path attention enhancement module, a deconvolution module and a double-task classification module; wherein the double-task classification module is used to perform a road pixel classification task and a road connectivity prediction task; inputting the multi-source sample data into the road extraction network, sequentially extracting multi-level features of the optical RGB images, optical panchromatic images and SAR images through the multi-level feature extraction module, fusing each of the multi-level features through the multi-source feature fusion module, performing feature enhancement through the multi-path attention enhancement module, upsampling through the deconvolution module, and then obtaining road pixel classification results and classification results of road connectivity prediction through the double-task classification module; calculating a first loss of the road pixel classification results and the first sample labels, calculating a second loss of the classification results of road connectivity prediction and the second sample labels, and calculating a feature alignment loss between each of the multi-level features; jointly training the road extraction network based on the first loss, the second loss and the feature alignment loss, so that the weighted total loss of the first loss, the second loss and the feature alignment loss is minimized.

2. The road extraction network construction method according to claim 1, characterized by, The multi-level feature extraction module comprises three ConvNeXt encoders for respectively extracting multi-level features of the optical RGB images, optical panchromatic images and SAR images. Each of the ConvNeXt encoders comprises N stacked depth convolution blocks, and N-1 depth convolution blocks other than the first depth convolution block are used to respectively extract N-1 level features, and N≥4.

3. The road extraction network construction method according to claim 2, characterized by, The multi-source feature fusion module comprises three CCFF modules and a multi-source feature splicing module. Each of the CCFF modules obtains a feature fusion representation of N-1 level features through deconvolution upsampling and convolution downsampling operations to obtain fused multi-level features. The multi-source feature splicing module is used to splice the fused multi-level features of the optical RGB images, optical panchromatic images and SAR images to obtain multi-source fusion features.

4. The road extraction network construction method according to claim 3, characterized by, The multi-path attention enhancement module comprises a spatial attention module and a channel attention module. In the spatial attention module, spatial attention weights of the multi-source fusion features are calculated, and the spatial attention weights are applied to the multi-source fusion features to obtain spatial attention enhanced features. In the channel attention module, channel attention weights of the spatial attention enhanced features are calculated, and the channel attention weights are applied to the spatial attention enhanced features to obtain multi-path attention enhanced features.

5. The road extraction network construction method according to claim 1, characterized by, The first loss is calculated by using a pixel number weighted binary cross-entropy loss function, and is represented as: wherein, denotes a first loss, is a total number of pixels of the image, is a pixel a first sample label of the pixel, denotes that pixel i is road, denotes that pixel i is non-road; predicted pixel for the model probability that the road, predicted pixel probability that the non-road; , are weights for road and non-road pixels, respectively, , denotes the number of pixels in the image for which the first sample label is road.

6. The road extraction network construction method according to claim 1, characterized by, The second loss is calculated by using a multi-class cross-entropy loss function, and is represented as: wherein, denotes a second loss, is the total number of connected regions in the image, is the class label of the true connected region, is the probability vector that the predicted connected region belongs to each class label.​ 7. The road extraction network construction method according to claim 1, characterized by, The feature alignment loss is calculated based on an MSE loss function, and is expressed as: wherein, denotes the feature alignment loss, , , denote multi-level features of optical RGB imagery, optical panchromatic imagery and SAR imagery, respectively, denotes the computation of the mean square error.

8. A road extraction method characterized by, The method comprises the following steps: An optical RGB image, an optical panchromatic image and a SAR image of a region to be processed are acquired; The optical RGB image, the optical panchromatic image and the SAR image are preprocessed; the preprocessing comprises unifying spatial resolutions and georeferencing; The preprocessed optical RGB image, the optical panchromatic image and the SAR image are input into a road extraction network to obtain a road pixel classification result and a road connectivity prediction classification result; the road extraction network is obtained by the road extraction network construction method in any one of claims 1-7.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the road extraction network construction method in any one of claims 1-7 or the road extraction method in claim 8.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the road extraction network construction method in any one of claims 1-7 or the road extraction method in claim 8.

Images