Remote sensing image road extraction method, system, medium and equipment

Abstract

The present invention relates to the field of remote sensing image processing technology, and more particularly to a method, system, medium, and device for extracting roads from remote sensing images. The method includes a label preprocessing step, which performs morphological expansion on original road area labels; a feature enhancement step, which inputs the remote sensing image into a lightweight super-resolution adaptation module and outputs an enhanced feature map and a super-resolution image; a multi-scale feature adaptation step, which freezes encoder weights and extracts multi-scale adaptation features using a lightweight convolution adaptation module. These features are then fused with pre-trained feature maps of different scales to obtain multi-scale fused features; a prediction map generation step, which inputs the multi-scale fused features into a decoder and outputs at least two prediction maps of different scales; a supervised training step, which calculates losses based on the prediction maps of different scales; and a road extraction step, which inputs the remote sensing image of the road to be extracted into a trained road extraction model and outputs a prediction map. Through the above configuration, both road extraction accuracy and model efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a remote sensing image road extraction method, system, medium and equipment. Background Art

[0002] Road extraction from remote sensing imagery is a critical foundational task in areas such as map updating, traffic analysis, urban planning, and disaster response. With the advancement of satellite remote sensing technology, imagery data from low-resolution satellites such as Gaofen-1 (with a resolution of 2 meters for panchromatic images and 8 meters for multispectral images) has become widely used. However, existing deep learning-based road extraction methods face challenges in practical applications, including insufficient accuracy in extracting small roads from low-resolution remote sensing imagery, the large amount of labeled data required for training, and the high computational resources required for model training.

[0003] Therefore, there is an urgent need for a road extraction method that can improve road extraction accuracy in low-resolution remote sensing images, reduce the demand for annotation data, and reduce computing resource consumption. Summary of the Invention

[0004] In order to solve at least one of the shortcomings of the road extraction method of remote sensing images in the above-mentioned prior art, the embodiments of the present invention provide a remote sensing image road extraction method, system, medium and equipment to effectively reduce the difficulty of algorithm training and improve the road extraction accuracy.

[0005] In a first aspect, an embodiment of the present invention provides a method for extracting roads from remote sensing images, comprising the following steps:

[0006] In the label preprocessing step, the original road area label is morphologically expanded to obtain the original road area label and the expanded area label;

[0007] In the feature enhancement step, the remote sensing image is input into the lightweight super-resolution adaptation module, the image is super-resolved through the convolutional network, and the enhanced feature map and super-resolution image are output;

[0008] A multi-scale feature adaptation step freezes the encoder weights of the pre-trained segmentation model, inputs the enhanced feature map into a lightweight convolution adaptation module, and extracts multi-scale adaptation features; the multi-scale adaptation features are fused with the pre-trained feature maps of different scales of the pre-trained segmentation model encoder to obtain multi-scale fused features;

[0009] Prediction map generation step: inputting the multi-scale fusion feature into a decoder, and outputting at least two prediction maps of different scales through upsampling and feature extraction;

[0010] In the supervised training step, the loss is calculated based on the prediction images at different scales and the corresponding original road area labels or expanded area labels. The total loss is the weighted sum of the losses at each scale. The model parameters are then optimized to obtain the trained road extraction model.

[0011] In the road extraction step, the remote sensing image of the road to be extracted is input into the trained road extraction model, and a predicted image with the same resolution as the original image is output as the final road extraction result image.

[0012] Furthermore, in the label preprocessing step, the morphological expansion is to perform multiple expansion iterations on the original road area label, and the generated expanded area label is a continuous area that is 1 to 2 pixels expanded outward from the original road area.

[0013] Furthermore, the remote sensing images in the feature enhancement step include panchromatic images, multispectral images and fused images; the input of the lightweight super-resolution adaptation module is panchromatic images, multispectral images and fused images, the output enhanced feature map has 64 channels, the super-resolution image is a 3-channel RGB image and the resolution is increased to twice that of the original image.

[0014] Furthermore, the pre-trained segmentation model in the multi-scale feature adaptation step is a SAM series model; the lightweight convolution adaptation module downsamples the enhanced feature map through a 3×3 convolution with a step size of 2, and extracts adaptation features of 1 / 4 scale, 1 / 8 scale, and 1 / 16 scale respectively. After the number of channels of each scale adaptation feature is adjusted by 1×1 convolution, it is added and fused with the pre-trained feature map of the corresponding scale.

[0015] Furthermore, the multi-scale fusion features include 1 / 4 scale fusion features, 1 / 8 scale fusion features and 1 / 16 scale fusion features, and the fusion features of each scale are fused step by step to the original image resolution through the upsampling module of the decoder.

[0016] Furthermore, in the supervised training step, the prediction images of different scales include a first-scale prediction image with the same resolution as the original image and a second-scale prediction image with a lower resolution than the original image; wherein the first-scale prediction image and the original road area label are used to calculate the pixel-level classification loss, and the second-scale prediction image and the expanded area label are used to calculate the same type or different type of pixel-level classification loss, and the total loss is the weighted sum of the losses of each scale calculated according to a preset weight coefficient.

[0017] Furthermore, the AdamW optimizer and cosine annealing learning rate scheduler are used to keep the encoder weights of the pre-trained segmentation model frozen during training.

[0018] In a second aspect, an embodiment of the present invention further provides a remote sensing image road extraction system, comprising:

[0019] The label preprocessing module performs morphological expansion on the original road area label to obtain the original road area label and the expanded area label;

[0020] The feature enhancement module inputs the remote sensing image into the lightweight super-resolution adaptation module, performs super-resolution reconstruction on the image through the convolutional network, and outputs the enhanced feature map and super-resolution image;

[0021] A multi-scale feature adaptation module freezes the encoder weights of the pre-trained segmentation model, inputs the enhanced feature map into the lightweight convolution adaptation module, extracts multi-scale adaptation features; and fuses the multi-scale adaptation features with the pre-trained feature maps of different scales of the pre-trained segmentation model encoder to obtain multi-scale fused features.

[0022] A prediction map generation module inputs the multi-scale fusion feature into a decoder, and outputs at least two prediction maps of different scales through upsampling and feature extraction;

[0023] The supervised training module calculates losses based on the predicted images at different scales and the corresponding original road area labels or expanded area labels. The total loss is the weighted sum of the losses at each scale. The model parameters are then optimized to obtain the trained road extraction model.

[0024] The road extraction module inputs the remote sensing image of the road to be extracted into the trained road extraction model and outputs a predicted image with the same resolution as the original image as the final road extraction result image.

[0025] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer is executed by a processor, the remote sensing image road extraction method as described in any embodiment of the first aspect above is implemented.

[0026] In a fourth aspect, an embodiment of the present invention provides an electronic device comprising at least one processor and a memory communicatively connected to the processor, wherein the memory stores instructions executable by at least one processor, and the instructions are executed by at least one processor so that the processor executes the remote sensing image road extraction method as described in any embodiment of the first aspect above.

[0027] Based on the above content, compared with the existing technology, the remote sensing image road extraction method provided by the embodiment of the present invention not only effectively alleviates the sparse gradient problem of roads (small targets) in low-resolution remote sensing images through the synergistic effect of steps such as label preprocessing, feature enhancement, multi-scale feature adaptation, and multi-scale supervised training, but also enhances the model's ability to capture road boundaries and small branches. It also effectively improves road extraction accuracy, reduces the demand for annotation data, and reduces computing resource consumption, achieving a dual improvement in remote sensing image road extraction accuracy and model efficiency, and is effectively applicable to complex scenarios.

[0028] Other features and advantages of the present invention will be set forth in the following description, and in part will be obvious from the description, or may be learned by practicing the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work. The positional relationships described in the drawings in the following description are based on the directions of the components drawn in the diagrams, unless otherwise specified.

[0030] Figure 1 A flowchart of a method for extracting roads from remote sensing images according to an embodiment of the present invention;

[0031] Figure 2 This is a comparison example of various types of image data;

[0032] Figure 3 A model architecture diagram of a method for extracting roads from remote sensing images provided by one embodiment of the present invention;

[0033] Figure 4 Schematic diagram of the processing flow for selecting SR Adaptor for the lightweight super-resolution adaptation module;

[0034] Figure 5 Enhanced feature maps for lightweight convolution adaptation modules Schematic diagram of the process of multi-scale downsampling;

[0035] Figure 6 This is an image comparison diagram containing the fused image, the real road, the road extraction result of this embodiment, and the conventional CoANet road extraction result;

[0036] Figure 7 A schematic diagram of the structure of a remote sensing image road extraction system provided by an embodiment of the present invention;

[0037] Figure 8 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments; the technical features designed in different implementation modes of the present invention described below can be combined with each other as long as they do not conflict with each other; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0039] In the description of the present invention, it should be noted that all terms used in the present invention (including technical terms and scientific terms) have the same meanings as those generally understood by ordinary technicians in the field to which the present invention belongs, and should not be understood as limiting the present invention; it should be further understood that the terms used in the present invention should be understood to have the same meanings as these terms in the context of this specification and the relevant field, and should not be understood in an idealized or overly formal sense, unless explicitly defined as such in the present invention.

[0040] Road extraction from remote sensing images is an important task for map updating and traffic analysis. Currently, existing road extraction methods based on deep learning have the following main problems:

[0041] First, the accuracy of extracting small roads in low-resolution remote sensing images is insufficient. Since the width of roads in low-resolution images is usually only 1 to 2 pixels, road features are easily suppressed or lost after multiple downsampling by deep networks (such as the ResNet series), resulting in sparse gradients and unstable training during model training. The final extracted road boundaries are blurred and have a high missed detection rate, making it difficult to meet the needs of high-precision applications. In particular, due to the large differences in resolution, imaging angle, weather conditions, etc. of remote sensing images, traditional trained models have poor generalization capabilities on different data sets and are difficult to adapt to new data. Therefore, when the sample resolution is low, the extracted road boundaries are inaccurate.

[0042] Second, the amount of labeled data required for training is excessive. Existing methods rely on large quantities of manually labeled remote sensing images for model training. Labeling remote sensing images requires professionals to mark road areas pixel by pixel, which is costly and easily affected by human factors (e.g., poor labeling consistency), severely limiting the large-scale application of these methods.

[0043] Third, model training requires excessive computing resources. Existing models have a large number of parameters and take a long time to train, requiring significant computing resources (such as high-performance GPUs / TPUs). This is especially true for high-resolution remote sensing imagery. Directly fine-tuning all parameters to adapt to remote sensing road extraction tasks consumes significant graphics memory, making deployment difficult in environments with limited computing resources (such as edge devices and small and medium-sized enterprises).

[0044] Based on the above problems, the embodiments of the present invention provide a remote sensing image road extraction method, system, medium and equipment, aiming to solve the problems of insufficient accuracy, large labeling data requirements and high training resource consumption in existing methods in low-resolution road extraction.

[0045] The following describes and describes in detail the method, system, medium and equipment for extracting roads from remote sensing images in conjunction with specific implementation methods and accompanying drawings.

[0046] Example 1

[0047] See also Figure 1 The method for extracting roads from remote sensing images provided by one embodiment of the present invention comprises at least the following steps:

[0048] In the label preprocessing step, the original road area label is morphologically expanded to obtain the original road area label and the expanded area label.

[0049] In practice, after obtaining the original road region labels, they can be processed using a morphological dilation algorithm. This can be achieved by setting a structuring element (such as a circle, rectangle, or cross). Assuming a 3×3 rectangular structuring element, for each pixel in the original road region label, align the center of the structuring element with that pixel. If a road label pixel exists within the area covered by the structuring element, the pixel is marked as a road region. This morphological dilation process effectively expands the range of the road label, generating the original road region label and the dilated region label. This step enhances the features of the road region, providing richer annotation information for subsequent model training, helping the model better learn the boundaries and shape characteristics of the road.

[0050] Preferably, in the label preprocessing step, the morphological dilation is performed by performing multiple dilation iterations on the original road area label, and the generated dilated area label is a continuous area that is 1 to 2 pixels outward from the original road area. For example, a 3×3 square structure element (SE) is selected, and the cv2.dilate function of the OpenCV library is used to perform two dilation iterations on the original label; in the dilated label, the original road area is expanded outward by 2 pixels (i.e., the dilated area), and two label files are generated: the original road area label (retaining only the originally labeled road area) and the dilated area label (including the original road area + the dilated area), for example Figure 2 shown.

[0051] This embodiment performs morphological dilation on the original road region labels to expand the positive sample area, increase the pixel ratio of small road targets, and alleviate the sparse gradient problem of road features (only 1-2 pixels wide) in low-resolution imagery after deep network downsampling. This improves the model's sensitivity to small roads. In other words, it effectively addresses the issues of blurred road boundaries and sparse gradients in low-resolution imagery (small targets).

[0052] In the feature enhancement step, the remote sensing image is input into the lightweight super-resolution adaptation module, the image is super-resolved through the convolutional network, and the enhanced feature map and super-resolution image are output.

[0053] To further enhance the road detail features of low-resolution images, this embodiment introduces a lightweight super-resolution adaptation module, which significantly reduces the overall computational complexity and improves the ability to restore feature details.

[0054] See also Figure 3 In this embodiment, the pre-processed remote sensing image is input into the lightweight super-resolution adaptation module, where Figure 2 As shown, the input remote sensing image may include three types of image data, namely, panchromatic image, multispectral image and fusion image. In this embodiment, the panchromatic image is preferably a GF-1 panchromatic image. , set to a resolution of 2 meters, size 512×512, and 1 channel, which is represented by ; Multispectral image is RGB multispectral image , set to 8 meters resolution, 128×128 size, 3 channels (blue, green, red), which is represented by ; The fused image is the Gram-Schmidt fused image , which fuses panchromatic and multispectral images through the Gram-Schmidt spectral fusion algorithm, can be set to a resolution of 2 meters, a size of 512×512, and 1 channel, which is expressed as .

[0055] The lightweight super-resolution adaptation module can select SR Adaptor, and its formula example is:

[0056] ;

[0057] ;

[0058] Where, Expressed as ; The convolution kernel size is The convolutional layer ( ,Convolutional Layer), the convolution layer performs convolution operation with the input data through the convolution kernel to extract features from the data; Batch Normalization is a process that normalizes data to a distribution with a mean of 0 and a standard deviation of 1. This suppresses the internal covariate shift problem during training, thereby accelerating model training and improving the model's generalization ability. Rectified Linear Unit is a commonly used nonlinear activation function that is used to enhance the nonlinear expression ability of the model and helps prevent overfitting. The overall representation is to first perform a convolution kernel of size Convolution operation, followed by batch normalization, and finally rectified linear units to process data, extract and transform features; It means that the multispectral image input is upsampled n times. In this embodiment, it is preferred ; (Concatenate) represents the concatenation operation of each feature map in the channel dimension; It is expressed as 、 、 The data are spliced ​​and merged together in the channel dimension for subsequent The input of the operation; the output of SR Adaptor is the feature map With super-resolution image .

[0059] In this embodiment, the number of channels of the enhanced feature map outputted is preferably 64, the super-resolved image is a 3-channel RGB image, and the resolution is increased to twice that of the original image.

[0060] Among them, see Figure 4 The processing flow of using SR Adaptor is as follows: First, 、 and upsampled by 4 times Splicing ( Operation), after two 3×3 convolution + BN + ReLU (CBR) processes, the feature map is generated (64 channels, 12×512). After CBR processing, the pixel is upsampled by 2 times through PixelShuffle. Add to get the super-resolution image (3 channels, 1024×1024).

[0061] Super-resolution reconstruction is achieved through the lightweight convolutional network used above. While increasing image resolution (by a factor of 2), it also enhances road detail features (such as edges and textures) and reduces computational complexity (compared to full-parameter SAM2 fine-tuning, video memory usage is reduced by approximately 40%).

[0062] The multi-scale feature adaptation step freezes the encoder weights of the pre-trained segmentation model, inputs the enhanced feature map into the lightweight convolution adaptation module, and extracts multi-scale adaptation features; the multi-scale adaptation features are fused with the pre-trained feature maps of different scales of the pre-trained segmentation model encoder to obtain multi-scale fused features.

[0063] In specific implementation, this embodiment uses the encoder of the pre-trained segmentation model to extract features of different scales, freezes its pre-trained weights, and only inserts lightweight convolutional adaptation modules at key positions to achieve fast and low-cost domain adaptation.

[0064] Preferably, please continue to see Figure 3 The pre-trained segmentation model in the multi-scale feature adaptation step is a SAM series model, and in this embodiment, a SAM2 encoder model (i.e., SAM2 Encoder) is preferably used. The lightweight convolution adaptation module downsamples the enhanced feature map through a 3×3 convolution with a stride of 2, and extracts adapted features at 1 / 4 scale, 1 / 8 scale, and 1 / 16 scale, respectively. After the number of channels of each scale adapted feature is adjusted through a 1×1 convolution, the features are added and fused with the pre-trained feature map of the corresponding scale.

[0065] Among them, the formula example for inserting a lightweight convolution adaptation module at the key position of the trained SAM2 model encoder for feature adaptation is:

[0066] ;

[0067] ;

[0068] ;

[0069] in Expressed as a step size of 2 Convolution, that is Specifically, 、 and go through After channel adaptation, the 1 / 4 scale feature map of the SAM 2 encoder is respectively , 1 / 8 scale feature map , 1 / 16 scale feature map Fusion: The following fusion formula is obtained:

[0070] .

[0071] Based on the above formula, see Figure 5 , through the lightweight convolution adapter module (Adaptor) to enhance the feature map Multi-scale downsampling includes the following three groups:

[0072] Adaptor 1: Extracts 1 / 4 scale features through a 3×3 convolution (DS operation) with a stride of 2 , after 1×1 convolution adjustment channel, and SAM2 encoder 1 / 4 scale feature map Additive fusion.

[0073] Adaptor 2: Yes Perform two DS operations to extract 1 / 8 scale features , and the 1 / 8 scale feature map of the SAM2 encoder Fusion.

[0074] Adaptor 3: Yes Perform three DS operations to extract 1 / 16 scale features , and the 1 / 16 scale feature map of the SAM2 encoder Fusion.

[0075] The fused multi-scale features include 1 / 4, 1 / 8, and 1 / 16 scales, corresponding to different levels of semantic information (e.g., 1 / 16 scale captures global structure, and 1 / 4 scale retains details).

[0076] Through the above steps, this embodiment utilizes the generalization capability of the SAM2 pre-trained encoder and combines it with a lightweight Adaptor module to achieve domain adaptation, avoiding the high computational cost of full parameter fine-tuning (training time is shortened by approximately 60%), while retaining the multi-scale features of roads in remote sensing images and improving extraction robustness in complex scenarios.

[0077] Prediction map generation step: input the multi-scale fusion features into the decoder, and output at least two prediction maps of different scales through upsampling and feature extraction.

[0078] In specific implementation, this embodiment preferably includes the multi-scale fusion features including 1 / 4 scale fusion features, 1 / 8 scale fusion features and 1 / 16 scale fusion features, and the fusion features of each scale are fused step by step to the original image resolution through the upsampling module of the decoder.

[0079] Please continue reading Figure 3 The decoder can use ASPP (atrous spatial pyramid pooling) and four strip convolution blocks to upsample the multi-scale fusion features and extract linear features. The segmentation head is a 3×3 convolution + sigmoid activation function, which outputs prediction maps at at least two scales.

[0080] The predicted image preferably includes at least a first-scale predicted image with the same resolution as the original image and a second-scale predicted image with a lower resolution than the original image. The first-scale predicted image (512×512) is used to directly predict the road area. The second-scale predicted image, upsampled using bicubic interpolation to match the scale of the dilated label, is used to supervise the extraction of the dilated region.

[0081] By combining the above multi-scale prediction with an upsampling mechanism, both the overall road structure (large scale) and boundary accuracy (original scale) are taken into account, reducing missed detections of small roads.

[0082] In the supervised training step, losses are calculated based on the prediction images of different scales and the corresponding original road area labels or expanded area labels. The total loss is the weighted sum of the losses at each scale. The model parameters are optimized through backpropagation to obtain the trained road extraction model.

[0083] In specific implementation, this embodiment uses prediction maps of different scales to set original road area supervision and expansion area supervision, which can effectively improve the fineness of road extraction and boundary positioning accuracy.

[0084] Specifically, the prediction images of different scales include a first-scale prediction image with the same resolution as the original image and a second-scale prediction image with a lower resolution than the original image.

[0085] Among them, according to the first scale prediction map (denoted as ) and the original road area label (denoted as ) Calculate pixel-level classification loss , its formula is expressed as:

[0086] ;

[0087] The second scale prediction map (denoted as ) and the dilation region label (denoted as ) Compute pixel-wise classification losses of the same or different types , which includes binary cross entropy (BCE) and Dice loss, the formula is:

[0088] ;

[0089] The total loss is the weighted sum of the losses of each scale calculated according to the preset weight coefficients, and its formula is:

[0090] ;

[0091] Where, Expressed as the weight coefficient of the Dice loss component in the loss function, The larger it is, the higher the proportion of Dice loss in the total loss, thus paying more attention to optimizing the overlap of the segmented areas; Represents the weight coefficient of the loss weight of different scales, used to balance and The specific weight coefficient should be reasonably limited according to actual needs. , .

[0092] Furthermore, this embodiment may use the AdamW optimizer and the cosine annealing learning rate scheduler to keep the encoder weights of the pre-trained segmentation model frozen during training, and only train the SR Adaptor, Adaptor module, and decoder parameters.

[0093] As an example, the learning rate of the AdamW optimizer is set to 0.0004, and it is combined with a cosine annealing scheduler to reduce the learning rate with linear warm-up, the batch size is set to 4, and the maximum number of training rounds is 100.

[0094] In the above steps, multi-scale deep supervision can improve the accuracy of road boundary positioning through the dual constraints of the expanded area and the original area; freezing the pre-trained encoder reduces overfitting and reduces the dependence on large-scale annotated data.

[0095] In the road extraction step, the remote sensing image of the road to be extracted is input into the trained road extraction model, and a predicted image with the same resolution as the original image is output as the final road extraction result image.

[0096] In specific implementation, the remote sensing images to be extracted include panchromatic images, multispectral images and fusion images. The remote sensing images to be extracted are input into the trained model for processing, and a predicted image with the same resolution as the original image can be directly output, thereby realizing end-to-end road extraction without human intervention. It can still run efficiently in an environment with limited computing resources (such as low GPU memory), and is particularly suitable for different application scenarios.

[0097] To effectively illustrate the effects of the embodiments of the present invention, Figure 6Two sets of image comparisons are given, including the fused image, the real road, the road extraction results of this embodiment, and the conventional CoANet road extraction results. Figure 6 It can be seen that the results obtained by using the road extraction method of this embodiment can effectively extract and accurately display more road details, reduce the missed detection rate of fine roads, and have better road extraction precision than the conventional CoANet road extraction results.

[0098] In summary, this embodiment effectively solves the problems of low road extraction accuracy, large annotation data requirements, and high training resource consumption in low-resolution remote sensing images through the collaborative design of label preprocessing, lightweight super-resolution adaptation, multi-scale feature fusion, and multi-scale supervised training. It is suitable for various complex scenarios.

[0099] Example 2

[0100] See also Figure 7 , an embodiment of the present invention further provides a remote sensing image road extraction system, comprising:

[0101] The label preprocessing module performs morphological expansion on the original road area label to obtain the original road area label and the expanded area label;

[0102] The feature enhancement module inputs the remote sensing image into the lightweight super-resolution adaptation module, performs super-resolution reconstruction on the image through the convolutional network, and outputs the enhanced feature map and super-resolution image;

[0103] A multi-scale feature adaptation module freezes the encoder weights of the pre-trained segmentation model, inputs the enhanced feature map into the lightweight convolution adaptation module, extracts multi-scale adaptation features; and fuses the multi-scale adaptation features with the pre-trained feature maps of different scales of the pre-trained segmentation model encoder to obtain multi-scale fused features.

[0104] A prediction map generation module inputs the multi-scale fusion feature into a decoder, and outputs at least two prediction maps of different scales through upsampling and feature extraction;

[0105] The supervised training module calculates losses based on the predicted images at different scales and the corresponding original road area labels or expanded area labels. The total loss is the weighted sum of the losses at each scale. The model parameters are optimized through backpropagation to obtain the trained road extraction model.

[0106] The road extraction module inputs the remote sensing image of the road to be extracted into the trained road extraction model and outputs a predicted image with the same resolution as the original image as the final road extraction result image.

[0107] It should be noted that the specific processing methods and functions of each module can be referred to the above-mentioned embodiment 1, and will not be elaborated here.

[0108] Example 3

[0109] An embodiment of the present invention further provides a computer-readable storage medium, which stores computer instructions. When executed by a computer processor, the computer implements the remote sensing image road extraction method as described in the first embodiment.

[0110] In a specific implementation, the computer-readable storage medium is a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the computer-readable storage medium may also include a combination of the above types of memory.

[0111] Example 4

[0112] See also Figure 8 An embodiment of the present invention also provides an electronic device, comprising at least one processor and a memory communicatively connected to the processor, wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the processor to perform the remote sensing image road extraction method as described in Example 1.

[0113] In a specific implementation, the number of processors may be one or more, and the processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips. The general-purpose processor may be a microprocessor or any conventional processor.

[0114] The memory and the processor can be communicatively connected via a bus or other means. The memory stores instructions that can be executed by at least one processor. The instructions are executed by at least one processor so that the processor executes the remote sensing image road extraction method introduced in the above method embodiment.

[0115] In summary, compared with the prior art, the remote sensing image road extraction method, system, medium, and device provided by the present invention have the following advantages:

[0116] 1. Through morphological expansion label expansion and a lightweight super-resolution module, we solve the problem of extracting small roads in low-resolution remote sensing images. Compared with methods such as CoANet, this method significantly reduces the missed detection rate of small roads and improves the accuracy of road extraction.

[0117] Second, through the label expansion strategy, the demand for labeled data is greatly reduced, the labeling cost is lowered, and combined with the transfer learning of SAM2 pre-trained weights, high extraction accuracy is still maintained in small sample scenarios (such as only 100 labeled images).

[0118] 3. Through lightweight module design and the method of freezing the SAM2 encoder weights, the training resource requirements (such as training time and processor space) are greatly reduced, making it not only suitable for edge device deployment, but also for resource-constrained environments.

[0119] In addition, those skilled in the art should understand that, although there are many problems in the prior art, each embodiment or technical solution of the present invention may be improved in only one or several aspects, without having to simultaneously solve all the technical problems listed in the prior art or background art. Those skilled in the art should understand that any content not mentioned in a claim should not be construed as limiting the claim.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for extracting roads from remote sensing images, characterized in that: The following steps are involved: In the label preprocessing step, the original road area label is morphologically expanded to obtain the original road area label and the expanded area label; In the feature enhancement step, the remote sensing image is input into the lightweight super-resolution adaptation module, the image is super-resolved through the convolutional network, and the enhanced feature map and super-resolution image are output; A multi-scale feature adaptation step freezes the encoder weights of the pre-trained segmentation model, inputs the enhanced feature map into a lightweight convolution adaptation module, and extracts multi-scale adaptation features; Fusing the multi-scale adaptation features with pre-trained feature maps of different scales of the pre-trained segmentation model encoder to obtain multi-scale fused features; Prediction map generation step: inputting the multi-scale fusion feature into a decoder, and outputting at least two prediction maps of different scales through upsampling and feature extraction; In the supervised training step, the loss is calculated based on the prediction images at different scales and the corresponding original road area labels or expanded area labels. The total loss is the weighted sum of the losses at each scale. The model parameters are then optimized to obtain the trained road extraction model. In the road extraction step, the remote sensing image of the road to be extracted is input into the trained road extraction model, and a predicted image with the same resolution as the original image is output as the final road extraction result image.

2. The method for extracting roads from remote sensing images according to claim 1, wherein: In the label preprocessing step, the morphological expansion is to perform multiple expansion iterations on the original road area label, and the generated expanded area label is a continuous area that is 1 to 2 pixels expanded outward from the original road area.

3. The method for extracting roads from remote sensing images according to claim 1, wherein: The remote sensing images in the feature enhancement step include panchromatic images, multispectral images and fused images; the input of the lightweight super-resolution adaptation module is panchromatic images, multispectral images and fused images, the output enhanced feature map has 64 channels, the super-resolution image is a 3-channel RGB image and the resolution is increased to twice that of the original image.

4. The method for extracting roads from remote sensing images according to claim 1, wherein: The pre-trained segmentation model in the multi-scale feature adaptation step is a SAM series model; the lightweight convolution adaptation module downsamples the enhanced feature map through a 3×3 convolution with a step size of 2, and extracts adaptation features of 1 / 4 scale, 1 / 8 scale, and 1 / 16 scale respectively. After the number of channels of each scale adaptation feature is adjusted by 1×1 convolution, it is added and fused with the pre-trained feature map of the corresponding scale.

5. The method for extracting roads from remote sensing images according to claim 4, wherein: The multi-scale fusion features include 1 / 4 scale fusion features, 1 / 8 scale fusion features and 1 / 16 scale fusion features, and the fusion features of each scale are fused step by step to the original image resolution through the upsampling module of the decoder.

6. The method for extracting roads from remote sensing images according to any one of claims 1 to 5, characterized in that: In the supervised training step, the prediction images of different scales include a first-scale prediction image with the same resolution as the original image and a second-scale prediction image with a lower resolution than the original image; wherein the first-scale prediction image and the original road area label are used to calculate pixel-level classification loss, and the second-scale prediction image and the expanded area label are used to calculate the same or different types of pixel-level classification loss, and the total loss is a weighted sum of the losses of each scale calculated according to a preset weight coefficient.

7. The method for extracting roads from remote sensing images according to claim 1, wherein: The AdamW optimizer and cosine annealing learning rate scheduler are used to keep the encoder weights of the pre-trained segmentation model frozen during training.

8. A remote sensing image road extraction system, characterized in that: include: The label preprocessing module performs morphological expansion on the original road area label to obtain the original road area label and the expanded area label; The feature enhancement module inputs the remote sensing image into the lightweight super-resolution adaptation module, performs super-resolution reconstruction on the image through the convolutional network, and outputs the enhanced feature map and super-resolution image; A multi-scale feature adaptation module freezes the encoder weights of the pre-trained segmentation model, inputs the enhanced feature map into a lightweight convolution adaptation module, and extracts multi-scale adaptation features; Fusing the multi-scale adaptation features with pre-trained feature maps of different scales of the pre-trained segmentation model encoder to obtain multi-scale fused features; A prediction map generation module inputs the multi-scale fusion feature into a decoder, and outputs at least two prediction maps of different scales through upsampling and feature extraction; The supervised training module calculates losses based on the predicted images at different scales and the corresponding original road area labels or expanded area labels. The total loss is the weighted sum of the losses at each scale. The model parameters are then optimized to obtain the trained road extraction model. The road extraction module inputs the remote sensing image of the road to be extracted into the trained road extraction model and outputs a predicted image with the same resolution as the original image as the final road extraction result image.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer is executed by a processor, the remote sensing image road extraction method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: The method comprises at least one processor and a memory in communication with the processor, wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the processor executes the remote sensing image road extraction method according to any one of claims 1 to 7.

Images