A remote sensing image road extraction method based on a double-encoder network model

Abstract

The application discloses a kind of based on double encoder network model's remote sensing image road extraction method, belong to road extraction technical field, including: the remote sensing image in data set is carried out image enhancement;Enhanced image is extracted by network model, wherein one of the encoder backbone network in network model is used to strengthen the attention to road, and the backbone network of another encoder is used to expand the receptive field;Feature map extracted by Swin-T encoder is subjected to global domain target perception by hole convolution block, then is subjected to jump splicing with feature map extracted by ResNet encoder and is subjected to decoder, finally by road analysis module output one with the same size of input image gray picture, its foreground is the road extracted.The application improves the problem that the precision of extracted road is not high due to factors such as illumination, shielding and other factors of remote sensing image in prior art, and can be applied to map drawing, traffic rescue, vehicle navigation and other practical problems.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of road extraction, and particularly relates to a remote sensing image road extraction method based on a double-encoder network model. BACKGROUND

[0002] Road extraction is an important research topic in image processing specific application. Remote sensing image road extraction plays a very important role in map drawing, traffic rescue, vehicle navigation and other directions. In the aspect of map drawing, it can greatly reduce the use of human resources, and it is also very important to quickly find a suitable road in the aspect of traffic rescue; and it can also improve the accuracy of navigation.

[0003] The traditional remote sensing image road extraction adopts a network model stacked with convolution and pooling, and little attention is paid to the correlation between roads; although convolution can strengthen the global perception field layer by layer, with the depth getting deeper, the number of parameters is more and the amount of calculation is larger. In addition, when extracting features by convolution, the opportunities for extracting features of roads and buildings and other irrelevant objects are equal, but often the proportion of roads in remote sensing images is less than that of buildings or land. Moreover, the differences in spectral features, texture features and geometric features of remote sensing images caused by different geographical positions also greatly affect the extraction task. Based on the above analysis, improving the attention of road features and the effective perception of global field makes the remote sensing image road extraction accuracy go to a higher level. SUMMARY

[0004] The application proposes a remote sensing image road extraction method based on a double-encoder network model to solve the problem of low accuracy of remote sensing image road extraction in the prior art.

[0005] In order to achieve the above purpose, the application adopts the following technical scheme:

[0006] A remote sensing image road extraction method based on a double-encoder network model, comprising the following steps:

[0007] Step 1: image enhancement is performed on remote sensing images in a data set, and random saturation value, hue transformation and translation, rotation processing are performed on the remote sensing images;

[0008] Step 2: a road extraction network model is constructed;

[0009] Step 2.1: a Swin-T encoder is built;

[0010] Step 2.2: a ResNet encoder is built;

[0011] Step 2.3: a dilated convolution block is built;

[0012] Step 2.4: a decoder is built;​

[0013] Step 2.5: build a road extraction block to obtain a grayscale image;

[0014] Step 3: model training, input remote sensing image data into the training model for training.

[0015] Preferably, the specific steps of step 2 include:

[0016] Step 2.1: Swin-T encoder with Swin Transformer as the basic framework, the Swin-T encoder is: first a sequence mapping block, then sequentially connected four segments; the four segments are respectively 2 layers of Swin Transformer as a segment, 2 layers of Swin Transformer as a segment, 6 layers of Swin Transformer as a segment, and 2 layers of Swin Transformer as a segment, and there is a sequence fusion block between every two segments;

[0017] Step 2.1.1: the sequence mapping block construction step: perform a convolution to make the output channel 96, the convolution kernel size is 4, and the step is 4, so that the pixel points are mapped to a sequence; connect a Layer Norm layer;

[0018] Step 2.1.2: the sequence fusion block construction step: first connect a Layer Norm layer; use a dilated convolution to double the number of feature maps and reduce the size

[0019] Step 2.2: ResNet encoder with ResNet34 as the basic framework, the ResNet encoder is: first an adaptive convolution, a Batch Norm, and a ReLU activation function to make the picture specification 512x512x64, then a down-sampling, and finally four segments, the four segments are respectively ResNet34 layer1, layer2, layer3 and layer4;

[0020] Step 2.3: build and connect a dilated convolution block after the Swin-T encoder; the dilated convolution block construction step is as follows:

[0021] Step 2.3.1: connect a Layer Norm layer;

[0022] Step 2.3.2: connect n dilated convolution layers:

[0023] Conv2d(kernel_size=3,dilation=1,padding=1)

[0024] Conv2d(kernel_size=3, dilation=2, padding=2)

[0025] …

[0026] Conv2d(kernel_size=3, dilation=2 n-1 , padding=2 n-1 )

[0027] Conv2d() is a convolution function, kernel_size is the size of the convolution kernel, dilation is the dilation factor, the dilation factors of the n layers are 1, 2 1 ,..., 2 n-1 , padding is the convolution padding, the convolution paddings of the n layers are 1, 2 1 ,..., 2 n-1 , where size is the size of the feature map;

[0028] Step 2.3.3: Each layer of the empty convolution layer is followed by a ReLU activation function;

[0029] Step 2.3.4: The outputs of each layer of the empty convolution block are added to obtain the final output of the block;

[0030] Step 2.4: After the empty convolution block, a decoder is constructed and connected, the decoder is composed of five segments, if the feature map size reaches , the decoder can be exited in advance; then the outputs of each layer of the ResNet encoder are jump-spliced with the feature maps of the same size output by each segment of the decoder; the jump-splicing is specifically that the feature maps output by each layer of the ResNet encoder are jump-spliced with the feature maps of the same size output by each segment of the decoder, and the spliced feature maps are input into the next segment of the decoder;

[0031] The construction steps of the decoder are as follows:

[0032] Step 2.4.1: First, connect a convolution layer with a convolution kernel size of 1, then process it through Batch Norm and ReLU activation function;

[0033] Step 2.4.2: Then connect a transposed convolution:

[0034] ConvTranspose2d(kernel_size=3, stride=2, padding=1, output_padding=1)

[0035] ConvTranspose2d() is a transposed convolution function, the transposed convolution kernel size is 3, the stride is the step length, the convolution padding is 1, the output feature map padding is 1, and then the Batch Norm and ReLU activation function are processed;

[0036] Step 2.4.3: Finally, a convolution layer with a convolution kernel size of 1 is connected, and then the Batch Norm and ReLU activation function are processed;

[0037] Step 2.4.4: Steps 2.4.1 to 2.4.3 are encapsulated as a section, and five sections are constructed in this process;

[0038] Step 2.5: The decoder is constructed and connected to the road extraction block, and the construction of the road extraction block includes:

[0039] Step 2.5.1: A transposed convolution layer is connected, and then the Batch Norm and ReLU activation function are processed, so that the feature map size becomes the original image size;

[0040] Step 2.5.2: Three convolution layers with a convolution kernel size of 3, a convolution padding of 1 and a stride of 1 are connected, so that the number of feature maps is reduced to 1, and the ReLU activation function is processed between every two feature maps;

[0041] Step 2.5.3: A Sigmoid activation function is connected to obtain a result image with the same size as the input image and a single channel.

[0042] Preferably, during the training of the training model, the initial learning rate is 2e-4, if the loss value does not decrease for more than three consecutive rounds, the learning rate is reduced by 5 times, and if the loss value does not decrease for more than six consecutive rounds, the training is completed by default, and the single card RTX3060 is trained.

[0043] The present application has the advantages that it can be widely applied to map drawing, traffic rescue, vehicle navigation and other practical problems, and can improve the attention of road features and the effective perception of global vision to make the precision of remote sensing image road extraction more accurate. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 It is the technical flow chart of the present application;

[0045] Figure 2 It is the road extraction network structure diagram of the present application;

[0046] Figure 3 It is the hole convolution block of the present application;

[0047] Figure 4 It is the road extraction block of the present application. DETAILED DESCRIPTION

[0048] The application will be described in further detail below with reference to the drawings and specific embodiments:

[0049] A remote sensing image road extraction method based on a double-encoder network model, comprising the following steps: Figure 1

[0050] Step 1: image enhancement is performed on the remote sensing images in the data set, and random saturation value, hue transformation and translation, rotation processing are performed on the remote sensing images;

[0051] Step 2: as shown in Figure 2 , a road extraction network model is constructed;

[0052] Step 2.1: a Swin-T encoder is built;

[0053] Step 2.2: a ResNet encoder is built;

[0054] Step 2.3: as shown in Figure 3 , a dilated convolution block is built;

[0055] Step 2.4: a decoder is built;

[0056] Step 2.5: as shown in Figure 4 , a road extraction block is built to obtain a gray-scale image;

[0057] Step 3: model training, inputting the remote sensing image data into the training model for training.

[0058] The specific steps of step 2 include:

[0059] Step 2.1: the Swin-T encoder takes Swin Transformer as the basic framework, and the Swin-T encoder is: first, a sequence mapping block, then four segments are connected in order; the four segments are respectively 2 layers of Swin Transformer for a segment, 2 layers of Swin Transformer for a segment, 6 layers of Swin Transformer for a segment, and 2 layers of Swin Transformer for a segment, and there is a sequence fusion block between every two segments;

[0060] Step 2.1.1: the sequence mapping block construction step: one convolution is performed to make the output channel 96, the convolution kernel size is 4, and the step is 4, so that the pixel points are mapped to a sequence; a Layer Norm layer is connected;

[0061] Step 2.1.2: the sequence fusion block construction step: first, a Layer Norm layer is connected; one dilated convolution is used to double the number of feature maps and reduce the size ​​

[0062] Step 2.2: ResNet encoder takes ResNet34 as the basic framework, and the ResNet encoder is: first an adaptive convolution, a Batch Norm, a ReLU activation function, so that the picture specification is 512x512x64, then a down-sampling is performed, and finally four segments are connected, which are layer1, layer2, layer3 and layer4 of ResNet34 respectively;

[0063] Step 2.3: a dilated convolution block is constructed and connected after the Swin-T encoder; the construction steps of the dilated convolution block are as follows:

[0064] Step 2.3.1: a Layer Norm layer is connected;

[0065] Step 2.3.2: n dilated convolution layers are connected:

[0066] Conv2d(kernel_size=3,dilation=1,padding=1)

[0067] Conv2d(kernel_size=3,dilation=2,padding=2)

[0068] …

[0069] Conv2d(kernel_size=3,dilation=2 n-1 ,padding=2 n-1 )

[0070] Conv2d() is a convolution function, kernel_size is the convolution kernel size, dilation is the dilation factor, and the dilation factors of n layers are 1, 2 1 ,..., 2 n-1 , padding is the convolution padding, and the convolution paddings of n layers are 1, 2 1 ,..., 2 n-1 , wherein size is the feature map size;

[0071] Step 2.3.3: a ReLU activation function is followed after each dilated convolution layer;

[0072] Step 2.3.4: the outputs of each layer of the dilated convolution block are added to obtain the final output of the block;

[0073] Step 2.4: a decoder is constructed and connected after the dilated convolution block, and the decoder is composed of five segments, and if the feature map size reaches Then the decoder can be jumped out in advance; then the output of each layer of the ResNet encoder is jump-spliced with the feature map of the same size output by each segment of the decoder; the jump-splicing is specifically that the feature map output by each layer of the ResNet encoder is jump-spliced with the feature map of the same size output by each segment of the decoder, and the spliced feature map is input to the next segment of the decoder;

[0074] The building steps of the decoder are as follows:

[0075] Step 2.4.1: first connect a convolution layer with a convolution kernel size of 1, then process it through Batch Norm and ReLU activation function;

[0076] Step 2.4.2: then connect a transposed convolution:

[0077] ConvTranspose2d(kernel_size=3,stride=2,padding=1,output_padding=1)

[0078] ConvTranspose2d() is a transposed convolution function, the transposed convolution kernel size is 3, the stride is the step length, the convolution padding is 1, and the output feature map padding is 1, then process it through Batch Norm and ReLU activation function;

[0079] Step 2.4.3: finally connect a convolution layer with a convolution kernel size of 1, then process it through Batch Norm and ReLU activation function;

[0080] Step 2.4.4: encapsulate steps 2.4.1 to 2.4.3 as a segment, and build five segments in this process;

[0081] Step 2.5: build and connect the road extraction block after the decoder, and the construction of the road extraction block includes:

[0082] Step 2.5.1: connect a transposed convolution layer, then process it through Batch Norm and ReLU activation function, so that the feature map size becomes the size of the original image;

[0083] Step 2.5.2: connect three convolution layers with a convolution kernel size of 3, a convolution padding of 1, and a stride of 1, so that the number of feature maps is reduced to 1, and ReLU activation function is processed between every two feature maps;

[0084] Step 2.5.3: connect a Sigmoid activation function to get a single-channel result image with the same size as the input image.

[0085] The initial learning rate is 2e-4 during the training of the training model, if the loss value does not decrease for more than three rounds in succession, the learning rate is attenuated by 5 times, if the loss value does not decrease for more than six rounds in succession, the training is completed by default, and the single-card RTX3060 is trained.

[0086] Of course, the above description is not a limitation of the present application, and the present application is not limited to the above description. Changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present application should also be within the scope of the present application.

Claims

1. A method for road extraction from remote sensing images based on a dual-encoder network model, characterized in that, Includes the following steps: Step 1: Perform image enhancement on the remote sensing images in the dataset, and... The remote sensing images are processed with random saturation values, hue transformations, translations, and rotations. Step 2: Construct a road extraction network model; Step 2.1: Set up the Swin-T encoder; Step 2.2: Build the ResNet encoder; Step 2.3: Construct a dilated convolution block; Step 2.4: Build the decoder; Step 2.5: Construct road extraction blocks to obtain a grayscale image; Step 3: Model training. Input the remote sensing image data into the training model for training. Step 2 includes the following specific steps: Step 2.1: The Swin-T encoder uses Swin Transformer as its basic framework. The Swin-T encoder consists of: first, a sequence mapping block, and then four segments connected in sequence; the four segments are: one with 2 layers of Swin Transformer, one with 2 layers of Swin Transformer, one with 6 layers of Swin Transformer, and one with 2 layers of Swin Transformer, with a sequence fusion block between each pair of segments. Step 2.1.1: Construction steps of the sequence mapping block: Perform a convolution to make the output channel 96, the convolution kernel size 4, and the stride 4, so that the pixels are mapped to a sequence; connect a Layer Norm layer; Step 2.1.2: Construction steps of the sequence fusion block: First, connect a Layer Norm layer; use a dilated convolution to double the number of feature maps and reduce their size. ; Step 2.2: The ResNet encoder uses ResNet34 as its basic framework. The ResNet encoder consists of: first, an adaptive convolution, then a batch normalization, and finally a ReLU activation function, to normalize the image. Then, a downsampling is performed, and finally four segments are connected, which are ResNet34's layer 1, layer 2, layer 3 and layer 4 respectively; Step 2.3: Construct and connect dilated convolutional blocks after the Swin-T encoder; the steps for constructing the dilated convolutional blocks are as follows: Step 2.3.1: Connect a Layer Norm layer; Step 2.3.2: Connect n dilated convolutional layers: It is a convolution function. The kernel size is denoted by 'm', and the dilation factor is 'dilution factor'. The dilation factors for n layers are 1 and 2 respectively. 1 , . . , 2 n-1 padding is the padding for convolutions; the padding for n layers of convolutions is 1 or 2 respectively. 1 , . . , 2 n-1 ,in size is the size of the feature map; Step 2.3.3: Each dilated convolutional layer is followed by a ReLU activation function; Step 2.3.4: Add the outputs of each layer of the dilated convolution block to obtain the final output of this block; Step 2.4: After the dilated convolutional block, construct and connect the decoder. The decoder consists of five segments. If the feature map size reaches the target size in advance... If so, the decoder can be exited early; then the output of each layer of the ResNet encoder is spliced ​​with the feature maps of the same size of each segment of the decoder; the splicing is specifically, the feature maps output by each layer of the ResNet encoder are spliced ​​with the feature maps of the same size output by each segment of the decoder, and the spliced ​​feature map is input into the next segment of the decoder. The steps for building the decoder are as follows: Step 2.4.1: First, connect a convolutional layer with a kernel size of 1, and then process it with Batch Norm and ReLU activation functions; Step 2.4.2: Connect another transposed convolution: The transposed convolution function has a kernel size of 3, a stride of 1, a padding value of 1, and an output feature map padding value of 1. It is then processed by Batch Norm and ReLU activation functions. Step 2.4.3: Finally, connect a convolutional layer with a kernel size of 1, and then process it with Batch Norm and ReLU activation functions; Step 2.4.4: Encapsulate steps 2.4.1 to 2.4.3 into one segment, and build five segments in a loop following this process; Step 2.5: After decoding, construct and connect the road extract blocks. The construction of the road extract blocks includes: Step 2.5.1: Connect a transposed convolutional layer, and then process it with Batch Norm and ReLU activation functions to make the feature map size the same as the original image size; Step 2.5.2: Connect three convolutional layers with a kernel size of 3, padding of 1, and stride of 1 to reduce the number of feature maps to 1. Perform ReLU activation function processing between every two feature maps. Step 2.5.3: Connect a Sigmoid activation function to obtain a single-channel result image with the same size as the input image.

2. The method for road extraction from remote sensing images based on a dual-encoder network model according to claim 1, characterized in that, During training, the initial learning rate of the training model is 2e-4. If the loss value does not decrease for three or more consecutive rounds, the learning rate is reduced by 5 times. If the loss value does not decrease for six consecutive rounds, the training is considered complete. Training is performed on a single RTX 3060 GPU.

Images