Urban road remote sensing image extraction method focusing on context information

Abstract

The invention relates to an urban road remote sensing image extraction method focusing on context information. Compared with the prior art, the method solves the defect that the context information extraction of a target is insufficient. The method comprises the following steps: acquiring and preprocessing an urban road remote sensing image; constructing a road remote sensing image extraction model; training a road remote sensing image extraction model; acquiring a remote sensing image of an urban road to be segmented; and extracting remote sensing image road information. According to the method, the context information of the target is fully reserved by using the multi-scale aggregation module, on the basis, the self-attention segmentation module strengthens the boundary to prevent the target from being over-fitted with the context information, and finally, the graph convolution module performs global association integration, so that accurate extraction of the urban road remote sensing image is realized.

Description

Technical Field

[0001] The invention relates to the technical field of remote sensing image processing, and in particular to a method for extracting urban road remote sensing images focusing on context information. Background Art

[0002] Remote sensing extraction of urban roads is a part of extracting urban road network information using high-resolution remote sensing images, which is of great significance for the sustainable development of urban transportation. People have proposed various road extraction algorithms based on high-resolution remote sensing images. Manual and traditional methods of extracting roads from high-resolution remote sensing images are not very effective. The use of deep learning methods to automatically and finely extract urban roads from remote sensing images has become the current mainstream method. However, most deep learning networks use classic encoder and decoder structures. In the downsampling process, some important feature information will inevitably be lost, and the fixed standard convolution kernel will lose some context information, which is very important for feature extraction.

[0003] Context information is the background information of the target. Zhang et al. believe that background information helps to extract occlusions and small objects in the scene; Li et al. focus on background information and propose a large selection convolution kernel network for target detection scenarios, emphasizing the important role of background information in target detection. Their experimental results show that the use of context information can significantly improve the detection performance of small targets. Zhang et al. pointed out that good background information can effectively distinguish occluded or similar-colored objects, thereby improving detection accuracy. In the road segmentation task, context information cannot be ignored. Since roads are usually in a complex background, accurate extraction of road areas requires consideration of the influence of surrounding elements.

[0004] In traditional convolutional neural networks (CNNs), the encoder-decoder structure is an important design pattern and is widely used in image processing tasks such as image segmentation and super-resolution reconstruction. The encoder is responsible for converting the input image into a low-dimensional potential representation, which is generally achieved through a series of convolutional layers and pooling layers. This process extracts key features in the image and compresses the amount of information. The decoder uses deconvolution or transposed convolution layers to gradually restore the spatial dimensions of the image to generate the final output. The advantage of the encoder-decoder structure lies in its powerful feature extraction capability and flexibility, and is suitable for a variety of input and output formats, which also makes it widely used in image reconstruction tasks.

[0005] Since the decoder needs to rely on the information transmitted by the encoder, the collaboration between the encoder and the decoder is also extremely important. Once the representation generated by the encoder is not effective enough, the performance of the decoder will be seriously affected. Although the use of skip connections solves the problem of missing early information during upsampling, the feature information learned by the model during downsampling is still feature information that has lost some features. In order to solve the problem of feature information loss during downsampling, many researchers have proposed to reduce feature loss by changing the downsampling pooling operation to convolution downsampling, or using different dimensional skip connections such as feature pyramids, TopFormer and other network models, or using the Patch Merging module in Swin Trainsformer to downsample images.

[0006] The encoder and decoder structures in CNN networks have been widely used in road detection scenarios and are relatively mature. However, due to the complexity of roads, especially in densely built areas, the surrounding environment of roads is very complex, and there are still some challenges:

[0007] (1) Due to the abstract nature of roads, their shapes and textures are diverse, and it is difficult to obtain detailed feature information of roads using standard convolution kernels.

[0008] (2) The background information around roads in densely built-up areas is complex and diverse, which may block the roads or be too similar to the roads, making it difficult to extract and segment the roads. However, this background information is also very important for road extraction and cannot be discarded directly.

[0009] (3) The standard convolution kernel can gradually expand the receptive field by repeatedly performing convolution operations, thereby capturing global features. However, in this process, the loss of feature information is an inevitable problem.

[0010] To address these challenges and further improve the accuracy of road segmentation, we propose an encoder-decoder based network for extracting high-resolution remote sensing images of roads in densely built-up areas of cities. Summary of the invention

[0011] The purpose of the present invention is to solve the defect of insufficient extraction of target context information in the prior art and to provide a method for extracting urban road remote sensing images focusing on context information to solve the above problem.

[0012] In order to achieve the above object, the technical solution of the present invention is as follows:

[0013] A method for extracting urban road remote sensing images focusing on context information comprises the following steps:

[0014] Acquisition and preprocessing of urban road remote sensing images: Acquisition and preprocessing of urban road remote sensing images;

[0015] Construction of road remote sensing image extraction model: Construct a road remote sensing image extraction model based on UNet as the framework and Resnet-50 as the main body;

[0016] Training of road remote sensing image extraction model: input the pre-processed urban road remote sensing image into the road remote sensing image extraction model for training;

[0017] Acquisition of remote sensing images of urban roads to be segmented: Acquisition of remote sensing images of urban roads to be segmented and preprocessing;

[0018] Extraction of road information from remote sensing images: The preprocessed urban road remote sensing images to be segmented are input into the trained road remote sensing image extraction model to obtain the segmentation and extraction results of the remote sensing image road information.

[0019] The construction of the road remote sensing image extraction model includes the following steps:

[0020] The road remote sensing image extraction model is set to include a downsampling layer, a high-dimensional transition layer, and an upsampling layer;

[0021] Set the downsampling layer:

[0022] The initial input RGB image is reduced to a feature map of 1 / 4 of the original through three downsamplings. After each downsampling, the feature size is halved and the number of channels is doubled;

[0023] The first two downsampling structures are the same. Each downsampling first passes through three groups of 3×3 convolutions. After each group of 3×3 convolutions, Batch Normalization and Relu activation function are used to prevent gradient explosion. Multi-scale information is then aggregated through a multi-scale aggregation module. Finally, an adaptive Sobel operator module is used to prevent overfitting of target and environment information.

[0024] The third downsampling first passes through the Maxpool layer to halve the feature size, then aggregates global information through the graph convolution module, then passes through the first layer of ResNet-50, and finally passes through the multi-scale aggregation module and the adaptive Sobel operator module before output;

[0025] Set the high-dimensional transition layer:

[0026] The high-dimensional transition layer is composed of the second, third, and fourth layers of ResNet-50, followed by a 3×3 convolution to change the channel dimension;

[0027] Set the upsampling layer:

[0028] The upsampling layer restores the feature map output by the high-dimensional transition layer to the original image resolution through three upsamplings. Each time the upsampled feature passes through a transposed convolution and two 3×3 convolutions, the feature size is doubled and the number of channels is halved. Each 3×3 convolution must be followed by BN and ReLU activation functions in turn to prevent gradient explosion during training.

[0029] The first, second, and third upsampling correspond to the third, second, and first downsampling respectively. The output of the multi-scale aggregation module in each downsampling is added to the output of the transposed convolution in the corresponding upsampling process using a skip link. Finally, the image pixels are divided into two categories through 1×1 convolution layer dimension classification, one is the background and the other is the extracted road.

[0030] Setting ResNet-50: Set ResNet-50 to include four layers, each of which is composed of repeated residual blocks. The first layer of ResNet-50 is composed of three residual blocks stacked, the second layer is composed of four residual blocks stacked, the third layer is composed of six residual blocks stacked, and the fourth layer is composed of three residual blocks stacked;

[0031] Each residual block is composed of 1×1 convolution, 3×3 convolution and 1×1 convolution stacked in sequence, and each convolution is followed by BN normalization and Relu activation function to prevent gradient explosion. Each residual block introduces a skip connection, that is, when the feature size and dimension of the input and output of the residual block do not change, the input and output of the current residual block are added as the input of the next residual block;

[0032] Set up the multi-scale aggregation module:

[0033] The multi-scale aggregation module in the first and second downsampling takes the output feature tensors of the three groups of 3×3 convolutions in the downsampling as its input, and the multi-scale aggregation module in the third downsampling takes the output feature tensors of the first layer of ResNet-50 in the downsampling as its input, obtaining multi-scale information from the multi-scale image and the multi-scale convolution kernel;

[0034] The input feature tensor X passes through two 1×1 convolution kernels in parallel, outputting two tensors respectively, and the channel dimension is reduced to 1 / 2 of X;

[0035] A tensor passes through the multi-scale part of the convolution kernel and passes through four convolution blocks: one 1×1 convolution, three 3×3 convolutions with expansion rates of 1, 2, and 3 respectively, outputting four tensors, and the channel dimension is reduced to 1 / 8 of X;

[0036] Another tensor passes through the multi-scale part of the image: the feature map is extracted at intervals of one point in both length and width in order through the Patch Meaging operation, and is divided into four tensors with half the length and width. The length and width of the four tensors are changed to 1 / 2, 1 / 4, 1 / 6, and 1 / 8 of the starting X through the Maxpool operation, and each is passed through a standard convolution kernel with a convolution kernel size of 3×3, and then restored to the size before the Maxpool operation through Bilinear. Finally, according to the reverse operation of Patch Meaging, the four tensors are merged into one tensor output in the original order.

[0037] Finally, the outputs of the multi-scale part of the convolution kernel and the multi-scale part of the image are spliced ​​together through the channel dimension, added to the input jump connection, and then passed through a 1×1 convolution kernel to output a feature tensor of the same size and dimension as the starting input tensor X. After each convolution operation, a BN layer and Relu activation operation are performed to prevent gradient explosion;

[0038] Set the adaptive Sobel operator module:

[0039] The output feature tensor Y of the multi-scale aggregation module is used as the input of the adaptive Sobel operator module. Y passes through 5×5 convolution and 7×7 convolution in turn. The outputs of the two are respectively passed through 1×1 convolution to halve the number of channels. Then, they are respectively passed through the Avgpool layer and the Maxpool layer and input into the Sigmoid activation layer. After that, the results are multiplied with the output results of the 1×1 convolution layer and merged in the channel dimension to output a feature tensor of the same size as Y. After that, the feature tensor is divided into four feature parts through the Patch Meaging operation, and is respectively convolved with the Sobel operator and then reversed through the Patch Meaging operation to merge the four features. Then, after passing through the 1×1 convolution, the parts are added to the input feature tensor Y and then output. After each convolution operation, the BN layer and Relu activation operation are performed to prevent gradient explosion.

[0040] Setting up the graph convolution module:

[0041] The output feature tensor Z after the Maxpool operation in the third downsampling is used as the input of the graph convolution module. The feature tensor Z first passes through a 1×1 convolution, and then is divided into four feature tensors with a length and width of 1 / 2 of the feature tensor Z through a Patch Meaging operation. The four feature vectors pass through two layers of graph convolution respectively. The graph convolution calculation formula is as follows:

[0042] H (L+1) =σ(D -1 / 2 A 1 D -1 / 2 H (L) W (L) ) (1)

[0044] Among them, H (L) Represents the feature vector (b, c, w of the current layer of graph convolution module 0 ,h 0 ) Input, H (L+1) Represents the feature vector (b, c, w of the current layer of graph convolution module 0 ,h 0 ) output;

[0045] A1 in formula (1) is the final adjacency matrix calculated by formula (2). The relationship between each element is the edge to construct the initial adjacency matrix A. 0 , A 0 By transforming the feature tensor H (L) (b,c,w 0 ,h 0 ) are transformed into (b,c,w) through reshape operation. 0 ×h 0 ) and (b,w 0 ×h 0 ,c) size feature tensor, the two are multiplied to get (b,w 0 ×h 0 ,w 0 ×h 0 ) size tensor, i.e. A 0 , where I is the same as A 0 The same size unit matrix and A 0 Multiply and add A 0 , output tensor A 1 (b,w 0 ×h 0 ,w 0 ×h 0 );

[0046] A 1 =A 0 ×I+A 0 (2)

[0048] (1) D -1 / 2 Represents a mathematical operation of performing a half-order inverse matrix on the D matrix, where D is A 1 The degree matrix is ​​set to be composed of A 1 The sum of each row gives a diagonal matrix, D ii represents the i-th row and i-th column of the D matrix, which is composed of A 1 Add all the elements in the corresponding row i and output the tensor D -1 / 2 (b,w 0 ×h 0 ,w 0 ×h 0), the formula is as follows:

[0049] D ii =∑ j (A 1 ) ij (3)

[0050] (1)W in the formula (L) It is the weight parameter after each layer of graph convolution propagation. In the network model, it is replaced by a 1×1 convolution;

[0051] In (1), σ represents the activation function, which is replaced by the ReLU activation function in the model;

[0052] After repeating the graph convolution twice, the input of the second graph convolution is the output of the first graph convolution, and four feature vectors H are output. (L+1) (b,c,w 0 ,h 0 ), H (L+1) It passes through 4 convolution blocks respectively: one 1×1 convolution, three 3×3 convolutions with expansion rates of 1, 2, and 3 respectively. After each convolution operation, a BN layer and Relu activation operation are performed to prevent gradient explosion. The four feature vectors obtained are then inversely operated by Patch Meaging and then 1×1 convolution and added to the original feature vector Z of the graph convolution module. Finally, a feature vector of the same size as the feature tensor Z is output.

[0053] The training of the road remote sensing image extraction model includes the following steps:

[0054] The preprocessed remote sensing image data was input into the road remote sensing image extraction model, and the binary cross entropy function was used as the loss function. The initial learning rate was 0.0001, the number of training iterations was 50, 50, and 100, and the step size was 8.

[0055] Start by inputting an 8-batch RGB image with a size of 256×256 and 3 channels, and convert it into a feature tensor X 00 The size is (8,3,256,256) and enters the downsampling layer of the road remote sensing image extraction model:

[0056] First downsampling, feature tensor X 00 (8,3,256,256) After the first set of 3×3 convolutions, the feature tensor size becomes X 01 (8,64,256,256), X 01 After two sets of 3×3 convolutions, the feature tensor size is not changed and the output feature tensor X is obtained. 02 (8,64,256,256), X 02 Output feature tensor Y through the first multi-scale aggregation module 00(8,64,256,256), Y 00 After passing the first adaptive Sobel operator, the output feature tensor Y 01 (8,64,256,256);

[0057] Second downsampling, feature tensor Y 01 (8, 64, 256, 256) After the first set of 3×3 convolutions, the feature tensor size becomes X 11 (8,64,128,128), X 11 After the second set of 3×3 convolutions, the feature tensor size is not changed and the output feature tensor X is obtained. 12 (8,64,128,128), X 12 After the third set of 3×3 convolutions, the output feature tensor X 13 (8,128,128,128), X 13 Output feature tensor Y through the second multi-scale aggregation module 10 (8,128,128,128), Y 10 After passing the second adaptive Sobel operator, the output feature tensor Y 11 (8,128,128,128);

[0058] The third downsampling, feature tensor Y 11 (8,128,128,128) First use Maxpool to halve the feature size and keep the number of channels unchanged to output the feature tensor X 20 (8,128,64,64), X 20 Output feature tensor X without changing feature size through graph convolution module 21 (8,128,64,64), X 21 The number of feature map channels is halved through the 1×1 convolution of the first residual block of the first layer of ResNet-50, and the output feature tensor X remains unchanged in size. 22 (8,64,64,64), X 22 The number of feature map channels is quadrupled through the 3×3 convolution of the first residual block of the first layer of ResNet-50, and the output feature tensor X remains unchanged in size. 23 (8,256,64,64), X 23 After passing the last 1×1 convolution of the first residual block of the first layer of ResNet-50, the output feature tensor X 24 (8,256,64,64), X 24 After passing through the second and third residual blocks of the first layer of ResNet-50, the feature tensor X is output. 25 (8,256,64,64), X 25 After passing through the third multi-scale aggregation module, the output feature tensor Y20 (8,256,64,64), Y 20 After passing the third adaptive Sobel operator, the feature size is not changed and the output feature tensor Y 21 (8,256,64,64);

[0059] The downsampled output enters the high-dimensional transition layer:

[0060] The feature tensor Y output by the downsampling layer 21 (8,256,64,64), the number of feature map channels is halved through the 1×1 convolution of the first residual block of the second layer of ResNet-50, and the output feature tensor Y is unchanged in size 31 (8,128,64,64), Y 31 The feature map size is halved through the 3×3 convolution of the first residual block in the second layer, and the number of channels is quadrupled to output the feature tensor Y 32 (8,512,32,32), Y 32 Output Y through the last 1×1 convolution of the first residual block in the second layer 33 (8,512,32,32), Y 33 After passing through the second, third, and fourth residual blocks of the second layer of ResNet-50 in sequence, the feature vector size is not changed and the feature tensor Y is output 34 (8,512,32,32);

[0061] Y 34 The number of feature map channels is halved through the 1×1 convolution of the first residual block in the third layer of ResNet-50, and the output Y is unchanged in size. 41 (8,256,32,32), Y 41 The feature map size is halved through the 3×3 convolution of the first residual block in the third layer, and the number of channels is quadrupled to output Y 42 (8,1024,32,32), Y 42 Output Y through the last 1×1 convolution of the first residual block in the third layer 43 (8,1024,32,32), Y 43 After passing through the five residual blocks after the third layer of ResNet-50, the feature vector size is not changed and the feature tensor Y is output. 44 (8,1024,32,32);

[0062] Y 44 The number of feature map channels is halved through the 1×1 convolution of the first residual block in the fourth layer of ResNet-50, and the output Y is unchanged in size. 51 (8,512,32,32), Y 51 The feature map size is halved through the 3×3 convolution of the first residual block in the fourth layer, and the number of channels is quadrupled to output Y52 (8,2048,32,32), Y 52 Output Y through the last 1×1 convolution of the first residual block in the fourth layer 53 (8,2048,32,32), Y 53 After passing through the two residual blocks after the fourth layer of ResNet-50, the feature vector size is not changed and the feature tensor Y is output 54 (8,2048,32,32);

[0063] Y 54 Through a 3×3 convolution, the number of channels is reduced to 1 / 4 and the size remains unchanged to output the feature tensor Y 55 (8,512,32,32);

[0064] 34) The outputs of the high-dimensional transition layer and the downsampling layer are input into the upsampling layer, and the upsampling is performed three times:

[0065] The first upsampling converts the output Y of the high-dimensional transition layer 55 (8,512,32,32) uses transposed convolution to halve its number of channels and double its size to output the feature vector Z 01 (8,256,64,64) and the output Y after the multi-scale aggregation module in the third downsampling process 20 Concatenate the output feature vector Z in the channel dimension 02 (8,512,64,64), Z 02 The number of channels is halved through 3×3 convolution to output the feature vector Z 03 (8,256,64,64), Z 03 The feature vector Z is output without changing the feature size through the last set of 3×3 convolutions 04 (8,256,64,64);

[0066] The second upsampling will Z 04 Through transposed convolution, the number of channels is halved and the size is doubled to output the feature vector Z 11 (8,128,128,128) and the output Y after the multi-scale aggregation module in the second downsampling process 10 Concatenate the output feature vector Z in the channel dimension 12 (8,256,128,128), Z 12 The number of channels is halved through 3×3 convolution to output the feature vector Z 13 (8,128,128,128), Z 13 The feature vector Z is output without changing the feature size through the last set of 3×3 convolutions 14 (8,128,128,128);

[0067] The third upsampling will Z14 Through the transposed convolution, the number of channels is halved and the size is doubled to output the feature vector Z 21 (8,64,256,256) and the output Y after the multi-scale aggregation module in the second downsampling process 00 Concatenate the output feature vector Z in the channel dimension 22 (8,128,256,256), Z 22 The number of channels is halved through 3×3 convolution to output the feature vector Z 23 (8,64,256,256), Z 23 The feature vector Z is output without changing the feature size through the last set of 3×3 convolutions 24 (8,64,256,256);

[0068] Z 24 Finally, a 1×1 convolution is used to adjust the channels of the final feature layer to the number of categories and then output the feature tensor Z(8,2,256,256). The feature tensor Z(8,2,256,256) is converted into an image representation of 8 batches of images with a size of 256×256 and a channel number of 2, which is used as the final urban road remote sensing image extraction result.

[0069] Beneficial Effects

[0070] The present invention provides a method for extracting urban road remote sensing images that focuses on context information. Compared with the prior art, the method fully retains the target context information by using a multi-scale aggregation module. On this basis, the self-attention segmentation module strengthens the boundary to prevent overfitting of the target and context information. Finally, the graph convolution module performs global correlation integration, thereby realizing the accurate extraction of urban road remote sensing images.

[0071] The multi-scale aggregation module of the present invention effectively integrates the multi-scale information of the convolution kernel and the multi-scale information of the image; the graph convolution module adopts the idea of ​​Patch Merging to reduce the size of the feature map and obtain global correlation information, which greatly reduces the amount of calculation while ensuring accuracy; the self-attention segmentation module based on the Sobel operator strengthens the boundary between the target and the surrounding environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 is a method sequence diagram of the present invention;

[0073] Figure 2 It is a structural schematic diagram of the road remote sensing image extraction model involved in the present invention;

[0074] Figure 3 It is a schematic diagram of the structure of the multi-scale aggregation module involved in the present invention;

[0075] Figure 4It is a structural schematic diagram of the adaptive Sobel operator module involved in the present invention;

[0076] Figure 5 It is a structural schematic diagram of the graph convolution module involved in the present invention;

[0077] Figure 6 The output comparison results of the model involved in the present invention with different network models in the Massachusetts road dataset;

[0078] Figure 7 This is the output comparison result of the model involved in the present invention with different network models in DeepGlobe Roads Dataset. DETAILED DESCRIPTION

[0079] In order to have a further understanding and recognition of the structural features and the effects achieved by the present invention, a preferred embodiment and accompanying drawings are used for detailed description as follows:

[0080] like Figure 1 As shown, the urban road remote sensing image extraction method focusing on context information described in the present invention comprises the following steps:

[0081] The first step is to obtain and preprocess the remote sensing images of urban roads: obtain the remote sensing images of urban roads and perform preprocessing.

[0082] The second step is to build a road remote sensing image extraction model: Figure 2 As shown in the figure, a road remote sensing image extraction model is constructed based on UNet as the framework and ResNet-50 as the main body. The UNet architecture performs well in binary classification tasks. Its jump connection design can effectively capture contextual information of different scales and is suitable for scenarios that require precise boundaries and details. ResNet-50, due to its residual learning concept, allows deeper network training without the problem of gradient disappearance, and can extract deep features in more detail.

[0083] (1) The road remote sensing image extraction model is set to include a downsampling layer, a high-dimensional transition layer, and an upsampling layer.

[0084] (2) Setting the downsampling layer:

[0085] The initial input RGB image is reduced to a feature map of 1 / 4 of the original through three downsamplings. After each downsampling, the feature size is halved and the number of channels is doubled;

[0086] The first two downsampling structures are the same. Each downsampling first passes through three groups of 3×3 convolutions. After each group of 3×3 convolutions, Batch Normalization and Relu activation function are used to prevent gradient explosion. Multi-scale information is then aggregated through a multi-scale aggregation module. Finally, an adaptive Sobel operator module is used to prevent overfitting of target and environment information.

[0087] The third downsampling first passes through the Maxpool layer to halve the feature size, then aggregates global information through the graph convolution module, then passes through the first layer of ResNet-50, and finally passes through the multi-scale aggregation module and the adaptive Sobel operator module before output.

[0088] (3) Setting a high-dimensional transition layer:

[0089] The high-dimensional transition layer is composed of the second, third, and fourth layers of ResNet-50, followed by a 3×3 convolution to change the channel dimension.

[0090] (4) Set the upsampling layer:

[0091] The upsampling layer restores the feature map output by the high-dimensional transition layer to the original image resolution through three upsamplings. Each time the upsampled feature passes through a transposed convolution and two 3×3 convolutions, the feature size is doubled and the number of channels is halved. Each 3×3 convolution must be followed by BN and ReLU activation functions in turn to prevent gradient explosion during training.

[0092] The first, second, and third upsampling correspond to the third, second, and first downsampling respectively. The output of the multi-scale aggregation module in each downsampling is added to the output of the transposed convolution in the corresponding upsampling process using a skip link. Finally, the image pixels are divided into two categories through 1×1 convolutional layer dimension classification, one is the background and the other is the extracted road.

[0093] (5) Setting ResNet-50: ResNet-50 is set to include four layers, each of which is composed of repeated residual blocks. The first layer of ResNet-50 is composed of three residual blocks stacked, the second layer is composed of four residual blocks stacked, the third layer is composed of six residual blocks stacked, and the fourth layer is composed of three residual blocks stacked;

[0094] Each residual block is composed of 1×1 convolution, 3×3 convolution and 1×1 convolution stacked in sequence, and each convolution is followed by BN normalization and Relu activation function to prevent gradient explosion. Each residual block introduces a skip connection, that is, when the feature size and dimension of the input and output of the residual block do not change, the input and output of the current residual block are added as the input of the next residual block.

[0095] (6) Setting the multi-scale aggregation module, such as Figure 3 As shown in Figure 1, the multi-scale aggregation module is an important mechanism for improving the performance of deep learning models, especially in object detection and image segmentation tasks. It processes features of multiple scales simultaneously, allowing the model to capture information at different levels, thereby improving the detection of small and large objects.

[0096] The multi-scale aggregation module in the first and second downsampling takes the output feature tensors of the three groups of 3×3 convolutions in the downsampling as its input, and the multi-scale aggregation module in the third downsampling takes the output feature tensors of the first layer of ResNet-50 in the downsampling as its input, obtaining multi-scale information from the multi-scale image and the multi-scale convolution kernel;

[0097] The input feature tensor X passes through two 1×1 convolution kernels in parallel, outputting two tensors respectively, and the channel dimension is reduced to 1 / 2 of X;

[0098] A tensor passes through the multi-scale part of the convolution kernel: it passes through 4 convolution blocks respectively: one 1×1 convolution, three 3×3 convolutions with expansion rates of 1, 2, and 3 respectively, outputting four tensors, and the channel dimension is reduced to 1 / 8 of X;

[0099] Another tensor passes through the multi-scale part of the image: the feature map is extracted at intervals of one point in both length and width in order through the Patch Meaging operation, and is divided into four tensors with half the length and width. The length and width of the four tensors are changed to 1 / 2, 1 / 4, 1 / 6, and 1 / 8 of the starting X through the Maxpool operation, and each is passed through a standard convolution kernel with a convolution kernel size of 3×3, and then restored to the size before the Maxpool operation through Bilinear. Finally, according to the reverse operation of Patch Meaging, the four tensors are merged into one tensor output in the original order.

[0100] Finally, the outputs of the multi-scale part of the convolution kernel and the multi-scale part of the image are spliced ​​together through the channel dimension, added to the input jump connection, and then passed through a 1×1 convolution kernel to output a feature tensor of the same size and dimension as the starting input tensor X. After each convolution operation, a BN layer and ReLU activation operation are performed to prevent gradient explosion.

[0101] (7) Set the adaptive Sobel operator module, such as Figure 4 shown.

[0102] During the multi-scale aggregation process, when the target and edge information are too similar, the segmentation and detection performance may be degraded. This usually happens when the contrast between the edge of the object and the background is low, or the edge of the target is blurred. Since there is no obvious distinction between features of different scales when aggregating, the algorithm may misjudge the target as the background, or cause blurred edges during segmentation, reducing the recognizability of the target. This information confusion makes the subsequent feature extraction and classification process more complicated, ultimately affecting the overall detection accuracy.

[0103] Therefore, when designing a multi-scale aggregation method, the similarity between the target and edge information must be effectively processed to ensure accurate segmentation and recognition. In recent years, deep convolutional neural networks (CNNs) have achieved a fairly high level in edge detection with their rich and abstract edge representation capabilities. However, the high performance of CNN-based edge detection is achieved through a large amount of data pre-training, and the process is complicated. Although some traditional threshold-based segmentation algorithms (such as Sobel, Canny, Prewitt, and Scharr, etc.) have no target for edge segmentation, the algorithm is fast, does not require training data, and has a very obvious effect on edge segmentation.

[0104] In view of the advantages and disadvantages of the two, an adaptive Sobel operator module is designed to enhance the difference between the edge of the object and the background. Since it can dynamically adjust its large spatial receptive field to better simulate the ranging context of various objects in the remote sensing scene, it can achieve very good results in the target detection scenario. The lsk (Large Selective Kernel) module is adjusted and the Sobel operator is added to obtain the edge information of the target area from the x and y directions respectively, and then integrated and output.

[0105] The output feature tensor Y of the multi-scale aggregation module is used as the input of the adaptive Sobel operator module. Y passes through 5×5 convolution and 7×7 convolution in turn. The outputs of the two are respectively passed through 1×1 convolution to halve the number of channels. Then, they are respectively passed through the Avgpool layer and the Maxpool layer and input into the Sigmoid activation layer. After that, the results are multiplied with the output results of the 1×1 convolution layer and merged in the channel dimension to output a feature tensor of the same size as Y. After that, the feature tensor is divided into four feature parts through the Patch Meaging operation, which are respectively convolved with the Sobel operator and then reversed through the Patch Meaging operation to merge the four features. Then, after passing through the 1×1 convolution, they are added to the input feature tensor Y and output. After each convolution operation, the BN layer and Relu activation operation are performed to prevent gradient explosion.

[0106] (8) Set the graph convolution module, such as Figure 5As shown in the figure. On the basis of preserving context information as much as possible, it is crucial to use this context information to obtain global associations. Although traditional standard convolution can achieve global vision through multiple convolutions, it will inevitably cause the loss of target context information in the process. By using graph convolutional neural networks to analyze the association of global information on this basis, these rich context information can be fully utilized.

[0107] GCN extends the idea of ​​convolution to graph-structured data by constructing an adjacency matrix, providing a new method for processing non-Euclidean data. Initially, GCN was widely used in the field of knowledge graphs. In recent years, its application in natural image feature extraction has also gradually increased. Compared with traditional deep convolutional neural networks (DCNNs), GCN can effectively expand the receptive field and capture richer contextual information. This is because graph convolution not only considers the characteristics of the central node, but also combines the information of its neighborhood nodes, thereby achieving effective aggregation of information.

[0108] The output feature tensor Z after the Maxpool operation in the third downsampling is used as the input of the graph convolution module. The feature tensor Z first passes through a 1×1 convolution, and then is divided into four feature tensors with a length and width of 1 / 2 of the feature tensor Z through a Patch Meaging operation. The four feature vectors pass through two layers of graph convolution respectively. The graph convolution calculation formula is as follows:

[0109] H (L+1) = σ(D -1 / 2 A 1 D -1 / 2 H (L) W (L) ) (1)

[0110] Among them, H (L) Represents the feature vector (b, c, w of the current layer of graph convolution module 0 ,h 0 ) Input, H (L+1) Represents the feature vector (b, c, w of the current layer of graph convolution module 0 ,h 0 ) output;

[0111] A1 in formula (1) is the final adjacency matrix calculated by formula (2). The relationship between each element is the edge to construct the initial adjacency matrix A. 0 , A 0 By transforming the feature tensor H (L) (b,c,w 0 ,h 0 ) are transformed into (b,c,w) through reshape operation. 0 ×h 0 ) and (b,w 0 ×h0 ,c) size feature tensor, the two are multiplied to get (b,w 0 ×h 0 ,w 0 ×h 0 ) size tensor, i.e. A 0 , where I is the same as A 0 The same size unit matrix and A 0 Multiply and add A 0 , output tensor A 1 (b,w 0 ×h 0 ,w 0 ×h 0 );

[0112] A 1 =A 0 ×I+A 0 (2)

[0113] (1) D -1 / 2 Represents a mathematical operation of performing a half-order inverse matrix on the D matrix, where D is A 1 The degree matrix is ​​set to be composed of A 1 The sum of each row gives a diagonal matrix, D ii represents the i-th row and i-th column of the D matrix, which is composed of A 1 Add all the elements corresponding to the i-th row and finally output the tensor D -1 / 2 (b,w 0 ×h 0 ,w 0 ×h 0 ), the formula is as follows:

[0114] D ii =∑ j (A 1 ) ij (3)

[0115] (1)W in the formula (L) It is the weight parameter after each layer of graph convolution propagation. In the network model, it is replaced by a 1×1 convolution;

[0116] In (1), σ represents the activation function, which is replaced by the ReLU activation function in the model;

[0117] After repeating the graph convolution twice, the input of the second graph convolution is the output of the first graph convolution, and four feature vectors H are output. (L+1) (b,c,w 0 ,h 0 ), H (L+1)It passes through 4 convolution blocks respectively: one 1×1 convolution, three 3×3 convolutions with expansion rates of 1, 2, and 3 respectively. After each convolution operation, a BN layer and Relu activation operation are performed to prevent gradient explosion. The four feature vectors obtained are then inversely operated by Patch Meaging and then 1×1 convolution and added to the original feature vector Z of the graph convolution module. Finally, a feature vector of the same size as the feature tensor Z is output.

[0118] The third step is to train the road remote sensing image extraction model: input the preprocessed urban road remote sensing images into the road remote sensing image extraction model for training. The preprocessed high-quality images can improve the training efficiency and performance of the model, and can automatically learn and extract effective features from the original data through deep learning. The binary cross entropy loss function is specially designed for binary classification tasks, which can effectively measure the difference between the model output probability and the true label. Setting a smaller learning rate helps to make detailed parameter adjustments at the beginning of training, reducing the oscillation and instability that may be caused by large step sizes. The appropriate number of iterations can effectively reduce the risk of overfitting during model training, avoid unnecessary calculations, and save training time and computing resources. Training multiple batches simultaneously can increase training speed and reduce training time.

[0119] (1) The preprocessed remote sensing image data is input into the road remote sensing image extraction model, and the binary cross entropy function is used as the loss function. The initial learning rate is 0.0001, the number of training iterations is 50, 50, and 100 respectively, and the batch size is 8.

[0120] (2) Start by inputting an 8-batch RGB image with a size of 256×256 and 3 channels, and convert it into a feature tensor X 00 The size is (8,3,256,256) and enters the downsampling layer of the road remote sensing image extraction model:

[0121] First downsampling, feature tensor X 00 (8,3,256,256) After the first set of 3×3 convolutions, the feature tensor size becomes X 01 (8,64,256,256), X 01 After two sets of 3×3 convolutions, the feature tensor size is not changed and the output feature tensor X is obtained. 02 (8,64,256,256), X 02 Output feature tensor Y through the first multi-scale aggregation module 00 (8,64,256,256) ， Y 00 After passing the first adaptive Sobel operator, the output feature tensor Y 01 (8,64,256,256);

[0122] Second downsampling, feature tensor Y 01 (8, 64, 256, 256) After the first set of 3×3 convolutions, the feature tensor size becomes X 11 (8,64,128,128), X 11 After the second set of 3×3 convolutions, the feature tensor size is not changed and the output feature tensor X is obtained. 12 (8,64,128,128), X 12 After the third set of 3×3 convolutions, the output feature tensor X 13 (8,128,128,128), X 13 Output feature tensor Y through the second multi-scale aggregation module 10 (8,128,128,128), Y 10 After passing the second adaptive Sobel operator, the output feature tensor Y 11 (8,128,128,128);

[0123] The third downsampling, feature tensor Y 11 (8,128,128,128) First use Maxpool to halve the feature size and keep the number of channels unchanged to output the feature tensor X 20 (8,128,64,64), X 20 Output feature tensor X without changing feature size through graph convolution module 21 (8,128,64,64), X 21 The number of feature map channels is halved through the 1×1 convolution of the first residual block of the first layer of ResNet-50, and the output feature tensor X remains unchanged in size. 22 (8,64,64,64), X 22 The number of feature map channels is quadrupled through the 3×3 convolution of the first residual block of the first layer of ResNet-50, and the output feature tensor X remains unchanged in size. 23 (8,256,64,64), X 23 After passing the last 1×1 convolution of the first residual block of the first layer of ResNet-50, the output feature tensor X 24 (8,256,64,64), X 24 After passing through the second and third residual blocks of the first layer of ResNet-50, the feature tensor X is output. 25 (8,256,64,64), X 25 After passing through the third multi-scale aggregation module, the output feature tensor Y 20 (8,256,64,64), Y 20 After passing the third adaptive Sobel operator, the feature size is not changed and the output feature tensor Y 21 (8,256,64,64).

[0124] (3) The downsampled output enters the high-dimensional transition layer:

[0125] The feature tensor Y output by the downsampling layer 21 (8,256,64,64), the number of feature map channels is halved through the 1×1 convolution of the first residual block of the second layer of ResNet-50, and the size of the output feature tensor Y remains unchanged 31 (8,128,64,64), Y 31 The feature map size is halved through the 3×3 convolution of the first residual block in the second layer, and the number of channels is quadrupled to output the feature tensor Y 32 (8,512,32,32), Y 32 Output Y through the last 1×1 convolution of the first residual block in the second layer 33 (8,512,32,32), Y 33 After passing through the second, third, and fourth residual blocks of the second layer of ResNet-50 in sequence, the feature vector size is not changed and the feature tensor Y is output 34 (8,512,32,32);

[0126] Y 34 The number of feature map channels is halved through the 1×1 convolution of the first residual block in the third layer of ResNet-50, and the output Y is unchanged in size. 41 (8,256,32,32), Y 41 The feature map size is halved through the 3×3 convolution of the first residual block in the third layer, and the number of channels is quadrupled to output Y 42 (8,1024,32,32), Y 42 Output Y through the last 1×1 convolution of the first residual block in the third layer 43 (8,1024,32,32), Y 43 After passing through the five residual blocks after the third layer of ResNet-50, the feature vector size is not changed and the feature tensor Y is output. 44 (8,1024,32,32);

[0127] Y 44 The number of feature map channels is halved through the 1×1 convolution of the first residual block in the fourth layer of ResNet-50, and the output Y is unchanged in size. 51 (8,512,32,32), Y 51 The feature map size is halved through the 3×3 convolution of the first residual block in the fourth layer, and the number of channels is quadrupled to output Y 52 (8,2048,32,32), Y 52 Output Y through the last 1×1 convolution of the first residual block in the fourth layer 53 (8,2048,32,32), Y53 After passing through the two residual blocks after the fourth layer of ResNet-50, the feature vector size is not changed and the feature tensor Y is output 54 (8,2048,32,32);

[0128] Y 54 Through a 3×3 convolution, the number of channels is reduced to 1 / 4 and the size remains unchanged to output the feature tensor Y 55 (8,512,32,32);

[0129] 34) The outputs of the high-dimensional transition layer and the downsampling layer are input into the upsampling layer, and the upsampling is performed three times:

[0130] The first upsampling converts the output Y of the high-dimensional transition layer 55 (8,512,32,32) uses transposed convolution to halve its number of channels and double its size to output the feature vector Z 01 (8,256,64,64) and the output Y after the multi-scale aggregation module in the third downsampling process 20 Concatenate the output feature vector Z in the channel dimension 02 (8,512,64,64), Z 02 The number of channels is halved through 3×3 convolution to output the feature vector Z 03 (8,256,64,64), Z 03 The feature vector Z is output without changing the feature size through the last set of 3×3 convolutions 04 (8,256,64,64);

[0131] The second upsampling will Z 04 Through transposed convolution, the number of channels is halved and the size is doubled to output the feature vector Z 11 (8,128,128,128) and the output Y after the multi-scale aggregation module in the second downsampling process 10 Concatenate the output feature vector Z in the channel dimension 12 (8,256,128,128), Z 12 The number of channels is halved through 3×3 convolution to output the feature vector Z 13 (8,128,128,128), Z 13 The feature vector Z is output without changing the feature size through the last set of 3×3 convolutions 14 (8,128,128,128);

[0132] The third upsampling will Z 14 Through transposed convolution, the number of channels is halved and the size is doubled to output the feature vector Z 21 (8,64,256,256) and the output Y after the multi-scale aggregation module in the second downsampling process 00Concatenate the output feature vector Z in the channel dimension 22 (8,128,256,256), Z 22 The number of channels is halved through 3×3 convolution to output the feature vector Z 23 (8,64,256,256), Z 23 The feature vector Z is output without changing the feature size through the last set of 3×3 convolutions 24 (8,64,256,256);

[0133] Z 24 Finally, a 1×1 convolution is used to adjust the channels of the final feature layer to the number of categories and then output the feature tensor Z(8,2,256,256). The feature tensor Z(8,2,256,256) is converted into an image representation of 8 batches of images with a size of 256×256 and a channel number of 2, which is used as the final urban road remote sensing image extraction result.

[0134] The fourth step is to obtain the remote sensing image of the urban road to be segmented: obtain the remote sensing image of the urban road to be segmented and perform preprocessing.

[0135] The fifth step is to extract the road information from the remote sensing image: input the pre-processed urban road remote sensing image to be segmented into the trained road remote sensing image extraction model to obtain the segmentation and extraction results of the remote sensing image road information.

[0136] like Figure 6 As shown in the figure, three pictures from the Massachusetts road dataset are selected to compare the visualization effects of different network models and the proposed module ablation comparison, where GMA is the general abbreviation of multi-scale aggregation module, adaptive Sobel operator and graph convolution module. It can be seen that the network module proposed by the present invention has a better visualization effect in complex road areas. Figure 7 As shown, three pictures from DeepGlobe Roads Dataset are selected to compare the visualization effects of different network models and the proposed module ablation comparison. Similarly, the network module proposed in the present invention has a better visualization effect in complex road areas.

[0137] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for extracting urban road remote sensing images focusing on context information, characterized in that: The following steps are involved: 11) Acquisition and preprocessing of urban road remote sensing images: Acquisition and preprocessing of urban road remote sensing images; 12) Construction of road remote sensing image extraction model: Construct a road remote sensing image extraction model based on UNet as the framework and Resnet-50 as the main body; 13) Training of road remote sensing image extraction model: inputting the pre-processed urban road remote sensing image into the road remote sensing image extraction model for training; 14) Acquisition of remote sensing images of urban roads to be segmented: acquiring remote sensing images of urban roads to be segmented and performing preprocessing; 15) Extraction of road information from remote sensing images: The pre-processed urban road remote sensing images to be segmented are input into the trained road remote sensing image extraction model to obtain the segmentation and extraction results of the remote sensing image road information.

2. The urban road remote sensing image extraction method focusing on context information according to claim 1 is characterized in that: The construction of the road remote sensing image extraction model includes the following steps: 21) Setting the road remote sensing image extraction model includes a downsampling layer, a high-dimensional transition layer, and an upsampling layer; 22) Set the downsampling layer: The initial input RGB image is reduced to a feature map of 1 / 4 of the original through three downsamplings. After each downsampling, the feature size is halved and the number of channels is doubled; The first two downsampling structures are the same. Each downsampling first passes through three groups of 3×3 convolutions. After each group of 3×3 convolutions, Batch Normalization and Relu activation function are used to prevent gradient explosion. Multi-scale information is then aggregated through a multi-scale aggregation module. Finally, an adaptive Sobel operator module is used to prevent overfitting of target and environment information. The third downsampling first passes through the Maxpool layer to halve the feature size, then aggregates global information through the graph convolution module, then passes through the first layer of ResNet-50, and finally passes through the multi-scale aggregation module and the adaptive Sobel operator module before output; 23) Set the high-dimensional transition layer: The high-dimensional transition layer is composed of the second, third, and fourth layers of ResNet-50, followed by a 3×3 convolution to change the channel dimension; 24) Set the upsampling layer: The upsampling layer restores the feature map output by the high-dimensional transition layer to the original image resolution through three upsamplings. Each time the upsampled feature passes through a transposed convolution and two 3×3 convolutions, the feature size is doubled and the number of channels is halved. Each 3×3 convolution must be followed by BN and ReLU activation functions in turn to prevent gradient explosion during training. The first, second, and third upsampling correspond to the third, second, and first downsampling respectively. The output of the multi-scale aggregation module in each downsampling is added to the output of the transposed convolution in the corresponding upsampling process using a skip link. Finally, the image pixels are divided into two categories through 1×1 convolution layer dimension classification, one is the background and the other is the extracted road. 25) Set ResNet-50: Set ResNet-50 to include four layers, each layer is composed of repeated residual blocks. The first layer of ResNet-50 is composed of three residual blocks stacked, the second layer is composed of four residual blocks stacked, the third layer is composed of six residual blocks stacked, and the fourth layer is composed of three residual blocks stacked; Each residual block is composed of 1×1 convolution, 3×3 convolution and 1×1 convolution stacked in sequence, and each convolution is followed by BN normalization and Relu activation function to prevent gradient explosion. Each residual block introduces a skip connection, that is, when the feature size and dimension of the input and output of the residual block do not change, the input and output of the current residual block are added as the input of the next residual block; 26) Set up the multi-scale aggregation module: The multi-scale aggregation module in the first and second downsampling takes the output feature tensors of the three groups of 3×3 convolutions in the downsampling as its input, and the multi-scale aggregation module in the third downsampling takes the output feature tensors of the first layer of ResNet-50 in the downsampling as its input, obtaining multi-scale information from the multi-scale image and the multi-scale convolution kernel; The input feature tensor X passes through two 1×1 convolution kernels in parallel, outputting two tensors respectively, and the channel dimension is reduced to 1 / 2 of X; A tensor passes through the multi-scale part of the convolution kernel and passes through four convolution blocks: one 1×1 convolution, three 3×3 convolutions with expansion rates of 1, 2, and 3 respectively, outputting four tensors, and the channel dimension is reduced to 1 / 8 of X; Another tensor passes through the multi-scale part of the image: the feature map is extracted at intervals of one point in both length and width in order through the Patch Meaging operation, and is divided into four tensors with half the length and width. The length and width of the four tensors are changed to 1 / 2, 1 / 4, 1 / 6, and 1 / 8 of the starting X through the Maxpool operation, and each is passed through a standard convolution kernel with a convolution kernel size of 3×3, and then restored to the size before the Maxpool operation through Bilinear. Finally, according to the reverse operation of Patch Meaging, the four tensors are merged into one tensor output in the original order. Finally, the outputs of the multi-scale part of the convolution kernel and the multi-scale part of the image are spliced ​​together through the channel dimension, added to the input jump connection, and then passed through a 1×1 convolution kernel to output a feature tensor of the same size and dimension as the starting input tensor X. After each convolution operation, a BN layer and Relu activation operation are performed to prevent gradient explosion; 27) Set the adaptive Sobel operator module: The output feature tensor Y of the multi-scale aggregation module is used as the input of the adaptive Sobel operator module. Y passes through 5×5 convolution and 7×7 convolution in turn. The outputs of the two are respectively passed through 1×1 convolution to halve the number of channels. Then, they are respectively passed through the Avgpool layer and the Maxpool layer and input into the Sigmoid activation layer. After that, the results are multiplied with the output results of the 1×1 convolution layer and merged in the channel dimension to output a feature tensor of the same size as Y. After that, the feature tensor is divided into four feature parts through the Patch Meaging operation, and is respectively convolved with the Sobel operator and then reversed through the Patch Meaging operation to merge the four features. Then, after passing through the 1×1 convolution, the four features are added to the input feature tensor Y and then output. After each convolution operation, the BN layer and Relu activation operation are performed to prevent gradient explosion. 28) Set up the graph convolution module: The output feature tensor Z after the Maxpool operation in the third downsampling is used as the input of the graph convolution module. The feature tensor Z first passes through a 1×1 convolution, and then is divided into four feature tensors with a length and width of 1 / 2 of the feature tensor Z through a Patch Meaging operation. The four feature vectors pass through two layers of graph convolution respectively. The graph convolution calculation formula is as follows: H (L+1) =σ(D -1 / 2 A1D -1 / 2 H (L) W (L) ) (1) Among them, H (L) Represents the feature vector (b, c, w0, h0) input of the current layer of graph convolution module, H (L+1) Represents the feature vector (b, c, w0, h0) output of the current layer of graph convolution module; A1 in formula (1) is the final adjacency matrix calculated by formula (2). The relationship between each element is the edge to construct the initial adjacency matrix A0. A0 is obtained by converting the feature tensor H (L) (b,c,w0,h0) is transformed into feature tensors of size (b,c,w0×h0) and (b,w0×h0,c) through reshape operation. The two are multiplied to obtain a tensor of size (b,w0×h0,w0×h0), i.e. A0, where I is the unit matrix of the same size as A0, multiplied by A0 and then added to A0, and the output tensor A1 (b,w0×h0,w0×h0); A1= A0× I+A0 (2) (1) D -1 / 2 represents a mathematical operation of performing a half-order inverse matrix on the D matrix, where D is the degree matrix of A1, and it is assumed to be a diagonal matrix obtained by summing each row of A1, D ii represents the i-th row and i-th column of the D matrix, which is the sum of all elements of the i-th row of A1, and finally outputs the tensor D -1 / 2 (b,w0×h0,w0×h0), the formula is as follows (3): D ii =∑ j (A1) ij (3) (1)W in the formula (L) It is the weight parameter after each layer of graph convolution propagation. In the network model, it is replaced by a 1×1 convolution; In (1), σ represents the activation function, which is replaced by the ReLU activation function in the model; After repeating the graph convolution twice, the input of the second graph convolution is the output of the first graph convolution, and four feature vectors H are output. (L+1) (b,c,w0,h0),H (L+1) It passes through 4 convolution blocks respectively: one 1×1 convolution, three 3×3 convolutions with expansion rates of 1, 2, and 3 respectively. After each convolution operation, a BN layer and Relu activation operation are performed to prevent gradient explosion. The four feature vectors obtained are then inversely operated by Patch Meaging and then 1×1 convolution and added to the original feature vector Z of the graph convolution module. Finally, a feature vector of the same size as the feature tensor Z is output.

3. The urban road remote sensing image extraction method focusing on context information according to claim 1 is characterized in that: The training of the road remote sensing image extraction model includes the following steps: 31) Input the preprocessed remote sensing image data into the road remote sensing image extraction model, use the binary cross entropy function as the loss function, the initial learning rate is 0.0001, the number of training iterations is 50, 50, 100, and the step size is 8; 32) Start by inputting an 8-batch RGB image with a size of 256×256 and 3 channels, and convert it into a feature tensor X 00 The size is (8,3,256,256) and enters the downsampling layer of the road remote sensing image extraction model: First downsampling, feature tensor X 00 (8,3,256,256) After the first set of 3×3 convolutions, the feature tensor size becomes X 01 (8,64,256,256), X 01 After two sets of 3×3 convolutions, the feature tensor size is not changed and the output feature tensor X is obtained. 02 (8,64,256,256), X 02 Output feature tensor Y through the first multi-scale aggregation module 00 (8,64,256,256) ， Y 00 After passing the first adaptive Sobel operator, the output feature tensor Y 01 (8,64,256,256); Second downsampling, feature tensor Y 01 (8, 64, 256, 256) After the first set of 3×3 convolutions, the feature tensor size becomes X 11 (8,64,128,128), X 11 After the second set of 3×3 convolutions, the feature tensor size is not changed and the output feature tensor X is obtained. 12 (8,64,128,128), X 12 After the third set of 3×3 convolutions, the output feature tensor X 13 (8,128,128,128), X 13 Output feature tensor Y through the second multi-scale aggregation module 10 (8,128,128,128), Y 10 After passing the second adaptive Sobel operator, the output feature tensor Y 11 (8,128,128,128); The third downsampling, feature tensor Y 11 (8,128,128,128) First use Maxpool to halve the feature size and keep the number of channels unchanged to output the feature tensor X 20 (8,128,64,64), X 20 Output feature tensor X without changing feature size through graph convolution module 21 (8,128,64,64), X 21 The number of feature map channels is halved through the 1×1 convolution of the first residual block of the first layer of ResNet-50, and the output feature tensor X remains unchanged in size. 22 (8,64,64,64), X 22 The number of feature map channels is quadrupled through the 3×3 convolution of the first residual block of the first layer of ResNet-50, and the output feature tensor X remains unchanged in size. 23 (8,256,64,64), X 23 After passing the last 1×1 convolution of the first residual block of the first layer of ResNet-50, the output feature tensor X 24 (8,256,64,64), X 24 After passing through the second and third residual blocks of the first layer of ResNet-50, the feature tensor X is output. 25 (8,256,64,64), X 25 After passing through the third multi-scale aggregation module, the output feature tensor Y 20 (8,256,64,64), Y 20 After passing the third adaptive Sobel operator, the feature size is not changed and the output feature tensor Y 21 (8,256,64,64); 33) The downsampled output enters the high-dimensional transition layer: The feature tensor Y output by the downsampling layer 21 (8,256,64,64), the number of feature map channels is halved through the 1×1 convolution of the first residual block of the second layer of ResNet-50, and the size of the output feature tensor Y remains unchanged 31 (8,128,64,64), Y 31 The feature map size is halved through the 3×3 convolution of the first residual block in the second layer, and the number of channels is quadrupled to output the feature tensor Y 32 (8,512,32,32), Y 32 Output Y through the last 1×1 convolution of the first residual block in the second layer 33 (8,512,32,32), Y 33 After passing through the second, third, and fourth residual blocks of the second layer of ResNet-50 in sequence, the feature vector size is not changed and the feature tensor Y is output 34 (8,512,32,32); Y 34 The number of feature map channels is halved through the 1×1 convolution of the first residual block in the third layer of ResNet-50, and the output Y is unchanged in size. 41 (8,256,32,32), Y 41 The feature map size is halved through the 3×3 convolution of the first residual block in the third layer, and the number of channels is quadrupled to output Y 42 (8,1024,32,32), Y 42 Output Y through the last 1×1 convolution of the first residual block in the third layer 43 (8,1024,32,32), Y 43 After passing through the five residual blocks after the third layer of ResNet-50, the feature vector size is not changed and the feature tensor Y is output. 44 (8,1024,32,32); Y 44 The number of feature map channels is halved through the 1×1 convolution of the first residual block in the fourth layer of ResNet-50, and the output Y is unchanged in size. 51 (8,512,32,32), Y 51 The feature map size is halved through the 3×3 convolution of the first residual block in the fourth layer, and the number of channels is quadrupled to output Y 52 (8,2048,32,32), Y 52 Output Y through the last 1×1 convolution of the first residual block in the fourth layer 53 (8,2048,32,32), Y 53 After passing through the two residual blocks after the fourth layer of ResNet-50, the feature vector size is not changed and the feature tensor Y is output. 54 (8,2048,32,32); Y 54 Through a 3×3 convolution, the number of channels is reduced to 1 / 4 and the size remains unchanged to output the feature tensor Y 55 (8,512,32,32); 34) The outputs of the high-dimensional transition layer and the downsampling layer are input into the upsampling layer, and the upsampling is performed three times: The first upsampling converts the output Y of the high-dimensional transition layer 55 (8,512,32,32) uses transposed convolution to halve its number of channels and double its size to output the feature vector Z 01 (8,256,64,64) and the output Y after the multi-scale aggregation module in the third downsampling process 20 Concatenate the output feature vector Z in the channel dimension 02 (8,512,64,64), Z 02 The number of channels is halved through 3×3 convolution to output the feature vector Z 03 (8,256,64,64), Z 03 The feature vector Z is output without changing the feature size through the last set of 3×3 convolutions 04 (8,256,64,64); The second upsampling will Z 04 Through transposed convolution, the number of channels is halved and the size is doubled to output the feature vector Z 11 (8,128,128,128) and the output Y after the multi-scale aggregation module in the second downsampling process 10 Concatenate the output feature vector Z in the channel dimension 12 (8,256,128,128), Z 12 The number of channels is halved through 3×3 convolution to output the feature vector Z 13 (8,128,128,128), Z 13 The feature vector Z is output without changing the feature size through the last set of 3×3 convolutions 14 (8,128,128,128); The third upsampling will Z 14 Through transposed convolution, the number of channels is halved and the size is doubled to output the feature vector Z 21 (8,64,256,256) and the output Y after the multi-scale aggregation module in the second downsampling process 00 Concatenate the output feature vector Z in the channel dimension 22 (8,128,256,256), Z 22 The number of channels is halved through 3×3 convolution to output the feature vector Z 23 (8,64,256,256), Z 23 The feature vector Z is output without changing the feature size through the last set of 3×3 convolutions 24 (8,64,256,256); Z 24 Finally, a 1×1 convolution is used to adjust the channels of the final feature layer to the number of categories and then output the feature tensor Z(8,2,256,256). The feature tensor Z(8,2,256,256) is converted into an image representation of 8 batches of images with a size of 256×256 and a channel number of 2, which is used as the final urban road remote sensing image extraction result.

Images