A road recognition method and system based on remote sensing images

Abstract

The application discloses a road recognition method and system based on remote sensing images, wherein the method comprises the following steps: S1, constructing a road recognition model; S2, inputting a training set remote sensing image into the road recognition model to obtain a road recognition training prediction image; S3, calculating a loss value according to a training set label image and the training prediction image, and updating model parameters; S4, repeating the steps S2 and S3 to obtain a trained road recognition model; S5, inputting a test set remote sensing image into the trained road recognition model to obtain a road recognition test prediction image; and S6, comparing the test prediction image with a test set label image to obtain various evaluation indexes, and evaluating the trained road recognition model according to the evaluation index values. The application uses a multi-scale feature extraction module to enhance the U-Net feature extraction capability, and adds an attention mechanism at a U-Net model jump connection to enable the model to fully utilize effective feature information.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of semantic segmentation, in particular to a road recognition method and system based on remote sensing images. BACKGROUND

[0002] Deep learning has developed rapidly in recent years. Compared with traditional semantic segmentation methods, deep learning-based semantic segmentation methods can learn useful feature information in massive data and have higher segmentation accuracy. The proposal of fully convolutional neural network (FCN) has a milestone significance in the research process of semantic segmentation, but the continuous downsampling will lose feature information. The fully convolutional neural network up-samples the output feature map of the last convolutional layer in an inverse convolution manner to make its size the same as the input image, and the higher sampling rate will also lead to the loss of spatial information, which will greatly affect the segmentation accuracy. DeepLab v3 network improves the spatial pyramid pooling structure with a hollow, uses different sizes of hollow convolution kernels to build a spatial hollow pyramid module to obtain multi-scale feature information, and adds a batch normalization layer (BN) to avoid network overfitting, but using DeepLab v3 network for semantic segmentation research will cause discontinuous segmentation and rough segmentation boundary. SegNet is a symmetrical network structure, and its idea is also based on the fully convolutional neural network, but it has problems such as large number of parameters and slow speed. The U-Net network splices the image feature information extracted in the encoder stage with the feature information in the decoder stage in the channel dimension, i.e. the skip connection operation, on the basis of the SegNet network, which makes up for the loss of part of the feature information in the downsampling process and improves the segmentation accuracy of the model, but the effect is still limited.

[0003] In view of the adverse effects of the loss of feature information in the downsampling process of the U-Net model and the failure of the model to effectively use useful feature information on the prediction results, a road segmentation method based on multi-scale feature extraction and attention mechanism in remote sensing images is proposed. The method uses a multi-scale feature extraction module to enhance the feature extraction capability of U-Net, and adds an attention mechanism at the skip connection of the U-Net model to enable the model to fully utilize effective feature information. Experimental results show that the method is superior to common semantic segmentation methods. SUMMARY

[0004] The present application discloses a road recognition method and system based on remote sensing images, which uses a multi-scale feature extraction module to enhance the feature extraction capability of U-Net, and adds an attention mechanism at the skip connection of the U-Net model to enable the model to fully utilize effective feature information.

[0005] To achieve the above purpose, the present application provides the following solutions:

[0006] A road recognition method based on remote sensing images, the method comprising:

[0007] S1, constructing a road recognition model;

[0008] S2, inputting a training set remote sensing image into the road recognition model to obtain a road recognition training prediction map;

[0009] S3, calculating a loss value according to a training set label image and the training prediction map, and updating a model parameter;

[0010] S4, repeating the S2 and the S3 to obtain a trained road recognition model;

[0011] S5, inputting a test set remote sensing image into the trained road recognition model to obtain a road recognition test prediction map.

[0012] Optionally, the road recognition method based on remote sensing images further comprises S6:

[0013] comparing the test prediction map with a test set label image to obtain evaluation indexes, and evaluating the trained road recognition model according to the evaluation index values.

[0014] Optionally, the method for obtaining the road recognition training prediction map comprises:

[0015] convolving the training set remote sensing image, performing multi-scale feature extraction and downsampling to obtain a feature map;

[0016] upsampling the feature map to obtain a size-transformed feature map;

[0017] splicing the size-transformed feature map and a feature map obtained according to an attention mechanism to obtain a spliced feature map;

[0018] convolving the spliced feature map to obtain the road recognition training prediction map.

[0019] Optionally, the method for obtaining the feature map comprises:

[0020] convolving an input remote sensing image to obtain a feature map with a preset number of channels;

[0021] performing multi-scale feature extraction on the feature map with the preset number of channels to obtain a feature map after feature extraction;

[0022] performing maximum pooling on the feature map after feature extraction to obtain a feature map with a preset size;

[0023] The convolution operation, the multi-scale feature extraction operation and the maximum pooling operation are iterated on the feature map after the feature extraction until the feature map reaches a preset length and width and a preset number of channels.

[0024] Optionally, the multi-scale feature extraction specifically includes:

[0025] The feature map after the feature extraction is subjected to pyramid convolution to obtain a multi-scale feature extraction feature map consistent with the length, width and number of channels of the feature map after the feature extraction.

[0026] Optionally, the method for obtaining the feature map after the size transformation includes:

[0027] The feature map reaching the preset length, width and preset number of channels is subjected to an upsampling operation to obtain an upsampled feature map of a preset size.

[0028] Optionally, the method for obtaining the road recognition training prediction map includes:

[0029] The upsampled feature map of the preset size and the feature map obtained by the attention mechanism are spliced to obtain a spliced feature map;

[0030] The spliced feature map is subjected to two 3*3 convolutions to obtain a feature map of a preset number of channels;

[0031] The upsampled feature map of the preset size, the feature map obtained by the attention mechanism and the two 3*3 convolutions are iterated to obtain a feature map of a preset size;

[0032] The feature map of the preset size is subjected to convolution to obtain the road recognition training prediction map.

[0033] Optionally, the attention mechanism specifically includes:

[0034]

[0035] wherein, x c is an input feature map, is an attention weight, is an attention weight.

[0036] Optionally, the method for calculating the loss value includes:

[0037] The CrossEntropyLoss and DiceLoss two loss functions are added to calculate the loss value of the road recognition training label image and the road recognition training prediction map;

[0038] The CrossEntropyLoss loss function is:

[0039]

[0040] wherein p i is a training prediction map output by the model, g i is a training label image, and N is the number of samples.

[0041] The DiceLoss loss function is:

[0042]

[0043] wherein p i is a training prediction map output by the model, g i is a training label image, and N is the number of samples.

[0044] A road recognition system based on remote sensing images, comprising a model construction module, a training prediction map acquisition module, a parameter updating module, a model optimization module, a test prediction map acquisition module and a network model evaluation module.

[0045] The model construction module is configured to construct a road recognition model.

[0046] The training prediction map acquisition module is configured to input a training set remote sensing image into the road recognition model to obtain a road recognition training prediction map.

[0047] The parameter updating module is configured to calculate a loss value according to a training set label image and the training prediction map, and update model parameters.

[0048] The model optimization module is configured to repeatedly execute the training prediction map acquisition module and the parameter updating module to obtain a trained road recognition model.

[0049] The test prediction map acquisition module is configured to input a test set remote sensing image into the trained road recognition model to obtain a road recognition test prediction map.

[0050] The network model evaluation module is configured to compare the test prediction map with a test set label image to obtain various evaluation indexes, and evaluate the trained road recognition model according to the evaluation index values.

[0051] The application has the following beneficial effects:

[0052] A road recognition method and system based on remote sensing images, the application uses a multi-scale feature extraction module to enhance the feature extraction capability of U-Net, and adds an attention mechanism at the jump connection of the U-Net model to enable the model to make full use of effective feature information. Experimental results show that the method is better than common semantic segmentation methods. Road segmentation aims to identify the road part at the pixel level from the remote sensing image. Such segmentation results are of great significance and can be used as one of the judgment bases for disaster evacuation and transfer of personnel such as earthquakes. By segmenting the part of the image containing the road, the road condition can be accurately judged, so as to quickly transfer and evacuate the crowd from the road with good road conditions, thereby reducing the loss. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the present application, the following briefly introduces the drawings needed in the embodiments. Obviously, the drawings described in the following embodiments are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0054] Figure 1 The method steps of the road recognition method and system based on remote sensing images of the embodiment of the present application are shown in the figure.

[0055] Figure 2 The main structure diagram of the road recognition method and system based on remote sensing images of the embodiment of the present application is shown in the figure.

[0056] Figure 3 The multi-scale feature extraction module structure diagram of the road recognition method and system based on remote sensing images of the embodiment of the present application is shown in the figure.

[0057] Figure 4 The attention mechanism structure diagram of the road recognition method and system based on remote sensing images of the embodiment of the present application is shown in the figure.

[0058] Figure 5 The road segmentation effect comparison diagram of the road recognition method and system based on remote sensing images of the embodiment of the present application is shown in the figure.

[0059] Figure 6 The segmentation detail comparison diagram of the road recognition method and system based on remote sensing images of the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0060] The technical solutions of the embodiments of the present application will be described in detail below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0061] In order to make the above objects, features and advantages of the present application more apparent, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0062] In the embodiment, a road recognition method and system based on remote sensing images, as shown in the drawings, specifically comprises: Figures 1-6

[0063] A road recognition method based on remote sensing images, the method comprising:

[0064] S1, constructing a road recognition model;

[0065] In the embodiment, a remote sensing original image with a size of 3x256x256 is input into the model, wherein C=3 is the number of image channels, i.e. R, G, B channels, H=256 is the length of the image, and W=256 is the width of the image;

[0066] S2, inputting the training set remote sensing image into the road recognition model to obtain a road recognition training prediction map;

[0067] S3, calculating a loss value according to the training set label image and the training prediction map, and updating the model parameters;

[0068] S4, repeating the S2 and the S3 to obtain a trained road recognition model;

[0069] The method for obtaining the road recognition training prediction map comprises:

[0070] convolving the training set remote sensing image, performing multi-scale feature extraction and downsampling to obtain a feature map;

[0071] upsampling the feature map to obtain a feature map after size transformation;

[0072] splicing the feature map after size transformation and a feature map obtained according to an attention mechanism to obtain a spliced feature map;

[0073] convolving the spliced feature map to obtain the road recognition training prediction map.

[0074] The method for obtaining the feature map comprises:

[0075] convolving the input remote sensing image to obtain a feature map with a preset number of channels;

[0076] performing multi-scale feature extraction on the feature map with the preset number of channels to obtain a feature map after feature extraction;

[0077] performing maximum pooling on the feature map after feature extraction to obtain a feature map with a preset size;​

[0078] The convolution operation, the multi-scale feature extraction operation and the maximum pooling operation are iterated on the extracted feature map until the feature map reaches a preset length and width and a preset number of channels.

[0079] In the embodiment, the input image size is 3x256x256, and the image of the size is subjected to 3x3 convolution to obtain an image of the size of 64x256x256; after the initial convolution, the length and width of the image remain unchanged, but the number of channels changes from 3 to 64.

[0080] The feature map of the size of 64x256x256 is subjected to multi-scale feature extraction; in the process of model downsampling, the feature information is lost and the model cannot effectively use useful information to affect the prediction result, and the multi-scale feature extraction can enhance the feature extraction capability; the U-Net downsampling module is used for feature extraction of the target, the feature extraction capability of the SPC enhanced model is used in the EPSANet, and based on the U-Net downsampling module and the SPC, the second 3x3 convolution layer in the U-Net downsampling module is replaced by a multi-scale feature extraction module.

[0081] The process of the multi-scale feature extraction module includes: setting the size of the feature map X input into the multi-scale feature extraction module as CxHxW, wherein C is the number of channels of the feature map, H is the length of the feature map, and W is the width of the feature map; the feature map X obtained after 3x3 convolution is first divided into S groups of sub-feature maps [F0, F1,..., FS-1] by using pyramid convolution, and the number of channels of each group of sub-feature maps is compressed to C / S, wherein C is an integer multiple of S, and each group of sub-feature maps is obtained as F S-1 ]。 i (C / S)×H×W , wherein i=0, 1,..., S-1; the process is represented by equation (1):

[0082] i i i i i=0, 1,..., S-1 (1)

[0083] i is the convolution kernel size; G is a grouped convolution, which can reduce the parameter amount and effectively reduce the parameter amount of the model to improve the efficiency. After the above steps, S groups of sub-feature maps with the number of channels C / S, the length H and the width W are obtained. Then, the S groups of sub-feature maps are spliced in the channel dimension to obtain an output feature map, and the process is represented by equation (2):

[0084] ​​​​​​F = Cat([F0, F1,..., F S-1 ]) (2)

[0085] The result is then input into a pooling layer, and after the pooling layer, the output feature map has a length and width that is half of the input. In this embodiment, a multi-scale feature extraction module is used to extract features from an input of 64x256x256 to obtain a feature map of 64x256x256.

[0086] The obtained feature map of 64x256x256 is subjected to maximum pooling to obtain a feature map of 64x128x128. The feature map of 64x128x128 is subjected to 3x3 convolution, multi-scale feature extraction, and maximum pooling again to obtain a feature map of 128x64x64. The feature map of 128x64x64 is subjected to 3x3 convolution, multi-scale feature extraction, and maximum pooling to obtain a feature map of 256x32x32. The feature map of 256x32x32 is subjected to 3x3 convolution, multi-scale feature extraction, and maximum pooling to obtain a feature map of 512x16x16. The feature map of 512x16x16 is subjected to 3x3 convolution and multi-scale feature extraction to obtain a feature map of 512x16x16.

[0087] The size-transformed feature map and the feature map obtained according to the attention mechanism are spliced to obtain a spliced feature map, and the road recognition training prediction map is obtained according to the spliced feature map. The feature map of the up-sampling preset size and the feature map obtained according to the attention mechanism are spliced to obtain a spliced feature map; it is worth mentioning that the sizes of the two spliced feature maps need to be consistent. The spliced feature map is subjected to two 3x3 convolutions to obtain an image of a preset number of channels; the feature map of the preset size is obtained by iterating the up-sampling feature map of the preset size, the feature map obtained according to the attention mechanism, and the two 3x3 convolutions; and the road recognition training prediction map is obtained by performing 1x1 convolution on the feature map of the preset size.

[0088] In this embodiment, the attention mechanism includes an attention mechanism structure used in the application as shown in FIG. 3, which is improved based on the attention mechanism CA (Coordinate Attention). Figure 4

[0089] For input F∈ C×H×W , a pooling kernel with sizes (H, 1) and (1, W) is used to encode each channel along the horizontal coordinate direction and the vertical coordinate direction, and through the above transformation, the input features in the horizontal and vertical directions can be aggregated to obtain a pair of direction perception feature maps. Then, for input F∈ 512×32×32 ​The obtained direction-aware feature map sizes are 512x32x1 and 512x1x32 respectively. Each feature map contains the long-distance dependency relationship of the input feature map along one spatial direction and saves the precise position information along the other spatial direction, which helps the network to locate the target of interest more accurately.

[0090] After the transformation in the information embedding, the generated output feature map is spliced and transformed through a shared 1x1 convolutional transformation function F1: as shown in equation (3):

[0091] f = δ (F1 ([z h ,z w ])) (3)

[0092] wherein: f∈ (512 / r)×(32+32) is the intermediate feature map of the spatial information in the horizontal direction and the vertical direction, and r represents the down-sampling ratio. Reducing the channel number of f using a suitable reduction ratio r can reduce the complexity and computational cost of the model, and z h and z w are direction-aware feature maps, and [·,·] is the splicing operation along the spatial dimension; and δ is a nonlinear activation function.

[0093] After normalization and nonlinear processing, f is decomposed into two independent tensors f h ∈ (512 / r)×32 and f w ∈ (512 / r)×32 along the spatial dimension. Two 1x1 convolutional transformations F h and F w are used to transform the feature maps f h and f w into tensors with the same number of channels as the input feature layer F∈ 512×32×32 . This process can be represented as equation (4) and equation (5):

[0094] g h = σ (F h (f h )) (4)

[0095] g w = σ (F w (f w )) (5)

[0096] In order to improve the accuracy of the results, two SPC modules with S = 4 are used in the present application to replace the two 1x1 convolutional transformations in the above steps, and a BN layer is added. The modified attention mechanism structure diagram is as shown in Figure 4 .

[0097] At this time, the results of equation (4) and equation (5) are converted to equation (6) and equation (7):

[0098] s h = σ(S h (f h )) (6)

[0099] s w = σ(S w (f w )) (7)

[0100] wherein σ denotes a sigmoid activation function, which can reduce the complexity of the model and reduce the computational overhead. S h (·) and S w (·) respectively denote the outputs of the feature maps f h and f w after passing through the SPC module, and are specifically shown in equations (8) and (9):

[0101]

[0102]

[0103] wherein:

[0104] F i h = Conv(n i × n i , G i )(f h ) i = 0,..., 3 (10)

[0105] F i w = Conv(n i × n i , G i )(f w ) i = 0,..., 3 (11)

[0106] Finally, the obtained results s h and s w are expanded, and the final attention weight matrix is obtained using the matrix multiplication method, and the final output of the attention mechanism is shown in equation (12):

[0107]

[0108] At this time, the attention output is Y ∈ 512×32×32 .

[0109] According to the size-transformed feature map and the feature map obtained according to the attention mechanism, a spliced feature map is obtained by splicing, and a method for obtaining the road recognition training prediction map according to the spliced feature map comprises:

[0110] The up-sampled feature map of the preset size is spliced with the feature map obtained by the attention mechanism to obtain a spliced feature map;

[0111] The spliced feature map is convolved twice to obtain a feature map of a preset number of channels;

[0112] The up-sampled feature map of the preset size, the feature map obtained by the attention mechanism, and the twice 3x3 convolution are iteratively used to obtain a feature map of a preset size.

[0113] The feature map of the preset size is convolved to obtain the road recognition training prediction map.

[0114] The up-sampling includes: performing an up-sampling operation on the feature map reaching the preset length and width and the preset number of channels.

[0115] The feature map obtained by the up-sampling is spliced with an attention mechanism feature map of the same size to obtain a feature map of a preset size.

[0116] The feature map of the preset size is convolved twice to obtain a feature map of a preset number of channels;

[0117] The up-sampling, splicing, and convolution operations are repeated until the length, width, and number of channels of the image reach the final preset value, and in this embodiment, the final preset value is 64x256x256.

[0118] In the embodiment, the 512x16x16 feature map obtained by multi-scale feature extraction is transformed in size by upsampling, and a 512x32x32 feature map is obtained after upsampling; the feature map obtained by the attention mechanism from the multi-scale feature extraction channel and the 512x32x32 feature map are spliced to obtain a 1024x32x32 feature map; the obtained 1024x32x32 feature map is convolved twice with 3x3, the first convolution obtains a 512x32x32 feature map, and the second convolution obtains a 256x32x32 feature map, and the channel number is transformed; the obtained 256x32x32 feature map is transformed in length and width by upsampling, and a 256x64x64 feature map is obtained after upsampling; the feature map obtained by the attention mechanism from the multi-scale feature extraction length and channel number 256x64x64 feature map, the two feature maps are spliced to obtain a 512x64x64 feature map in length and channel number; the obtained 512x64x64 feature map is convolved twice with 3x3, the first convolution obtains a 256x64x64 feature map, and the second convolution obtains a 128x64x64 feature map, and the channel number is changed by twice convolution; the obtained 128x64x64 feature map is transformed in size by upsampling, and a 128x128x128 feature map is obtained after upsampling; the feature map obtained by the attention mechanism from the multi-scale feature extraction size and channel number 128x128x128 feature map, the two feature maps are spliced to obtain a 256x128x128 feature map in length and channel number; the obtained 256x128x128 feature map is convolved twice with 3x3, the first convolution obtains a 128x128x128 feature map, and the second convolution obtains a 64x128x128 feature map, and the channel number is changed by twice convolution; the obtained 64x128x128 feature map is transformed in size by upsampling, and a 64x256x256 feature map is obtained after upsampling; the feature map obtained by the attention mechanism from the multi-scale feature extraction length and channel number 64x256x256 feature map, the two feature maps are spliced to obtain a 128x256x256 feature map in length and channel number; the obtained 128x256x256 feature map is convolved twice with 3x3, the first convolution obtains a 64x256x256 feature map, and the second convolution obtains a 64x256x256 feature map, and the channel number is changed by twice convolution; the obtained 64x256x256 feature map is convolved with 1x1 to obtain a 2x256x256 feature map, which is a road recognition training prediction map.

[0119] The CrossEntropyLoss and the DiceLoss are added to calculate the loss value of the training set label image and the training prediction image.

[0120] L = L CEL + L DL (13)

[0121] The CrossEntropyLoss loss function comprises:

[0122]

[0123] Wherein, p i is the training prediction image output by the model, g i is the training label image, and N is the sample number.

[0124] The DiceLoss loss function comprises:

[0125]

[0126] Wherein, p i is the training prediction image output by the model, g i is the training label image, and N is the sample number.

[0127] The loss function is used to measure the difference between the prediction image output by the model and the real label image, and an optimizer is needed to update and adjust the model parameters so that the loss value gradually decreases.

[0128] S5, inputting the test set remote sensing image into the trained road recognition model to obtain a road recognition test prediction image.

[0129] S6, comparing the test prediction image with the test set label image to obtain each evaluation index, and evaluating the trained road recognition model according to the evaluation index value.

[0130] 230 test set images (including 230 corresponding label images) are put into the trained road recognition model to obtain a road recognition test prediction image.

[0131] In this embodiment, for the road segmentation data set, we use the pixel accuracy (PA), class average pixel accuracy (mPA) and average intersection over union (mIoU) three indexes to evaluate the performance of the segmentation model. The three indexes are calculated as follows:

[0132]

[0133]

[0134]

[0135] k represents the number of prediction categories, p ij represents the number of points of category i predicted as category j, p ii and p jj , that is, the result of pixels of category i and category j being correctly judged. PA represents the proportion of the number of pixels correctly predicted to the total number of pixels; mPA represents the proportion of the number of pixels of each category correctly predicted to the total number of pixels, which is accumulated and then averaged; mIoU represents the ratio of the intersection to the union of the prediction result and the true result of each category, which is accumulated and then averaged.

[0136] The method of the present application is compared with other semantic segmentation methods. The model network structure is built based on the pytorch deep learning framework, and the Adam optimizer algorithm is used to optimize the weight update. The data set samples come from Massachusetts Roads Dataset, and after cropping, a new data set containing 5800 training sets, 230 validation sets and 230 test sets is obtained, and the length and width of each image are both 256x256.

[0137] 1. Comparative experiment:

[0138] The network framework proposed in the present application is compared with other commonly used semantic segmentation networks, as shown in Table 1. The method proposed in the present application is superior to the existing common methods in all indicators. In the indicator mIoU, it is improved by 1.8% compared with DeepLabv3, by 1.7% compared with FCN, by 0.6% compared with SegNet, and by 0.4% compared with U-Net; in the indicator PA, it is also superior to other methods, of which it is improved by 0.5% compared with FCN and DeepLabv3, and by 0.1% compared with SegNet and U-Net; in the indicator mPA, it is also superior to all methods in the table by about 0.3% to 0.7%. The result segmentation accuracy of the method proposed in the present application is higher compared with other methods. Compared with U-Net, although the improvement of the three evaluation indicators is not as obvious as other networks, as shown in Table 1, the method of the present application has better results in some segmentation details. The comparison of road segmentation indicators is shown in Table 1: Figure 6

[0139] Table 1

[0140]

[0141] 2. Ablation experiment

[0142] In order to verify the effectiveness of each module of the network proposed in the present application, ablation experiments are carried out on different modules in this section. In this section, U-Net is selected as the baseline network for ablation experiment.​

[0143] (1) Multi-scale feature extraction module

[0144] In the basic network U-Net, the multi-scale feature extraction module is used and compared with the basic network, as shown in Table 2, after adding the multi-scale feature extraction module, the PA of the network increases by 0.1% compared with U-Net, but the mIoU index increases by 0.3% compared with U-Net. This shows that adding a multi-scale feature extraction module (MFE) in the down-sampling module can better extract road features.

[0145] (2) Attention mechanism

[0146] On the basis of using the multi-scale feature extraction module in the down-sampling module of the basic network U-Net, the attention mechanism CA is added in the middle jump connection, as can be seen from Table 2, the index mPA increases by 0.3%, which shows that the attention mechanism is effective. On this basis, the changed attention mechanism proposed in the application is used, as can be seen from the table, compared with the unaltered attention mechanism CA, the method of the application improves the index mIoU by 0.1%, and the index mPA is improved by 0.5%. This shows that the changed attention mechanism has a relatively significant effect on improving the network segmentation performance. The performance comparison of adding different modules is shown in Table 2:

[0147] Table 2

[0148]

[0149] Example two:

[0150] A road recognition system based on remote sensing images, comprising: a model construction module, a training prediction image acquisition module, a parameter updating module, a model optimization module, a test prediction image acquisition module and a network model evaluation module;

[0151] The model construction module is used to construct a road recognition model;

[0152] The training prediction image acquisition module is used to input the training set remote sensing image into the road recognition model to obtain a road recognition training prediction image;

[0153] The parameter updating module is used to calculate the loss value according to the training set label image and the training prediction image, and update the model parameters;

[0154] The model optimization module is used to repeat the training prediction image acquisition module and the parameter updating module to obtain a trained road recognition model;

[0155] The test prediction image acquisition module is used to input the test set remote sensing image into the trained road recognition model to obtain a road recognition test prediction image;

[0156] The network model evaluation module is configured to compare the test prediction map with a test set label map to obtain each evaluation index, and evaluate the trained road recognition model according to the evaluation index value.

[0157] The above-described embodiments are merely intended to describe the preferred modes of the present application, and are not intended to limit the scope of the present application. Various modifications and improvements to the technical solutions of the present application made by those of ordinary skill in the art without departing from the design spirit of the present application shall fall within the scope of protection of the present application as defined by the claims.

Claims

1. A method for road recognition based on remote sensing images, characterized in that, The method comprises: S1, constructing a road recognition model; S2, inputting a training set remote sensing image into the road recognition model to obtain a road recognition training prediction map; S3, calculating a loss value according to a training set label image and the road recognition training prediction map, and updating a model parameter; S4, repeating the S2 and the S3 to obtain a trained road recognition model; S5, inputting a test set remote sensing image into the trained road recognition model to obtain a road recognition test prediction map; The method for obtaining the road recognition training prediction map comprises: performing convolution, multi-scale feature extraction and down-sampling on the training set remote sensing image to obtain a feature map; performing up-sampling on the feature map to obtain a feature map with a size changed; splicing the feature map with the size changed and a feature map obtained according to an attention mechanism to obtain a spliced feature map; performing convolution on the spliced feature map to obtain the road recognition training prediction map; The method for obtaining the feature map comprises: performing convolution on an input remote sensing image to obtain a feature map with a preset channel number; performing multi-scale feature extraction on the feature map with the preset channel number to obtain a feature map after feature extraction; performing maximum pooling on the feature map after feature extraction to obtain a feature map with a preset size; iterating the convolution operation, the multi-scale feature extraction operation and the maximum pooling operation on the feature map after feature extraction until the feature map reaches a preset length and width and a preset channel number; replacing a second 3*3 convolution layer in a U-Net down-sampling module with a multi-scale feature extraction module; The multi-scale feature extraction specifically comprises: performing pyramid convolution on the feature map with the preset channel number to obtain a multi-scale feature extraction feature map consistent with the length and width and the channel number of the feature map with the preset channel number; The method for obtaining the road recognition training prediction map according to the feature map with the size changed and the feature map obtained according to the attention mechanism comprises: splicing the up-sampled feature map with the preset size and the feature map obtained according to the attention mechanism to obtain a spliced feature map; performing 3*3 convolution twice on the spliced feature map to obtain a feature map with a preset channel number; iterating the up-sampled feature map with the preset size, the feature map obtained according to the attention mechanism and the 3*3 convolution twice to obtain a feature map with a preset size; performing convolution on the feature map with the preset size to obtain the road recognition training prediction map; The up-sampling comprises: performing an up-sampling operation on the feature map reaching the preset length and width and the preset channel number; splicing the feature map obtained by up-sampling and an attention mechanism feature map with the same size to obtain a feature map with a preset size; wherein, is an input feature map, is an attention weight, is an attention weight.

2. The method of claim 1, wherein, The attention mechanism specifically comprises: The road recognition method based on a remote sensing image further comprises S6: 3.The road recognition method based on a remote sensing image according to claim 1, characterized in that, comparing the test prediction map with a test set label image to obtain evaluation indexes, and evaluating the trained road recognition model according to the evaluation index values. The method for obtaining the feature map with the size changed comprises: The feature map reaching the preset length-width and the preset number of channels is subjected to an upsampling operation to obtain a feature map of an upsampling preset size, i.e., a size-transformed feature map.

4. The method of claim 1, wherein, The method for calculating the loss value comprises: The loss value of the road recognition training label image and the road recognition training prediction image is calculated by adding two loss functions of CrossEntropyLoss and DiceLoss; The CrossEntropyLoss loss function is: wherein, is a training prediction map output by the model, is a training label image, is a number of samples; The DiceLoss loss function is: wherein, is a training prediction map output by the model, is a training label image, is a number of samples.

5. A remote sensing image based road recognition system, the system being configured to implement the method of any one of claims 1 to 4, characterized in that, It comprises: a model construction module, a training prediction image acquisition module, a parameter updating module, a model optimization module, a test prediction image acquisition module and a network model evaluation module; The model construction module is used for constructing a road recognition model; The training prediction image acquisition module is used for inputting a training set remote sensing image into the road recognition model to obtain a road recognition training prediction image; The parameter updating module is used for calculating a loss value according to a training set label image and the road recognition training prediction image, and updating model parameters; The model optimization module is used for repeatedly performing the training prediction image acquisition module and the parameter updating module to obtain a trained road recognition model; The test prediction image acquisition module is used for inputting a test set remote sensing image into the trained road recognition model to obtain a road recognition test prediction image; The network model evaluation module is used for comparing the test prediction image with a test set label image to obtain various evaluation indexes, and evaluating the trained road recognition model according to the evaluation index values.

Images