Method and device for constructing road segmentation model of remote sensing image

Abstract

The invention provides a method and device for constructing a road segmentation model of a remote sensing image, and the method comprises the steps: obtaining a remote sensing image training sample set, enabling any group of remote sensing image training samples to comprise a sample image of which the road is partially covered, and a corresponding road label image, inputting the sample image of the remote sensing image training sample set into an initial model, and obtaining a road segmentation prediction image of the sample image, and performing parameter iteration on the initial model based on the difference between the road label image of the sample image and the road segmentation prediction image to obtain a road segmentation model. The method is suitable for the road segmentation task of the remote sensing image, and solves the problem that the road extraction result is missing due to the fact that the road is shielded or the road is densely distributed.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of remote sensing image technology, and in particular, to a method and device for constructing a road segmentation model of a remote sensing image. Background Art

[0002] Currently, deep learning road extraction technology can automatically and accurately identify and extract road networks from large-scale remote sensing image data, greatly improving the efficiency of urban planning and traffic control, and providing sufficient data support for the construction and sustainable development of smart cities.

[0003] However, the problem that road images extracted by deep learning road models may be partially missing due to occlusion by obstacles or overly dense roads is a technical problem that needs to be solved urgently.

[0004] Application Contents

[0005] This application describes a method and device for constructing a road segmentation model of a remote sensing image, which can solve the above technical problems.

[0006] According to a first aspect, a method for constructing a road segmentation model of a remote sensing image is provided, the method comprising:

[0007] Obtain a remote sensing image training sample set, where any set of remote sensing image training samples includes sample images of partially covered roads and their corresponding road label images;

[0008] Inputting a sample image of the remote sensing image training sample set into an initial road segmentation model to obtain a road segmentation prediction image of the sample image;

[0009] Based on the difference between the road label image of the sample image and the road segmentation prediction image, performing parameter iteration on the initial model to obtain a road segmentation model;

[0010] The initial road segmentation model includes a dynamic road detail matcher, a cross-context adaptive attention device and a multi-scale global information integrator, wherein:

[0011] The dynamic road detail matcher is used to obtain a shallow feature map through a feature image of a sample image, wherein the shallow feature map includes a geometric form of a road;

[0012] The cross-context adaptive attention device is used to obtain a middle-layer feature map according to the feature image and the shallow feature map, wherein the middle-layer feature map contains road surrounding environment information of the blocked road;

[0013] The multi-scale global information integrator is used to obtain a deep feature map based on the feature image and the middle-layer feature map, wherein the deep feature map contains deep semantic information and global information of the road, the deep semantic information of the road contains the relationship and attributes between the obscured road and surrounding objects, and the global information contains information on the overall layout and road structure of the obscured road.

[0014] According to a second aspect, a device for constructing a road segmentation model of a remote sensing image is provided, the device comprising:

[0015] An acquisition module is used to obtain a remote sensing image training sample set, wherein any set of remote sensing image training samples includes a sample image of a road being partially covered and a corresponding road label image;

[0016] An input module, used for inputting a sample image of the remote sensing image training sample set into an initial road segmentation model to obtain a road segmentation prediction image of the sample image;

[0017] A training module, configured to iterate parameters of the initial model based on the difference between the road label image of the sample image and the road segmentation prediction image, so as to obtain a road segmentation model;

[0018] The initial road segmentation model includes a dynamic road detail matcher, a cross-context adaptive attention device and a multi-scale global information integrator, wherein:

[0019] The dynamic road detail matcher is used to obtain a shallow feature map through a feature image of a sample image, wherein the shallow feature map includes a geometric form of a road;

[0020] The cross-context adaptive attention device is used to obtain a middle-layer feature map according to the feature image and the shallow feature map, wherein the middle-layer feature map contains road surrounding environment information of the blocked road;

[0021] The multi-scale global information integrator is used to obtain a deep feature map based on the feature image and the middle-layer feature map, wherein the deep feature map contains deep semantic information and global information of the road, the deep semantic information of the road contains the relationship and attributes between the obscured road and surrounding objects, and the global information contains information on the overall layout and road structure of the obscured road.

[0022] According to a third aspect, a method for road segmentation of a remote sensing image is provided, the method comprising:

[0023] Acquire a target remote sensing image and a road segmentation model constructed according to the road segmentation model construction method described in the above technical solution, wherein the target remote sensing image includes a partially covered target road;

[0024] The target remote sensing image is input into the road segmentation model to obtain an uncovered target road image in the target remote sensing image.

[0025] According to a fourth aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the program, the method for constructing a road segmentation model of a remote sensing image as described in the above technical solution is implemented.

[0026] According to a fifth aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the program, the road segmentation method for remote sensing images as described in the above technical solution is implemented.

[0027] In the above-mentioned system and method provided in the embodiments of this specification, a dynamic road detail matcher suitable for extracting a large amount of road detail information from a shallow feature map, a cross-context adaptive attention suitable for full context information extraction and cross-level feature interaction, and a multi-scale global information integrator capable of efficiently enhancing road global information and improving road continuity are proposed. The present invention is applicable to the field of road segmentation in high-resolution remote sensing images, and is particularly suitable for solving the problem of partial road loss caused by road occlusion or dense distribution. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0029] Figure 1 It is a schematic diagram of the network structure of a road segmentation model for remote sensing images provided by the present invention;

[0030] Figure 2 It is a structural schematic diagram of FusedMBConv in a road segmentation model of a remote sensing image provided by the present invention;

[0031] Figure 3 It is a structural schematic diagram of MBConv in a road segmentation model of a remote sensing image provided by the present invention;

[0032] Figure 4 It is a structural schematic diagram of a dynamic road detail matcher in a road segmentation model of a remote sensing image provided by the present invention;

[0033] Figure 5It is a structural schematic diagram of cross-context adaptive attention in a road segmentation model of a remote sensing image provided by the present invention;

[0034] Figure 6 It is a structural schematic diagram of a multi-scale global information integrator in a road segmentation model of a remote sensing image provided by the present invention;

[0035] Figure 7 is an example of a remote sensing image of the DeepGlobe dataset used in an embodiment of the present invention;

[0036] Figure 8 is an example of a remote sensing image with labels in the DeepGlobe dataset used in an embodiment of the present invention;

[0037] Fig. 9 It is a schematic diagram of road results extracted from the DeepGlobe data set using the road segmentation method of the present invention;

[0038] Fig.10 is an example of a remote sensing image of the Massachusetts data set used in an embodiment of the present invention;

[0039] Fig.11 is an example of a remote sensing image with labels in the Massachusetts dataset used in the embodiments of the present invention;

[0040] Fig.12 is a road result extracted from a Massachusetts data set using the road segmentation method of the present invention;

[0041] Fig.13 It is a flow chart of a method for constructing a road segmentation model of a remote sensing image provided by the present invention;

[0042] Fig.14 It is a structural schematic diagram of a device for constructing a road segmentation model of a remote sensing image provided by the present invention. DETAILED DESCRIPTION

[0043] The solution provided in this specification is described below in conjunction with the accompanying drawings.

[0044] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0045] When the road surface is blocked by objects such as trees, the complex environmental occlusion factors greatly reduce the recognizability of road features in remote sensing images. The surrounding contextual information and global information can be used to assist the model in assisted inference. For example, the existence of a road nearby can be inferred through contextual information such as crosswalks, buildings along the street, and traffic signs. In terms of global information, inference can be assisted by analyzing factors such as regional boundaries, urban layout, and transportation networks. For example, if adjacent farmland and woods can be identified, then there is likely to be a road at the junction of the two; at the same time, these roads next to the woods are also likely to face the problem of being blocked.

[0046] When roads are distributed too densely, how to accurately process and distinguish closely connected road structures within extremely small spatial intervals among highly crowded and visually intertwined road elements is a current difficulty.

[0047] The present invention is based on the U-Net architecture and implements a strategy of synchronous hierarchical learning and cross-level feature interaction, so as to fully utilize information from each level for accurate and complete road extraction. Specifically, synchronous hierarchical learning refers to the learning of feature maps of different levels and their information of different granularities in the encoder and jump connection parts of the U-Net architecture: shallow feature maps are large in size, with less loss of detail features, and are suitable for extracting small-scale road details; mid-level feature maps are balanced in size and content, rich in key contextual information, and help the model infer road paths; and deep feature maps have sufficient global information and high-level semantic information, which are suitable for learning the global dependencies of road structures and improving the model's understanding of coherent road structures.

[0048] In view of these hierarchical characteristics, the present invention designs corresponding modules at the corresponding positions of the model to fully learn these three types of information that are crucial to solving the missing problem. The cross-level feature interaction uses context information as a medium to integrate road details, global features and context information respectively. By integrating features from different levels, the model can better understand the road structure, ensuring that the model can maintain the continuity and integrity of the structure even in a highly complex road network, solving the problem of partial road missing.

[0049] like Figures 1 to 13 As shown, the present invention provides a method for constructing a road segmentation model of a remote sensing image, comprising the following steps:

[0050] Data are collected from two high-resolution remote sensing image road datasets, DeepGlobe and Massachusetts, which contain sample pairs of 1024×1024 pixels and 1500×1500 pixels, i.e., corresponding high-resolution remote sensing images and high-quality road labels.

[0051] These sample pairs are divided into training and test sets in a ratio of about 4:1. During the training process, these images are preprocessed and multiple data augmentation techniques, such as image flipping and random rotation, are implemented to enhance the diversity and richness of the training data. These operations are applied not only to images, but also to the corresponding label data.

[0052] The training set images that have been preprocessed and enhanced with various data are input into the established road extraction network, appropriate training parameters are set, the road extraction network is trained, and the optimal training weights are saved during the training process for testing.

[0053] The randomly divided test set images are input into the road segmentation model, the optimal weights saved during the training process are loaded, and the model's accurate segmentation results for high-resolution remote sensing image road data are output.

[0054] Specifically, Figure 1 As shown in the figure, the remote sensing image road extraction network model includes a five-layer structure. From top to bottom, the first layer includes the first stage of the encoder and the upsampling stage of the decoder. The second layer includes the second stage of the encoder, the dynamic road detail matcher and the fourth stage of the decoder. The third layer includes the third stage of the encoder, the dynamic road detail matcher and the third stage of the decoder. The fourth layer includes the fourth stage of the encoder, the cross-context adaptive attention and the second stage of the decoder. The fifth layer includes the fifth stage of the encoder, the cross-context adaptive attention, the multi-scale global information integrator and the first stage of the decoder. The processing results of each module in each stage are simultaneously output to other modules in the same stage and the same module in the next stage to form

[0055] The encoder uses the EfficientNet V2 network parameters pre-trained on the ImageNet-1K dataset to initialize the encoder module, while other parameters of the network are initialized randomly.

[0056] The EfficientNet V2 network is a highly optimized neural network architecture that is optimized through compound scaling technology, which not only improves the processing efficiency of the network but also ensures high accuracy.

[0057] The encoder module is mainly downsampled through the backbone network. The backbone network has five downsampling stages. Each stage will output feature maps of different dimensions for transfer to other parts or modules for learning. The structure of each stage is as follows:

[0058] The first stage consists of two layers. The first layer is configured with a standard convolution kernel of size 3×3 and a convolution step of 2, followed by batch normalization and SiLU activation function. The second layer is composed of two FusedMBConv blocks in the backbone network Efficientnet V2.

[0059] like Figure 2 As shown in Figure 1, the FusedMBConv block structure includes a standard convolution kernel of 3×3 size with a stride of 1, an SE layer and a standard convolution kernel of 1×1 size. After the feature image is processed by a standard convolution kernel of 3×3 size with a stride of 1, an SE layer and a standard convolution kernel of 1×1 size, the processed image is added to the input feature image to obtain the output feature image.

[0060] The second stage consists of the first-stage encoder, which contains four groups of FusedMBConv blocks. This encoder module passes the processed feature maps to the second-stage encoder and the first-stage skip-connected dynamic road detail matcher for learning road details.

[0061] The third stage consists of the second-level encoder, which contains four groups of FusedMBConv blocks. This encoder module passes the processed feature maps to the third-level encoder and the second dynamic road detail matcher to learn road details.

[0062] The fourth stage consists of the third-level encoder, including two stacked convolution blocks, one consisting of six groups of MBConv blocks and the other consisting of nine groups of MBConv blocks. The encoder of this stage passes the feature map to the fourth-level encoder and the first cross-context adaptive attention module.

[0063] like Figure 3 The MBConv block structure shown includes a 1×1 standard convolution kernel, a 3×3 depth-separable convolution kernel, an SE layer, and a 1×1 standard convolution kernel. After the feature image is processed by a 1×1 standard convolution kernel, a 3×3 depth-separable convolution kernel, an SE layer, and a 1×1 standard convolution kernel, the processed image is added to the input feature image to obtain the output feature image.

[0064] The fifth stage consists of the fourth-level encoder, which consists of two layers. The first layer contains 15 groups of MBConv blocks, and the second layer is configured with a standard convolution kernel of size 3×3, a stride of 2, batch normalization and SiLU activation function. This encoder outputs the feature map to the decoder and the multi-scale global information integrator.

[0065] The encoders of the above five stages output a total of five batches of feature maps. Among them, the feature map output by the first stage is the most superficial feature map, which is rich in the most road detail information. The feature map output by the next stage contains less detail information and more context information. The feature map output by the final stage is the deepest feature map, which contains the most global information and deep semantic information.

[0066] The dynamic road detail matcher is set in the first and second layer jump connections and belongs to the shallow part of the overall model. The feature map output by the encoder of the corresponding stage is matched with road details through multi-scale and multi-directional strip convolution, and a large amount of detail information in the shallow feature map is learned. The feature map output by the dynamic road detail matcher is then passed to the dynamic detail matcher of the next layer or the cross-context adaptive attention that can fully learn contextual information and perform cross-level feature interaction. The feature map is also passed to the decoder of the corresponding stage.

[0067] The dynamic road detail matcher includes strip convolution, dynamic snake convolution and depth-separable convolution, which improves the recognition ability of dense road structures and optimizes the accurate capture of road geometry through the dynamic adjustment characteristics of deformable convolution. The dynamic road detail matcher can maximize the use of rich detail information in the original feature map to achieve more complete road feature extraction, which can effectively solve the problem of missing parts of the road in dense scenes.

[0068] like Figure 4 As shown, the detailed structure of the dynamic road detail matcher is as follows:

[0069] The dynamic road detail matcher combines the feature map transmitted by the previous layer of skip connection and the feature map output by the encoder of this layer as the input of the module. Since the size and number of channels of the two do not match, the feature map transmitted by the previous layer of skip connection is first processed using depthwise separable convolution and pointwise convolution to match the feature map output by the encoder of this layer, and the result obtained by adding the two is used as the original input result. The original input result is input into three parallel branches to learn the detailed information of the road and match the geometric shape of the road. The first branch contains horizontal 1×15, 1×13 depthwise separable convolution and 1×11 dynamic snake convolution in sequence, the second branch contains vertical 15×1, 13×1 depthwise separable convolution and 11×1 dynamic snake convolution in sequence, and the third branch contains large-scale 15×15, 13×13 and 11×11 depthwise separable convolution in sequence.

[0070] The detail matching results of the three branches are added to the original input results, and then processed using 1×1 convolution, batch normalization and ReLU activation function respectively. The processed results are then added to the original input results through a jump connection. The nonlinear processing capability of the module is enhanced through the GELU activation function and used as the final output of the dynamic road detail matcher.

[0071] The output feature maps of the two dynamic road detail matchers are passed to the next level of skip connection and the decoder module of the corresponding level, respectively.

[0072] The cross-context adaptive attention is set at the third and fourth level jump connections, which belongs to the middle layer of the overall model. The cross-context adaptive attention receives the output of the previous layer dynamic detail matcher or the output of the previous layer cross-context adaptive attention, and uses multi-class and multi-scale convolution to learn the context information of the intermediate layer feature map, and uses cross-multi-head attention to fully interact the shallow detail information with the context information across levels. The feature map processed by the cross-context adaptive attention is then passed to the cross-context adaptive attention or multi-scale global information integrator of the next layer, and the feature map is also passed to the decoder of the corresponding stage.

[0073] Cross-context adaptive attention introduces a temperature parameter to adjust the sensitivity of the multi-head attention mechanism, allowing the model to automatically adjust its processing strategy according to the complexity of the input features. This method not only improves the recognition accuracy of road features, but also enhances the model's ability to cope with occlusion and complex backgrounds.

[0074] In addition, cross-context adaptive attention integrates multiple types of convolutions. Through multi-scale context information enhancement and cross-level cross-attention mechanism, it significantly improves the model's ability to accurately extract and restore road features in complex backgrounds, especially in scenarios with visual interference and road occlusion. It can effectively improve road continuity and optimize segmentation results.

[0075] like Figure 5 As shown in the figure, the detailed structure of the cross-context adaptive attention is as follows: the cross-context adaptive attention combines the feature map transmitted by the previous layer of jump connection and the feature map output by the encoder of this layer as the input of the module. Since the sizes and number of channels of the two do not match, it is necessary to first use depth-wise separable convolution and point-by-point convolution to process the feature map transmitted by the previous layer of jump connection to match the feature map output by the encoder of this layer. After this processing, the sizes and number of channels of the two are the same.

[0076] The two batches of feature maps are input into the multi-scale context supplement mixer in two branches respectively for full context information mixing. The multi-scale context supplement mixer sequentially contains standard convolution with a convolution kernel size of 5×5, dilated convolution with a size of 7×7 and a dilation factor of 2, and a large-scale depthwise separable convolution with a size of 9×9.

[0077] The feature maps obtained from the two branches that have fully learned the contextual information are feature mapped using 1×1 convolution to generate attention mechanism query vectors, value vectors and key vectors for two batches of feature maps. The key vectors of the two batches of feature maps are cross-exchanged, and the value vectors and key vectors of the other batch of feature maps are used for attention mechanism weighting, so that the model can fully perform cross-level feature interaction learning on shallow road details and middle-layer contextual information. After learning the dependencies between channels using 1×1 convolution on the attention-weighted feature maps, the feature maps of the two branches are added as the final output of the cross-context adaptive attention. The outputs of the two cross-context adaptive attentions are respectively passed to the next level of the jump connection and the decoder module of the corresponding level.

[0078] The multi-scale global information integrator integrates the global information in the outputs of the two cross-context adaptive attentions and the output of the deepest encoder through efficient global average pooling operations, enhancing the model's ability to understand global information and helping the model learn more coherent paths. The multi-scale global information integrator multiplies the global vector after global average pooling with the output of the deepest encoder to highlight the global information of the feature map, and then inputs the three batches of feature maps into the efficient RepViT Blocks at the same time to perform deep modeling of the global information. The three batches of feature maps are integrated as the output of the module and passed to the decoder part.

[0079] The advantages of the multi-scale global information integrator are mainly reflected in its cross-level feature integration and powerful global information processing capabilities, which are particularly suitable for structured visual tasks such as road extraction. The multi-scale global information integrator emphasizes the global relevance of the road and improves the model's understanding of the overall image structure, which is particularly suitable for solving the problem of partial missing caused by road occlusion.

[0080] like Figure 6 As shown in Figure 2, the detailed structure of the multi-scale global information integrator is as follows:

[0081] The input of the multi-scale global information integrator includes the output of two cross-context adaptive attentions and the output feature map of the deepest encoder. The module uses a simple and effective global average pooling operation to learn the global information of the three batches of feature maps, and multiplies the learned global information vector with the output of the deepest encoder. The feature map is enhanced by the dot multiplication operation, and then the enhanced feature map is input into the efficient RepViT Blocks. The Transformer architecture is used for global information modeling, and the three batches of feature maps and the original output of the deepest encoder are concatenated. They are processed in sequence using 1×1 convolution, batch normalization, and ReLU activation functions to obtain the final output of the multi-scale global information integrator, and the output feature map is passed to the decoder for road reproduction and reconstruction.

[0082] The decoder mainly processes the feature map through deconvolution, batch normalization and ReLU activation function to reconstruct the extracted road.

[0083] The detailed structure of the encoder part is as follows:

[0084] The decoder part is divided into five stages, where the processing flow of the first to fourth stages is the same.

[0085] The processing flow from the first stage to the fourth stage is as follows:

[0086] First, a 1×1 convolution is used to reduce the number of channels of the feature map to a quarter of the original number, and batch normalization and ReLU activation functions are used for integration and nonlinear enhancement. Next, a 3×3 inverse convolution with a step size of 2 is used to enlarge the feature map, and batch normalization and ReLU activation functions are used again for processing. Finally, a 1×1 convolution is used to adjust the number of channels to match the feature map of the next stage, and batch normalization and ReLU activation functions are used for final integration. The output of each stage will be passed to the next decoder stage.

[0087] The decoder in the fifth stage uses inverse convolution with a size of 4 and a stride of 2, ReLU activation function, standard convolution with a size of 3, ReLU activation function, standard convolution with a size of 3, and Sigmoid function to obtain the final road segmentation result.

[0088] In order to further verify the feasibility and effectiveness of the present invention, the relevant configurations of the remote sensing road extraction network were built: Ubuntu 18.04, Python 3.9.6, Pytorch 1.13.1, and CUDA 11.6. The test results of the model were quantified using four evaluation indicators commonly used in semantic segmentation tasks, namely, accuracy, recall, intersection-over-union, and harmonic mean.

[0089] use Figure 7Experiments were conducted on road images in the DeepGlobe dataset shown in the figure. Figure 8 is a road remote sensing image with labels in the DeepGlobe dataset, Fig. 9 This is a schematic diagram of the road segmentation results extracted from the DeepGlobe dataset. Figure 7 , Figure 8 and Fig. 9 It can be seen that the road segmentation results are very similar to the labeled road remote sensing images. The segmentation of densely populated road areas achieves a very high accuracy, and the recognition rate of covered roads is also very high. Only the broken road at the bottom of the picture is not recognized.

[0090] The experimental results are shown in Table 1:

[0091]

[0092] Table 1 Specific indicators on the DeepGlobe road extraction dataset

[0093] use Fig.10 The experiments were conducted using remote sensing images from the DMassachusetts dataset shown in the figure. Fig.11 is a road remote sensing image with labels in the Massachusetts dataset remote sensing image, Fig.12 This is a schematic diagram of the road segmentation results extracted from the Massachusetts dataset. Fig.10 , Fig.11 and Fig.12 It can be seen that the road segmentation results are very similar to the labeled road remote sensing images. The segmentation of densely populated road areas achieves extremely high accuracy, and the recognition rate of covered roads is also very high. Only some broken roads in the middle of the picture are not recognized.

[0094] The experimental results are shown in Table 2:

[0095]

[0096] Table 2 Specific indicators on the Massachusetts road extraction dataset

[0097] Compared with the prior art, the present invention proposes a dynamic road detail matcher suitable for extracting a large amount of road detail information from shallow feature maps, a cross-context adaptive attention suitable for full context information extraction and cross-level feature interaction, and a multi-scale global information integrator that can efficiently enhance road global information and improve road continuity. The present invention is applicable to the field of road segmentation in high-resolution remote sensing images, and is particularly suitable for solving the problem of partial road loss caused by road occlusion or dense distribution.

[0098] Fig.13A flow chart of a method for constructing a road segmentation model of a remote sensing image provided by an embodiment of this specification is shown. It includes:

[0099] 110. Obtain a remote sensing image training sample set, wherein any set of remote sensing image training samples includes sample images of partially covered roads and corresponding road label images.

[0100] 120. Input the sample image of the remote sensing image training sample set into the initial road segmentation model to obtain the road segmentation prediction image of the sample image.

[0101] 130. Based on the difference between the road label image of the sample image and the road segmentation prediction image, the initial road segmentation model is iterated to obtain a road segmentation model.

[0102] The initial road segmentation model includes a dynamic road detail matcher, a cross-context adaptive attention unit, and a multi-scale global information integrator, where:

[0103] The dynamic road detail matcher is used to obtain a shallow feature map through the feature image of the sample image, and the shallow feature map contains the geometric shape of the road.

[0104] A cross-context adaptive attention device is used to obtain a middle-layer feature map based on the feature image and the shallow feature map, where the middle-layer feature map contains the road surrounding environment information of the occluded road.

[0105] A multi-scale global information integrator is used to obtain a deep feature map based on the feature image and the mid-level feature map. The deep feature map contains deep semantic information and global information of the road. The deep semantic information of the road contains the relationship and attributes between the occluded road and surrounding objects, and the global information contains information on the overall layout and structure of the occluded road.

[0106] According to a further embodiment, the initial road segmentation model also includes an encoder; the encoder is used to obtain a feature image of the sample image based on the sample image.

[0107] The encoder includes a preprocessor, a first-stage encoder, a second-stage encoder, a third-stage encoder and a fourth-stage encoder.

[0108] The preprocessor is used to extract image features of the sample image to obtain a first feature image, and output the first feature image to the first-level encoder and the dynamic road detail matcher, wherein the size of the first feature image is smaller than the sample image.

[0109] The first-stage encoder is used to extract image features of the first feature image to obtain a second feature image, and output the second feature image to the second-stage encoder and the dynamic road detail matcher, wherein the size of the second feature image is smaller than the first feature image, the number of channels of the second feature image is greater than the first feature image, and the context information of the second feature image is more than that of the first feature image.

[0110] The second-stage encoder is used to extract image features of the second feature image to obtain a third feature image, and output the third feature image to the third-stage encoder and the dynamic road detail matcher.

[0111] The size of the third feature image is smaller than that of the second feature image, the number of channels of the third feature image is greater than that of the second feature image, and the context information of the third feature image is more than that of the second feature image.

[0112] The third-stage encoder is used to extract image features of the third feature image to obtain a fourth feature image, and output the fourth feature image to the third-stage encoder and the cross-context adaptive attention device.

[0113] The size of the fourth feature image is smaller than that of the third feature image, the number of channels of the fourth feature image is greater than that of the third feature image, and the context information of the fourth feature image is more than that of the third feature image.

[0114] The fourth-stage encoder is used to extract image features of the fourth encoder feature image to obtain a fifth feature image of the sample image, wherein the size of the fifth feature image is smaller than that of the fourth feature image, the number of channels of the fifth feature image is greater than that of the fourth feature image, and the context information of the fifth feature image is more than that of the fourth feature image.

[0115] According to a further embodiment, the road segmentation model includes a first dynamic road detail matcher and a second dynamic road detail matcher.

[0116] The first dynamic road detail matcher includes at least two parallel branch processing modules, wherein at least one parallel branch processing module includes a dynamic snake convolution layer;

[0117] A first dynamic road detail matcher, used for obtaining a first shallow feature map according to the first feature image and the second feature image;

[0118] The second dynamic road detail matcher includes at least two parallel branch processing modules, at least one of which includes a dynamic snake convolution layer.

[0119] The second dynamic road detail matcher is used to obtain a second shallow feature map according to the second feature image and the first shallow feature map, wherein the size of the second shallow feature map is smaller than that of the first shallow feature map, the number of channels of the second shallow feature map is more than that of the first shallow feature map, and the context information contained in the second shallow feature map is more than that of the first shallow feature map.

[0120] Based on a further embodiment, the road segmentation model includes a first cross-context adaptive attention device and a second cross-context adaptive attention device, the first cross-context adaptive attention device includes two multi-scale context supplement mixers, and the second cross-context adaptive attention device includes two multi-scale context supplement mixers.

[0121] The first cross-context adaptive attention device is used to match the second shallow feature map and the fourth feature image, and the obtained matching result images are respectively input into two multi-scale context supplement mixers to obtain a first cross-attention feature image and a second cross-attention feature image.

[0122] Feature mapping is performed on the first cross-attention feature image to obtain the query vector, value vector and key vector of the first attention mechanism, and feature mapping is performed on the second cross-attention feature image to obtain the query vector, value vector and key vector of the second attention mechanism.

[0123] Swap the query vector of the first attention mechanism with the query vector of the second attention mechanism;

[0124] The first cross-attention feature image is weighted by using the value vector and the key vector of the first attention mechanism and the query vector of the second attention mechanism to obtain a weighted first cross-attention feature image.

[0125] The second cross-attention feature image is weighted by using the value vector and the key vector of the second attention mechanism and the query vector of the first attention mechanism to obtain a weighted second cross-attention feature image.

[0126] The weighted first cross-attention feature image and the weighted second cross-attention feature image are added after learning the dependency between channels to obtain a first middle-level feature map.

[0127] A second cross-context adaptive attention device is used to match the first middle-layer feature map and the fifth feature image, and the obtained matching result images are respectively input into two multi-scale context supplement mixers to obtain a third cross-attention feature image and a fourth cross-attention feature image;

[0128] Performing feature mapping on the third cross-attention feature image to obtain a query vector, a value vector, and a key vector of the third attention mechanism; performing feature mapping on the fourth cross-attention feature image to obtain a query vector, a value vector, and a key vector of the fourth attention mechanism;

[0129] Swap the query vector of the third attention mechanism and the query vector of the fourth attention mechanism;

[0130] The third fork attention feature image is weighted by using the value vector and the key vector of the third attention mechanism and the query vector of the fourth attention mechanism to obtain a weighted third cross attention feature image;

[0131] After weighting the fourth cross-attention feature image using the value vector and the key vector of the fourth attention mechanism and the query vector of the third attention mechanism, a weighted fourth cross-attention feature image is obtained;

[0132] The weighted third cross-attention feature image and the weighted fourth cross-attention feature image are added after learning the dependency between channels to obtain a second middle-level feature map, wherein the size of the second middle-level feature map is smaller than that of the first middle-level feature map, the number of channels of the second middle-level feature map is more than that of the first middle-level feature map, and the second middle-level feature map contains more contextual information than the first middle-level feature map.

[0133] According to a further embodiment, the multi-scale global information integrator includes three global average pooling layers;

[0134] A multi-scale global information integrator, used to input the fifth feature map, the first middle-layer feature map, and the second middle-layer feature map into three global average pooling layers in parallel to obtain a global information vector output by each global average pooling layer;

[0135] After multiplying each global information vector by the first feature map, a global information image corresponding to each global information vector is obtained;

[0136] Input each global information image into the RepViT Blocks module for global information modeling, and obtain the global information feature image corresponding to each global information image;

[0137] A deep feature map is obtained according to the global information feature images corresponding to each global information image and the first feature map.

[0138] Based on a further embodiment, the initial road segmentation model also includes a decoder; the decoder is used to obtain a road segmentation prediction image according to the feature image, the shallow feature map, the middle feature map and the deep feature map;

[0139] The decoder includes a first decoder, a second decoder, a third decoder, a fourth decoder and a fifth decoder;

[0140] A first decoder is used to integrate the deep feature map and the second middle feature map to obtain a first decoded feature image;

[0141] A second decoder is used to integrate the first decoding feature image and the first middle-layer feature image to obtain a second decoding feature image, wherein the size of the second decoding feature image is larger than the first decoding feature image, and the number of channels of the second decoding feature image is smaller than the first decoding feature image;

[0142] A third decoder is used to integrate the second decoding feature image and the second shallow feature image to obtain a third decoding feature image, wherein the size of the third decoding feature image is larger than the second decoding feature image, and the number of channels of the third decoding feature image is smaller than the second decoding feature image;

[0143] A fourth decoder is used to integrate the third decoded feature image and the first shallow feature image to obtain a fourth decoded feature image, wherein the size of the fourth decoded feature image is larger than the third decoded feature image, and the number of channels of the fourth decoded feature image is smaller than the third decoded feature image;

[0144] The fifth decoder is used to process the fourth decoded feature image to obtain a road segmentation prediction image of the sample image.

[0145] Corresponding to the above method provided by the present invention, Fig.14 A schematic diagram of the structure of a device for constructing a road segmentation model of a remote sensing image provided in an embodiment of this specification is shown. The device includes:

[0146] An acquisition module is used to obtain a remote sensing image training sample set, wherein any set of remote sensing image training samples includes a sample image of a road being partially covered and a corresponding road label image;

[0147] An input module is used to input a sample image of a remote sensing image training sample set into an initial road segmentation model to obtain a road segmentation prediction image of the sample image;

[0148] A training module, for iterating parameters of an initial road segmentation model based on a difference between a road label image of a sample image and a road segmentation prediction image, to obtain a road segmentation model;

[0149] The initial road segmentation model includes a dynamic road detail matcher, a cross-context adaptive attention unit, and a multi-scale global information integrator, where:

[0150] Dynamic road detail matcher, used to obtain shallow feature map through feature image of sample image, in which the geometric shape of road is highlighted;

[0151] A cross-context adaptive attention device is used to obtain a middle-level feature map based on the feature image and the shallow feature map, where the middle-level feature map contains road surrounding environment information for predicting the occluded road;

[0152] A multi-scale global information integrator is used to obtain a deep feature map based on the feature image and the mid-level feature map. The deep feature map contains deep semantic information and global information of the road. The deep semantic information of the road describes the relationship and attributes between the occluded road and the surrounding objects, and the global information describes the information of the overall layout and road structure of the road used to predict the occluded road.

[0153] According to a further embodiment, the initial road segmentation model also includes an encoder; the encoder is used to obtain a feature image of the sample image based on the sample image.

[0154] The encoder includes a preprocessor, a first-stage encoder, a second-stage encoder, a third-stage encoder and a fourth-stage encoder.

[0155] The preprocessor is used to extract image features of the sample image to obtain a first feature image, and output the first feature image to the first-level encoder and the dynamic road detail matcher, wherein the size of the first feature image is smaller than the sample image.

[0156] The first-stage encoder is used to extract image features of the first feature image to obtain a second feature image, and output the second feature image to the second-stage encoder and the dynamic road detail matcher, wherein the size of the second feature image is smaller than the first feature image, the number of channels of the second feature image is greater than the first feature image, and the context information of the second feature image is more than that of the first feature image.

[0157] The second-stage encoder is used to extract image features of the second feature image to obtain a third feature image, and output the third feature image to the third-stage encoder and the dynamic road detail matcher.

[0158] The size of the third feature image is smaller than that of the second feature image, the number of channels of the third feature image is greater than that of the second feature image, and the context information of the third feature image is more than that of the second feature image.

[0159] The third-stage encoder is used to extract image features of the third feature image to obtain a fourth feature image, and output the fourth feature image to the third-stage encoder and the cross-context adaptive attention device.

[0160] The size of the fourth feature image is smaller than that of the third feature image, the number of channels of the fourth feature image is greater than that of the third feature image, and the context information of the fourth feature image is more than that of the third feature image.

[0161] The fourth-stage encoder is used to extract image features of the fourth encoder feature image to obtain a fifth feature image of the sample image, wherein the size of the fifth feature image is smaller than that of the fourth feature image, the number of channels of the fifth feature image is greater than that of the fourth feature image, and the context information of the fifth feature image is more than that of the fourth feature image.

[0162] According to a further embodiment, the road segmentation model includes a first dynamic road detail matcher and a second dynamic road detail matcher.

[0163] The first dynamic road detail matcher includes at least two parallel branch processing modules, wherein at least one parallel branch processing module includes a dynamic snake convolution layer;

[0164] A first dynamic road detail matcher, used for obtaining a first shallow feature map according to the first feature image and the second feature image;

[0165] The second dynamic road detail matcher includes at least two parallel branch processing modules, at least one of which includes a dynamic snake convolution layer.

[0166] The second dynamic road detail matcher is used to obtain a second shallow feature map according to the second feature image and the first shallow feature map, wherein the size of the second shallow feature map is smaller than that of the first shallow feature map, the number of channels of the second shallow feature map is more than that of the first shallow feature map, and the context information contained in the second shallow feature map is more than that of the first shallow feature map.

[0167] Based on a further embodiment, the road segmentation model includes a first cross-context adaptive attention device and a second cross-context adaptive attention device, the first cross-context adaptive attention device includes two multi-scale context supplement mixers, and the second cross-context adaptive attention device includes two multi-scale context supplement mixers.

[0168] The first cross-context adaptive attention device is used to match the second shallow feature map and the fourth feature image, and the obtained matching result images are respectively input into two multi-scale context supplement mixers to obtain a first cross-attention feature image and a second cross-attention feature image.

[0169] Feature mapping is performed on the first cross-attention feature image to obtain the query vector, value vector and key vector of the first attention mechanism, and feature mapping is performed on the second cross-attention feature image to obtain the query vector, value vector and key vector of the second attention mechanism.

[0170] Swap the query vector of the first attention mechanism with the query vector of the second attention mechanism;

[0171] The first cross-attention feature image is weighted by using the value vector and the key vector of the first attention mechanism and the query vector of the second attention mechanism to obtain a weighted first cross-attention feature image.

[0172] The second cross-attention feature image is weighted by using the value vector and the key vector of the second attention mechanism and the query vector of the first attention mechanism to obtain a weighted second cross-attention feature image.

[0173] The weighted first cross-attention feature image and the weighted second cross-attention feature image are added after learning the dependency between channels to obtain a first middle-level feature map.

[0174] A second cross-context adaptive attention device is used to match the first middle-layer feature map and the fifth feature image, and the obtained matching result images are respectively input into two multi-scale context supplement mixers to obtain a third cross-attention feature image and a fourth cross-attention feature image;

[0175] Performing feature mapping on the third cross-attention feature image to obtain a query vector, a value vector, and a key vector of the third attention mechanism; performing feature mapping on the fourth cross-attention feature image to obtain a query vector, a value vector, and a key vector of the fourth attention mechanism;

[0176] Swap the query vector of the third attention mechanism and the query vector of the fourth attention mechanism;

[0177] The third fork attention feature image is weighted by using the value vector and the key vector of the third attention mechanism and the query vector of the fourth attention mechanism to obtain a weighted third cross attention feature image;

[0178] After weighting the fourth cross-attention feature image using the value vector and the key vector of the fourth attention mechanism and the query vector of the third attention mechanism, a weighted fourth cross-attention feature image is obtained;

[0179] The weighted third cross-attention feature image and the weighted fourth cross-attention feature image are added after learning the dependency between channels to obtain a second middle-level feature map, wherein the size of the second middle-level feature map is smaller than that of the first middle-level feature map, the number of channels of the second middle-level feature map is more than that of the first middle-level feature map, and the second middle-level feature map contains more contextual information than the first middle-level feature map.

[0180] According to a further embodiment, the multi-scale global information integrator includes three global average pooling layers;

[0181] A multi-scale global information integrator, used to input the fifth feature map, the first middle-layer feature map, and the second middle-layer feature map into three global average pooling layers in parallel to obtain a global information vector output by each global average pooling layer;

[0182] After multiplying each global information vector by the first feature map, a global information image corresponding to each global information vector is obtained;

[0183] Input each global information image into the RepViT Blocks module for global information modeling, and obtain the global information feature image corresponding to each global information image;

[0184] A deep feature map is obtained according to the global information feature images corresponding to each global information image and the first feature map.

[0185] Based on a further embodiment, the initial road segmentation model also includes a decoder; the decoder is used to obtain a road segmentation prediction image according to the feature image, the shallow feature map, the middle feature map and the deep feature map;

[0186] The decoder includes a first decoder, a second decoder, a third decoder, a fourth decoder and a fifth decoder;

[0187] A first decoder is used to integrate the deep feature map and the second middle feature map to obtain a first decoded feature image;

[0188] A second decoder is used to integrate the first decoding feature image and the first middle-layer feature image to obtain a second decoding feature image, wherein the size of the second decoding feature image is larger than the first decoding feature image, and the number of channels of the second decoding feature image is smaller than the first decoding feature image;

[0189] A third decoder is used to integrate the second decoding feature image and the second shallow feature image to obtain a third decoding feature image, wherein the size of the third decoding feature image is larger than the second decoding feature image, and the number of channels of the third decoding feature image is smaller than the second decoding feature image;

[0190] A fourth decoder is used to integrate the third decoded feature image and the first shallow feature image to obtain a fourth decoded feature image, wherein the size of the fourth decoded feature image is larger than the third decoded feature image, and the number of channels of the fourth decoded feature image is smaller than the third decoded feature image;

[0191] The fifth decoder is used to process the fourth decoded feature image to obtain a road segmentation prediction image of the sample image.

[0192] According to another embodiment, a road segmentation method for a remote sensing image is also provided, comprising:

[0193] Acquire a target remote sensing image and a road segmentation model constructed according to the method for constructing a road segmentation model in the above embodiment, wherein the target remote sensing image includes a partially covered target road;

[0194] The target remote sensing image is input into the road segmentation model to obtain the target road image that is not covered in the target remote sensing image.

[0195] According to another embodiment, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the program, a method for constructing a road segmentation model of a remote sensing image as in the above technical solution is implemented.

[0196] According to another aspect of the embodiment, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute a method for constructing a road segmentation model of a remote sensing image.

[0197] Those skilled in the art should be aware that in one or more of the above examples, the functions described in this application can be implemented by hardware, software, firmware, or any combination thereof. When implemented by software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0198] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only the specific implementation methods of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present application should be included in the scope of protection of the present application.

Claims

1. A method for constructing a road segmentation model of a remote sensing image, characterized in that: The method comprises: Obtain a remote sensing image training sample set, where any set of remote sensing image training samples includes sample images of partially covered roads and their corresponding road label images; Inputting a sample image of the remote sensing image training sample set into an initial road segmentation model to obtain a road segmentation prediction image of the sample image; Based on the difference between the road label image of the sample image and the road segmentation prediction image, performing parameter iteration on the initial road segmentation model to obtain a road segmentation model; The initial road segmentation model includes a dynamic road detail matcher, a cross-context adaptive attention device and a multi-scale global information integrator, wherein: The dynamic road detail matcher is used to obtain a shallow feature map through a feature image of a sample image, wherein the shallow feature map includes a geometric form of a road; The cross-context adaptive attention device is used to obtain a middle-layer feature map according to the feature image and the shallow feature map, wherein the middle-layer feature map contains road surrounding environment information of the blocked road; The multi-scale global information integrator is used to obtain a deep feature map based on the feature image and the middle-layer feature map, wherein the deep feature map contains deep semantic information and global information of the road, the deep semantic information of the road contains the relationship and attributes between the obscured road and surrounding objects, and the global information contains information on the overall layout and road structure of the obscured road.

2. The method according to claim 1, characterized in that The initial road segmentation model also includes an encoder; the encoder is used to obtain a feature image of the sample image according to the sample image; The encoder comprises a preprocessor, a first-stage encoder, a second-stage encoder, a third-stage encoder and a fourth-stage encoder; The preprocessor is used to extract the image features of the sample image to obtain a first feature image, and output the first feature image to the first-stage encoder and the dynamic road detail matcher, wherein the size of the first feature image is smaller than that of the sample image; The first-stage encoder is used to extract image features of the first feature image to obtain a second feature image, and output the second feature image to the second-stage encoder and the dynamic road detail matcher, wherein the size of the second feature image is smaller than that of the first feature image, the number of channels of the second feature image is greater than that of the first feature image, and the context information of the second feature image is more than that of the first feature image; The second-stage encoder is used to extract image features of the second feature image to obtain a third feature image, and output the third feature image to the third-stage encoder and the dynamic road detail matcher, wherein the size of the third feature image is smaller than that of the second feature image, the number of channels of the third feature image is greater than that of the second feature image, and the context information of the third feature image is more than that of the second feature image; The third-stage encoder is used to extract image features of the third feature image to obtain a fourth feature image, and output the fourth feature image to the third-stage encoder and the cross-context adaptive attention device, wherein the size of the fourth feature image is smaller than that of the third feature image, the number of channels of the fourth feature image is greater than that of the third feature image, and the context information of the fourth feature image is more than that of the third feature image; The fourth-level encoder is used to extract image features of the fourth encoder feature image to obtain a fifth feature map of the sample image, wherein the size of the fifth feature image is smaller than that of the fourth feature image, the number of channels of the fifth feature image is greater than that of the fourth feature image, and the context information of the fifth feature image is more than that of the fourth feature image.

3. The method according to claim 2, characterized in that The road segmentation model includes a first dynamic road detail matcher and a second dynamic road detail matcher; The first dynamic road detail matcher includes at least two parallel branch processing modules, at least one of which includes a dynamic snake convolution layer; The first dynamic road detail matcher is used to obtain a first shallow feature map according to the first feature image and the second feature image; The second dynamic road detail matcher includes at least two parallel branch processing modules, at least one of which includes a dynamic snake convolution layer. The second dynamic road detail matcher is used to obtain a second shallow feature map based on the second feature image and the first shallow feature map, wherein the size of the second shallow feature map is smaller than that of the first shallow feature map, the number of channels of the second shallow feature map is more than that of the first shallow feature map, and the second shallow feature map contains more contextual information than the first shallow feature map.

4. The method according to claim 3, characterized in that: The road segmentation model includes a first cross-context adaptive attention device and a second cross-context adaptive attention device, wherein the first cross-context adaptive attention device includes two multi-scale context supplement mixers, and the second cross-context adaptive attention device includes two multi-scale context supplement mixers; The first cross-context adaptive attention device is used to match the second shallow feature map and the fourth feature image, and input the matching results into the two multi-scale context supplement mixers respectively to obtain a first cross-attention feature image and a second cross-attention feature image; Performing feature mapping on the first cross-attention feature image to obtain a query vector, a value vector, and a key vector of a first attention mechanism, and performing feature mapping on the second cross-attention feature image to obtain a query vector, a value vector, and a key vector of a second attention mechanism; Swapping the query vector of the first attention mechanism and the query vector of the second attention mechanism; weighting the first cross-attention feature image by using the value vector and the key vector of the first attention mechanism and the query vector of the second attention mechanism to obtain a weighted first cross-attention feature image; After weighting the second cross-attention feature image using the value vector and the key vector of the second attention mechanism and the query vector of the first attention mechanism, a weighted second cross-attention feature image is obtained; The first weighted cross-attention feature image and the second weighted cross-attention feature image are added after learning the dependency relationship between channels to obtain a first middle-level feature map; The second cross-context adaptive attention device is used to match the first middle-level feature map and the fifth feature image, and input the matching results into the two multi-scale context supplement mixers respectively to obtain a third cross-attention feature image and a fourth cross-attention feature image; Performing feature mapping on the third cross-attention feature image to obtain a query vector, a value vector, and a key vector of a third attention mechanism; performing feature mapping on the fourth cross-attention feature image to obtain a query vector, a value vector, and a key vector of a fourth attention mechanism; Swapping the query vector of the third attention mechanism and the query vector of the fourth attention mechanism; After weighting the third fork attention feature image using the value vector and the key vector of the third attention mechanism and the query vector of the fourth attention mechanism, a weighted third cross attention feature image is obtained; After weighting the fourth cross-attention feature image using the value vector and the key vector of the fourth attention mechanism and the query vector of the third attention mechanism, a weighted fourth cross-attention feature image is obtained; The weighted third cross-attention feature image and the weighted fourth cross-attention feature image are added after learning the dependency between channels to obtain the second middle-level feature map, wherein the size of the second middle-level feature map is smaller than that of the first middle-level feature map, the number of channels of the second middle-level feature map is more than that of the first middle-level feature map, and the context information contained in the second middle-level feature map is more than that of the first middle-level feature map.

5. The method according to claim 4, characterized in that The multi-scale global information integrator includes three global average pooling layers; The multi-scale global information integrator is used to input the fifth feature map, the first middle-level feature map and the second middle-level feature map into three global average pooling layers in parallel to obtain a global information vector output by each global average pooling layer; After multiplying each of the global information vectors by the first feature map, a global information image corresponding to each of the global information vectors is obtained; Inputting each of the global information images into the RepViT Blocks module to perform global information modeling, and obtaining a global information feature image corresponding to each of the global information images; The deep feature map is obtained according to the global information feature images corresponding to each of the global information images and the first feature map.

6. The method according to claim 5, characterized in that The initial road segmentation model also includes a decoder; the decoder is used to obtain the road segmentation prediction image according to the feature image, the shallow feature map, the middle feature map and the deep feature map; The decoders include a first decoder, a second decoder, a third decoder, a fourth decoder and a fifth decoder; The first decoder is used to integrate the deep feature map and the second middle feature map to obtain a first decoded feature image; The second decoder is used to integrate the first decoded feature image and the first middle-level feature image to obtain a second decoded feature image, wherein the size of the second decoded feature image is larger than the first decoded feature image, and the number of channels of the second decoded feature image is smaller than the first decoded feature image; The third decoder is used to integrate the second decoding feature image and the second shallow feature image to obtain a third decoding feature image, wherein the size of the third decoding feature image is larger than the second decoding feature image, and the number of channels of the third decoding feature image is smaller than the second decoding feature image; The fourth decoder is used to integrate the third decoding feature image and the first shallow feature image to obtain a fourth decoding feature image, wherein the size of the fourth decoding feature image is larger than that of the third decoding feature image, and the number of channels of the fourth decoding feature image is smaller than that of the third decoding feature image; The fifth decoder is used to obtain a road segmentation prediction image of the sample image after processing the fourth decoded feature image.

7. A device for constructing a road segmentation model of a remote sensing image, characterized in that: The device comprises: An acquisition module is used to obtain a remote sensing image training sample set, wherein any set of remote sensing image training samples includes a sample image of a road being partially covered and a corresponding road label image; An input module, used for inputting a sample image of the remote sensing image training sample set into an initial road segmentation model to obtain a road segmentation prediction image of the sample image; A training module, configured to iterate parameters of the initial model based on the difference between the road label image of the sample image and the road segmentation prediction image, so as to obtain a road segmentation model; The initial road segmentation model includes a dynamic road detail matcher, a cross-context adaptive attention device and a multi-scale global information integrator, wherein: The dynamic road detail matcher is used to obtain a shallow feature map through a feature image of a sample image, wherein the shallow feature map highlights the geometric form of the road; The cross-context adaptive attention device is used to obtain a middle-layer feature map according to the feature image and the shallow feature map, wherein the middle-layer feature map contains road surrounding environment information for predicting the obstructed road; The multi-scale global information integrator is used to obtain a deep feature map based on the feature image and the mid-level feature map, wherein the deep feature map contains deep semantic information and global information of the road, wherein the deep semantic information of the road describes the relationship and attributes between the occluded road and surrounding objects, and the global information describes information for predicting the overall layout and structure of the road for the occluded road.

8. A road segmentation method for remote sensing images, characterized in that: The method comprises: Acquire a target remote sensing image and a road segmentation model constructed according to the method for constructing a road segmentation model according to any one of claims 1 to 7, wherein the target remote sensing image includes a partially covered target road; The target remote sensing image is input into the road segmentation model to obtain an uncovered target road image in the target remote sensing image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the road segmentation method of remote sensing images as claimed in claim 8 is implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the method for constructing a road segmentation model of a remote sensing image as claimed in any one of claims 1 to 7 is implemented.

Images