A remote sensing image road segmentation method based on diffusion pre-training

Abstract

The application provides a remote sensing image road segmentation method based on diffusion pre-training, constructs a segmentation backbone model based on a UNet network, and encodes the segmentation backbone model by using one ConvBlock and four encoding blocks; the decoding part of the segmentation backbone model comprises four decoding blocks and a prediction head; after the remote sensing image is subjected to the ConvBlock, the four encoding blocks, the four decoding blocks and the prediction head, a road segmentation picture is output; based on a denoising diffusion probability model, noise is added to the remote sensing image in a pre-training data set to obtain a remote sensing image with noise; the remote sensing image with noise is input into the segmentation backbone model, a reconstructed image is output by the fourth decoding block, and the pre-training process of the segmentation backbone model is completed by taking the minimum value of the reconstructed image and the original remote sensing image as the target; the segmentation backbone model weight obtained by pre-training is loaded, the segmentation backbone model is fine-tuned based on a fine-tuning data set, and is used for an actual remote sensing image road segmentation task. The application can guarantee segmentation accuracy under the condition that the label is less, and can effectively process complex background and noise interference.

Description

TECHNICAL FIELD

[0001] The present application relates to computer vision and deep learning technology, in particular to a remote sensing image road segmentation method based on diffusion pre-training. BACKGROUND

[0002] Remote sensing image road segmentation algorithm is mainly applied to urban planning, traffic monitoring and disaster assessment fields. Accurate road segmentation is crucial for traffic flow prediction, urban expansion planning, public facility layout, etc. Automated road extraction can greatly improve the efficiency and accuracy of planning. Traditional fully supervised methods have a serious dependence on image annotation. Especially in remote sensing image road segmentation tasks, deep learning models usually face challenges such as difficulty in data set annotation and insufficient model generalization ability. The generalization ability of the model may be affected by the quality of the label, and annotation errors can lead to misjudgment of the model. In addition, obtaining high-quality labeled data is not only costly but also time-consuming, especially in dense prediction tasks such as segmentation, making this problem particularly prominent. SUMMARY

[0003] The purpose of the present application is to provide a remote sensing image road segmentation method based on diffusion pre-training to solve the shortcomings of existing road segmentation techniques in processing remote sensing images, especially in terms of precision, robustness and dependence on labeled data.

[0004] The technical solution to achieve the purpose of the present application is: a remote sensing image road segmentation method based on diffusion pre-training, including the following contents and steps:

[0005] Step 1, construct a pre-training data set and a fine-tuning data set, wherein the pre-training data set only contains remote sensing images of complete route maps, and the fine-tuning data set contains remote sensing images and corresponding road annotations;

[0006] Step 2, construct a segmentation backbone model based on a UNet network, the encoding part includes a ConvBlock and 4 encoding blocks, the decoding part includes 4 decoding blocks and a prediction head, and the remote sensing image is output after passing through the ConvBlock, 4 encoding blocks, 4 decoding blocks and a prediction head. Road segmentation picture;

[0007] Step 3, based on the denoising diffusion probability model, add noise to the remote sensing images in the pre-training data set to obtain noisy remote sensing images, input the segmentation backbone model, and output the reconstructed image from the fourth decoding block, with the goal of minimizing the reconstructed image and the original remote sensing image value. Complete the pre-training process of the segmentation backbone model;

[0008] Step 4, load the segmentation backbone model weight obtained by pre-training, and complete the fine-tuning of the segmentation backbone model based on the fine-tuning data set;

[0009] Step 5, input the remote sensing image to be tested into the fine-tuned segmentation backbone model to complete the actual remote sensing image road segmentation task.

[0010] Further, step 1, construct a pre-training data set and a fine-tuning data set, the specific method is:

[0011] Collect images from multiple public data sets, crop the unannotated images after removing images without complete route maps, and construct a pre-training data set;

[0012] Collect images from multiple public data sets, select data with images and corresponding road annotations, and construct a fine-tuning data set.

[0013] Further, step 2, construct a segmentation backbone model based on the UNet network, the encoding part includes ConvBlock and 4 encoding blocks, the decoding part includes 4 decoding blocks and a prediction head, and the remote sensing image is output after passing through ConvBlock, 4 encoding blocks, 4 decoding blocks and a prediction head, the specific method is:

[0014] Given a remote sensing image The input is changed to Then it is changed to a feature map after the first encoding block By analogy, after passing through the second, third and fourth encoding blocks, feature maps X2, X3 and X4 are obtained, in this process, the number of channels gradually increases and the size of the feature map gradually decreases; correspondingly, feature map X4 is spliced with feature map X3 and input to the fourth decoding block to obtain feature map Y4, Y4 is spliced with X2 and input to the third decoding block to obtain feature map Y3; by analogy, the remaining decoding blocks output Y2 and Y1; finally, the feature map Y1 is output after 1x1 Conv, and the road segmentation picture is output.

[0015] The encoding block is composed of a PatchMerging and a SSFBlock in series, and the decoding block is composed of a PatchExpand and a SSFBlock in series;

[0016] a. Spectral-spatial fusion block (SSFBlock)

[0017] After the PatchMerging operates on the feature map X l-1 , the feature map In the first SSFBlock, the feature map is first subjected to layer normalization (LayerNorm) and spectral convolution operation to obtain the feature map , which is added to the feature map to obtain the feature map After that, the feature map After layer normalization and MLP operation, the output is obtained The output obtained is added to the feature map to obtain the output X of this layer l ;

[0018] b, spectral convolution (SP-Conv)

[0019] The feature map of the first layer After normalization, the feature map is obtained In the spectral convolution module, the feature map is divided into two tensors along the channel dimension where X' l is directly input into the adaptive feature selection and enhancement (AFSE) layer to obtain the feature map X' l After DCT transformation, the feature map is input into the AFSE layer, and then inverse transformation is performed to obtain the feature map Finally, and are spliced along the channel to obtain the spectral convolution output

[0020] c, AFSE layer

[0021] The AFSE layer is composed of a detail enhancement branch (DEB), a meta attention branch (MAB), and a linear transformation branch (LTB) in parallel;

[0022] The DEB is a depth separable convolution that enhances local detailed features by sequentially combining depth convolution and point-wise convolution; the LTB is a parallel branch of the DEB that performs linear transformation in the channel dimension through 1×1 convolution; for a feature map X' l , the basic structure composed of DEB and LTB is described as follows:

[0023]

[0024] where, represents 1×1 convolution, represents 3×3 depth convolution;

[0025] The MAB is a meta attention module that aims to refine semantic information through an attention mechanism similar to squeeze-and-excitation to generate semantic perception weights for re-weighting and re-calibrating features from the DEB and LTB branches; for a given feature map The MAB process is described as follows:

[0026]

[0027] where GAP(·) and GMP(·) represent global average pooling and global max pooling operations, respectively, and FC(·) represents a fully connected operation. m (·) represents a fully connected operation;

[0028] Finally, the AFSE layer outputs are expressed as:

[0029]

[0030] Further, step 3, based on the denoising diffusion probability model, adds noise to the remote sensing images in the pre-training data set to obtain noisy remote sensing images, inputs the segmentation backbone model, and outputs the reconstructed images from the fourth decoding block, with the goal of minimizing the value of the reconstructed images and the original remote sensing images, to complete the segmentation backbone model pre-training process, the specific method is:

[0031] The clear remote sensing image x0 is first transformed into the frequency domain by DCT to obtain the spectrum Then, noise is gradually added to obtain noisy spectrum Then, the noisy spectrum is subjected to iDCT to obtain noisy remote sensing images x1,…,x t-1 ,x t ,…x T Then, they are respectively input into the segmentation backbone model, and the fourth decoding block outputs the reconstructed images, and the reconstructed images are With the goal of minimizing the difference between the reconstructed images and the original image x0, the segmentation backbone model pre-training process is completed.

[0032] During the pre-training of the segmentation backbone model, mean square error (MSE) is selected as the loss function to measure the difference between the recovered images and the true images. During the optimization process, the AdamW optimizer is used, with an initial learning rate of 1e-4 and a dynamic learning rate scheduling strategy to ensure the stability of the training process and accelerate convergence. The maximum diffusion step is 1000 and the batch size is 128.

[0033] Further, step 4, load the segmentation backbone model weights obtained by pre-training, and complete the segmentation backbone model fine-tuning based on the fine-tuning data set, the specific method is:

[0034] Fine-tuning uses the segmentation backbone model given in step 2 and the fine-tuning data set prepared in step 1, and loads the pre-trained segmentation backbone model weights obtained in step 3 and applies them to the labeled fine-tuning data set for fine-tuning. During fine-tuning, the model's prediction head is initialized to 0, and the other parts remain unchanged.

[0035] During the fine-tuning process, mean square error (MSE) is used as the loss function, and the AdamW optimizer is used for parameter update, with a batch size of 8.

[0036] Further, step 5, input the to-be-tested remote sensing image into the fine-tuned segmentation backbone model, complete the actual remote sensing image road segmentation task, and the specific method is:

[0037] First, the input remote sensing image is cropped, then the processed image is input into the segmentation backbone model for inference, the probability value of each pixel point belonging to the road region is output through the Sigmoid activation function, finally, the final road segmentation result is generated according to the predicted probability value and saved as a gray image, wherein black represents the background and white represents the road.

[0038] A remote sensing image segmentation system based on diffusion pre-training is implemented, and the remote sensing image segmentation method based on diffusion pre-training is implemented.

[0039] A computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, the remote sensing image segmentation method based on diffusion pre-training is implemented, and the remote sensing image segmentation based on diffusion pre-training is realized.

[0040] A computer readable storage medium has a computer program stored thereon, and when the processor executes the computer program, the remote sensing image segmentation method based on diffusion pre-training is implemented, and the remote sensing image segmentation based on diffusion pre-training is realized.

[0041] Compared with the prior art, the present application has the following advantages:

[0042] 1) Based on the improved UNet model, combined with the spectral-spatial fusion block (SSFBlock), the ability of the model in local and global feature extraction is enhanced, and the segmentation accuracy of the road region in the image is significantly improved.

[0043] 2) In the pre-training stage, the frequency domain diffusion model is used for self-supervised learning, which effectively improves the robustness of the model in complex background and solves the challenge of less labeling. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 The flowchart of the remote sensing image road segmentation of the present application.

[0045] Figure 2 The flowchart of the frequency domain diffusion pre-training of the present application.

[0046] Figure 3 The overall structure diagram of the road segmentation segmentation backbone model of the present application.

[0047] Figure 4The schematic diagram of the road segmentation backbone model SSFBlock module of the present application.

[0048] Figure 5 The schematic diagram of the spectrum convolution module used by the road segmentation backbone model of the present application

[0049] Figure 6 The test effect diagram of the embodiment of the present application DETAILED DESCRIPTION

[0050] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application will be further described in detail below in combination with the drawings and specific embodiments.

[0051] The present application can effectively improve the accuracy of road segmentation in remote sensing images by introducing a diffusion self-supervised pre-training strategy and transferring it to the frequency domain perspective, and significantly reducing the need for a large amount of labeled data; in order to further improve the segmentation effect, the present application designs a suitable backbone network, which combines the advantages of the UNet framework and innovatively integrates local and global features in the spatial domain and the discrete cosine transform (DCT) domain.

[0052] As shown in Figure 1 The remote sensing image road segmentation method based on diffusion pre-training includes the following steps:

[0053] Step 1, constructing a pre-training data set and a fine-tuning data set:

[0054] The data set includes a pre-training data set and a fine-tuning data set, wherein the pre-training data set only includes remote sensing images, and the fine-tuning data set includes remote sensing images and corresponding road labels. The pre-training and fine-tuning images are all from public data sets.

[0055] For the pre-training data set, collect pictures from multiple public data sets (including SpaceNet3, OSM, Ottawa, RoadTracer), a total of 3846 pictures, then crop them and remove images without complete route maps, resulting in about 50000 unlabelled images; the fine-tuning data set uses public data sets (such as DeepGlobe, Massachusetts) with remote sensing images and corresponding road labels, and is divided into training set and validation set according to 8:2.

[0056] All training data are cropped into 256x256 image blocks and data augmentation with random flipping is applied.

[0057] Step 2, constructing a road segmentation backbone model:

[0058] As shown in Figure 3As shown, the backbone model is based on the UNet architecture, employs a symmetrical encoder and decoder design, and achieves the fusion of shallow and deep features through skip connections. The encoding part consists of a stem block (ConvBlock) composed of a 3×3 convolution and 4 encoding blocks, while the corresponding decoding part includes 4 decoding blocks and a lightweight prediction head (1×1 Conv).

[0059] Given a remote sensing image The input is transformed by ConvBlock Then it is transformed into a feature map after passing through the first encoding block. Following this process, the feature maps X2, X3, and X4 are obtained after passing through the second, third, and fourth encoding blocks, respectively. In this process, the number of channels gradually increases, while the feature map size gradually decreases. Correspondingly, feature map X4 is concatenated with feature map X3 and then input into the fourth decoding block to obtain feature map Y4. Y4 is then concatenated with Y2 and input into the third decoding block to obtain feature map Y3. This process continues, with the remaining decoding blocks outputting Y2 and Y1. Finally, feature map Y1 is processed by a 1×1 Conv to output the road segmentation image.

[0060] The encoding block consists of a PatchMerging and an SSFBlock concatenated together, and the decoding block consists of a PatchExpand and an SSFBlock concatenated together.

[0061] a spectral-spatial fusion block (SSFBlock)

[0062] SSFBlock, such as Figure 4 As shown, it consists of spectral convolutional (SP-Conv) layers and multilayer perceptron (MLP) layers, and adopts the residual connection structure in Transformer. PatchMerging is applied to the feature map X. l-1 After the operation, the feature map is obtained. In the l-th SSFBlock, the feature map The feature map is obtained by first performing layer normalization (LayerNorm) and spectral convolution operations. With feature map Add them together to obtain the feature map. Subsequent feature maps After layer normalization and MLP operations, we obtain The obtained output and feature map Add them together to get the output X of this layer. l The formula is expressed as follows:

[0063]

[0064] b-spectral convolution (SP-Conv)

[0065] As Figure 5 shown, spectral convolution (SP-Conv) includes two parallel branches in frequency domain and spatial domain. The feature map After normalization, the feature map In the spectral convolution module, the feature map is divided into two tensors in the channel dimension where X' l is directly input into the adaptive feature selection and enhancement (AFSE) layer to obtain the feature map X' l After DCT transformation, the feature map is input into the AFSE layer, and then inverse transformation is performed to obtain the feature map Finally, and are spliced along the channel to obtain the spectral convolution output

[0066] c AFSE layer

[0067] The AFSE layer is composed of a detail enhancement branch (DEB), a meta attention branch (MAB), and a linear transformation branch (LTB) in parallel.

[0068] The DEB is a depth separable convolution that enhances local detailed features by sequentially combining depth convolution and point-wise convolution. The LTB is a parallel branch of the DEB that performs linear transformation in the channel dimension through 1×1 convolution. For the feature map X′ l , the basic structure composed of the DEB and the LTB can be expressed as follows:

[0069]

[0070] where, represents 1×1 convolution, and 3×3 depth convolution.

[0071] The MAB is a meta attention module that aims to refine semantic information through an attention mechanism similar to squeeze-and-excitation to generate semantic perception weights for re-weighting and re-calibrating features from the DEB and LTB branches. For a given feature map The MAB process can be described as follows:

[0072]

[0073] where GAP(·) and GMP(·) represent global average pooling and global maximum pooling operations, respectively, and FC m (·) represents a fully connected operation.

[0074] Finally, the AFSE layer output is expressed as:

[0075]

[0076] Step 3, Self-supervised Diffusion Pre-training

[0077] The pre-training framework draws on the classic Denoising Diffusion Probabilistic Model (DDPM), but extends it to the frequency domain, proposing a frequency domain diffusion pre-training method. Its training framework is shown in Figure 2 , where the clear remote sensing image x0 is first transformed into the frequency domain by DCT to obtain the spectrum Then gradually add noise to get noisy spectrum Then respectively on the noisy spectrum iDCT is performed to obtain noisy remote sensing images x1,…,x t-1 ,x t ,…x T , and then input into the backbone model, and the reconstructed image is output by the fourth decoding block The goal is to minimize the difference between the reconstructed image and the original image x0, and the pre-training process of the backbone model is completed.

[0078] Overall, the pre-training architecture can be divided into four processes: discrete cosine transform, forward diffusion noise addition, backward denoising, and inverse discrete cosine transform. This process is unsupervised training and does not rely on image labels, but only uses the information of the image itself for training.

[0079] During pre-training, the mean square error (MSE) is chosen as the loss function to measure the difference between the recovered image and the true image. During optimization, the AdamW optimizer is used, with an initial learning rate of 1e-4 and a dynamic learning rate scheduling strategy to ensure the stability of the training process and accelerate convergence. The maximum diffusion step for training is 1000, and the batch size is 128. The optimization parameters are set to effectively improve the model performance and reduce overfitting.

[0080] Step 4, Fine-tuning for Road Segmentation:

[0081] Fine-tuning uses the segmentation model given in Step 2 and the fine-tuning dataset prepared in Step 1, and loads the pre-trained model weights obtained in Step 3 and applies them to the labeled fine-tuning dataset for fine-tuning. During fine-tuning, the prediction head of the model (i.e., the 1x1Conv block of Figure 3 ) is initialized to 0, while the other parts of the model remain unchanged.

[0082] During training, the mean square error (MSE) is used as the loss function, and the AdamW optimizer is used for parameter update, with a batch size of 8 to ensure effective fine-tuning under limited samples.

[0083] Step 5, Inference and Prediction:

[0084] After fine-tuning is completed, a fine-tuned segmentation model is obtained. This step is to input the actual remote sensing image to be tested into the fine-tuned segmentation network of step 4. First, the input remote sensing image is cropped. Then, the processed image is input into the model as Figure 3 for inference. Through the Sigmoid activation function, the model outputs the probability value of each pixel point belonging to the road region. Finally, the final road segmentation result is generated according to the predicted probability value, and is saved as a gray image, in which black represents the background and white represents the road.

[0085] The application also provides a remote sensing image segmentation system based on diffusion pre-training, which implements the remote sensing image segmentation method based on diffusion pre-training.

[0086] A computer device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, the remote sensing image segmentation method based on diffusion pre-training is implemented, and the remote sensing image segmentation based on diffusion pre-training is realized.

[0087] A computer readable storage medium has a computer program stored thereon, wherein when the computer program is executed by a processor, the remote sensing image segmentation method based on diffusion pre-training is implemented, and the remote sensing image segmentation based on diffusion pre-training is realized.

[0088] In summary, by introducing diffusion self-supervised learning, the method does not require image annotation during pre-training and only requires a small amount of annotation during fine-tuning, can effectively learn the road features in the remote sensing image under the condition of less labeled data, and the pre-trained model can improve the segmentation precision, accelerate the convergence, and significantly improve the performance of the model in practical application.

[0089] Embodiments

[0090] In order to verify the effectiveness of the application scheme, the Massachusetts dataset is selected for the following simulation experiment:

[0091] 1) Constructing a pre-training dataset and a fine-tuning dataset

[0092] The SpaceNet3, OSM, Ottawa, and RoadTracer public datasets of remote sensing images with complete route maps are collected, a total of 3846 images, which are then cropped to obtain about 50,000 unlabeled remote sensing images to construct a pre-training dataset; the Massachusetts dataset is divided into a training set and a validation set according to a ratio of 8:2 to obtain a fine-tuning dataset.

[0093] All images are cropped to 256x256 image patches and random flip data augmentation is applied during pre-training and fine-tuning.

[0094] 2) Building road segmentation backbone model

[0095] The backbone model is based on UNet structure, with symmetric encoder and decoder design, and through the jump connection to realize the fusion of shallow and deep features. The encoding part is composed of a stem block (ConvBlock) with 3x3 convolution and 4 encoding blocks, while the corresponding decoding part includes 4 decoding blocks and a lightweight prediction head (1x1Conv).

[0096] 3) Self-supervised diffusion pre-training

[0097] During pre-training, mean square error (MSE) is chosen as the loss function to measure the difference between the recovered image and the real image. During optimization, the AdamW optimizer is used, with an initial learning rate of 1e-4, and a dynamic learning rate scheduling strategy is adopted to ensure the stability of the training process and accelerate convergence. The maximum diffusion step for training is 1000, and the batch size is 128, which can effectively improve the model performance and reduce overfitting.

[0098] 4) Fine-tuning for road segmentation

[0099] Fine-tuning uses the segmentation model given in step 2 and the fine-tuning dataset made in step 1, and loads the pre-trained model weights trained in step 3, and applies it to the labeled fine-tuning dataset for fine-tuning. During fine-tuning, the prediction head of the model (i.e. Figure 3 1x1Conv block) is initialized to 0, while the other parts of the model remain the same as the original pre-training weights.

[0100] During training, mean square error (MSE) is used as the loss function, and the AdamW optimizer is used for parameter update, with a batch size of 8 to ensure effective fine-tuning under the condition of limited samples.

[0101] The fine-tuning results are shown in Table 1.

[0102] Table 1 Fine-tuning results comparison

[0103] Method Acc P R mIoU UNet 78.04 63.23 70.32 76.39 DDPM 77.14 84.24 79.35 83.86 Ours-model 78.81 85.88 78.81 83.29 Ours-pretrain 79.34 86.66 79.56 84.46

[0104] 5) Inference and prediction

[0105] The actual remote sensing image to be tested is input into the segmentation network fine-tuned in step 4. First, the input remote sensing image is cropped. Then, the processed image is input into the model as Figure 3The model performs inference on the input image. Through the Sigmoid activation function, the model outputs the probability value of each pixel point belonging to the road region. Finally, as shown in Figure 6 FIG. 3B, according to the predicted probability value, a final road segmentation result is generated and saved as a gray image, in which black represents the background and white represents the road.

[0106] The technical features of the above embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not contradict, they should be considered within the scope of the present disclosure.

[0107] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the scope of the present application. It should be pointed out that, for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A diffusion pre-training based remote sensing image road segmentation method, characterized in that, The following contents and steps are included: Step 1, construct a pre-training dataset and a fine-tuning dataset, wherein the pre-training dataset only contains remote sensing images of complete route maps, and the fine-tuning dataset contains remote sensing images and corresponding road labels; Step 2, construct a segmentation backbone model based on the UNet network, the encoding part includes a ConvBlock and 4 encoding blocks, the decoding part includes 4 decoding blocks and a prediction head, and the remote sensing image is output after passing through the ConvBlock, 4 encoding blocks, 4 decoding blocks and a prediction head, and the specific method is: The encoding block is composed of a PatchMerging and a SSFBlock in series, and the decoding block is composed of a PatchExpand and a SSFBlock in series; Given a remote sensing image The input is changed to a feature map by a ConvBlock Then it is changed to a feature map by a first encoding block Similarly, it is changed to feature maps X2, X3, X4 by second, third, and fourth encoding blocks, respectively. In this process, the number of channels increases gradually, and the size of the feature map decreases gradually. Correspondingly, feature map X4 is spliced with feature map X3 and then input to a fourth decoding block to obtain feature map Y4, Y4 is spliced with X2 and then input to a third decoding block to obtain feature map Y3. Similarly, the remaining decoding blocks output Y2 and Y1. Finally, feature map Y1 is output by a 1×1 Conv to obtain a road segmentation picture. a, spectral-spatial fusion block (SSFBlock) b, spectral convolution (SP-Conv) PatchMerging on the feature map X of the (l-1)-th layer l-1 After the operation, the feature map X is obtained In the l-th SSFBlock, the feature map X is obtained After layer normalization (LayerNorm) and spectral convolution operations, the feature map X is obtained is added to the feature map X to obtain the feature map X After that, the feature map X is obtained After layer normalization and MLP operations, the feature map X is obtained The output obtained is added to the feature map X to obtain the output X of this layer l ; c, AFSE layer Feature map of the first layer Feature map after normalization In the spectral convolution module, the feature map Split into two tensors along the channel dimension Where X ′ l Directly input into an adaptive feature selection and enhancement (AFSE) layer to obtain a feature map X" l Input into the AFSE layer after DCT transformation, and then perform inverse transformation to obtain a feature map Finally, And Spliced along the channel to obtain a spectral convolution output The AFSE layer is composed of a detail enhancement branch (DEB), a meta attention branch (MAB) and a linear transformation branch (LTB) in parallel; DEB is a kind of depth separable convolution, which enhances local detail features by sequentially combining depth convolution and pointwise convolution; Finally, the output of the AFSE layer is represented as: LTB is a parallel branch of DEB, which performs linear transformation in the channel dimension by 1x1 convolution; for feature map X ′ l The basic structure composed of DEB and LTB is expressed as follows: wherein, denotes a 1 x 1 convolution, denotes a 3 x 3 depthwise convolution; MAB is a meta-attention module, for a given feature map The MAB procedure is described as follows: where GAP(·) and GMP(·) denote global average pooling and global max pooling operation, respectively; FC(·) denotes a fully connected operation; m where GAP(·) and GMP(·) denote global average pooling and global max pooling operation, respectively; FC(·) denotes a fully connected operation; Step 3, based on the denoising diffusion probability model, add noise to the remote sensing images in the pre-training dataset to obtain noisy remote sensing images, input the segmentation backbone model, and output the reconstructed image from the fourth decoding block, with the minimum value of the reconstructed image and the original remote sensing image as the target, to complete the pre-training process of the segmentation backbone model, and the specific method is: Step 4, load the segmentation backbone model weight obtained by pre-training, and complete the fine-tuning of the segmentation backbone model based on the fine-tuning dataset; A clear remote sensing image x0 is first transformed to the frequency domain using DCT to obtain a spectrogram. Then, noise is gradually added to obtain the noisy spectrum diagrams. Next, the spectrum diagrams with noise were analyzed. Performing iDCT yields noisy remote sensing images x1,…,x t-1 ,x t ,…x T Then, the data is input into the segmentation backbone model, and the reconstructed image is output by the fourth decoding block. The reconstructed image is... Step 5, input the to-be-tested remote sensing image into the fine-tuned segmentation backbone model to complete the road segmentation task of the actual remote sensing image. Step 1, construct a pre-training dataset and a fine-tuning dataset, and the specific method is:

2. The diffusion-pretraining-based remote sensing image road segmentation method according to claim 1, characterized in that, Collect images from multiple public datasets, crop and remove images without complete route maps, and construct a pre-training dataset; Collect images from multiple public datasets, select images with corresponding road labels, and construct a fine-tuning dataset. Step 3, based on the denoising diffusion probability model, add noise to the remote sensing images in the pre-training dataset to obtain noisy remote sensing images, input the segmentation backbone model, and output the reconstructed image from the fourth decoding block, with the minimum value of the reconstructed image and the original remote sensing image as the target, to complete the pre-training process of the segmentation backbone model, and the specific method is:

3. The diffusion-pretraining-based remote sensing image road segmentation method according to claim 1, characterized in that, ​ During the pre-training of the segmentation backbone model, the mean square error (MSE) is selected as the loss function to measure the difference between the restored image and the real image, and during the optimization process, the AdamW optimizer is used, the initial learning rate is set to 1e-4, and the dynamic learning rate scheduling strategy is adopted to ensure the stability of the training process and accelerate the convergence, and the maximum diffusion step of the training is 1000 and the batch size is 128.

4. The diffusion-pretraining-based remote sensing image road segmentation method according to claim 1, characterized in that, Step 4: Load the pre-trained segmentation backbone model weight, and complete the fine-tuning of the segmentation backbone model based on the fine-tuning dataset. The specific method is: The fine-tuning uses the segmentation backbone model given in step 2 and the fine-tuning dataset prepared in step 1, and loads the pre-trained segmentation backbone model weight trained in step 3 and applies it to the labeled fine-tuning dataset for fine-tuning. During fine-tuning, the prediction head of the model is initialized to 0, and the other parts remain the original pre-training weight unchanged. During the fine-tuning process, the mean square error (MSE) is used as the loss function, and the AdamW optimizer is used for parameter update, and the batch size is set to 8.

5. The diffusion-pretraining-based remote sensing image road segmentation method according to claim 1, characterized in that, Step 5: Input the to-be-tested remote sensing image into the fine-tuned segmentation backbone model to complete the actual remote sensing image road segmentation task. The specific method is: First, the input remote sensing image is cropped, then the processed image is input into the segmentation backbone model for inference, the probability value of each pixel belonging to the road region is output through the Sigmoid activation function, and finally the final road segmentation result is generated according to the predicted probability value and saved as a grayscale image, where black represents the background and white represents the road.

6. A remote sensing image segmentation system based on diffusion pre-training, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the remote sensing image segmentation method based on diffusion pre-training according to any one of claims 1-5, and realize remote sensing image segmentation based on diffusion pre-training.

7. A computer device, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the remote sensing image segmentation method based on diffusion pre-training according to any one of claims 1-5, and realize remote sensing image segmentation based on diffusion pre-training.

8. A computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the remote sensing image segmentation method based on diffusion pre-training according to any one of claims 1-5, and realize remote sensing image segmentation based on diffusion pre-training.

Images