Self-adaptive comparison view generation method for remote sensing image semantic segmentation

Abstract

The invention discloses a self-adaptive comparison view generation method for semantic segmentation of a remote sensing image. The method comprises the following steps: step 1, preprocessing an original remote sensing image; 2, constructing a contrast learning view; step 3, inputting the contrast learning view in the step 2e into a pre-constructed self-supervised contrast learning network for training; the self-supervised comparative learning network is composed of a backbone network and a multi-layer sensor, and the backbone network adopts a ResNet-18 structure; step 4, repeating the step 2 and the step 3 until a preset number of iterations is reached; and meanwhile, a trained self-supervised contrast learning network is obtained. According to the method, the diversity and quality of the view are improved, so that the precision of remote sensing image semantic segmentation and the generalization ability of the model are improved.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to an adaptive contrast view generation method for semantic segmentation of remote sensing images. Background Art

[0002] Remote sensing images usually contain complex and large-scale scene information. How to effectively understand and analyze this scene information is a core issue in remote sensing image processing. As a basic task of remote sensing image scene understanding, semantic segmentation aims to assign semantic labels to each pixel to achieve accurate classification and segmentation of images. Compared with traditional remote sensing image semantic segmentation methods, deep learning can extract more advanced and robust semantic features. However, due to limited labeled samples, the interpretation ability of existing models is insufficient. To address these problems, self-supervised contrastive learning has attracted widespread attention and has been widely used in remote sensing image scene understanding. Self-supervised contrastive learning enables the model to learn meaningful semantic features on unlabeled data by designing pre-training tasks, so that downstream segmentation tasks can benefit from these good feature representations.

[0003] In contrastive learning, the general idea is to maximize the similarity between different views of the same sample, while minimizing the similarity between views of different samples. Therefore, one of the core of contrastive learning is to generate diverse views to promote the learning of discriminative features. Data augmentation is a popular method for generating views, which is mainly divided into photometric data augmentation and geometric data augmentation. Photometric data augmentation involves changing the pixel values ​​of the image, such as adjusting brightness and contrast, blurring, etc.; geometric data augmentation mainly involves changing the geometric structure of the image, such as rotation, scaling, cropping, flipping, etc. Most current contrastive learning methods rely on manually designed data augmentation methods, which may not be applicable to different datasets and downstream tasks. Therefore, it is urgent to design an effective adaptive contrastive learning view generation method framework.

[0004] In recent years, many studies have shown that view-based contrastive learning relies on view generation. Existing view generation methods can be divided into two categories: one is to use a style transfer network, which uses an encoder-decoder architecture to encode semantic information and then stylizes it randomly through style amplification, generating multiple stylized views while retaining the original semantic information; the other is to use an adversarial strategy to adaptively generate views by adding perturbations to the input image during training, which are optimized by maximizing the contrast loss.

[0005] However, current research mainly focuses on adaptive photometric data augmentation, while the application of adaptive geometric data augmentation in view generation has not been fully explored. In remote sensing semantic segmentation tasks, there are significant differences in the orientations of different objects, such as buildings and roads. Due to the lack of research on geometric transformation, current methods are limited in generating diverse views and cannot effectively extract remote sensing image features. Summary of the invention

[0006] The purpose of the present invention is to provide an adaptive contrast view generation method for remote sensing image semantic segmentation, which improves the diversity and quality of views, thereby improving the accuracy of remote sensing image semantic segmentation and the generalization ability of the model.

[0007] The present invention adopts the following technical solution: an adaptive contrast view generation method for remote sensing image semantic segmentation, comprising the following steps: a contrast learning view construction method,

[0008] Step 1: Preprocessing of original remote sensing images:

[0009] Preprocess the original remote sensing images to obtain a pre-training set;

[0010] Step 2: Construct a comparative learning view:

[0011] Step 2a: Optimize the hue transformation parameters of the sample x in the pre-training set to obtain the sample after hue transformation is the hue value of the optimized view, HSV -1 Indicates converting samples from the HSV color space to the original color space;

[0012] Step 2b: Optimize the sample in step 2a The saturation transformation parameters are used to obtain the samples after hue transformation and saturation transformation. is the saturation of the optimized view;

[0013] Step 2c: Optimize the sample in step 2b The brightness transformation parameters are used to obtain the samples after hue transformation, saturation transformation and brightness transformation.

[0014] Step 2d: Optimize the sample in step 2c The contrast transformation parameters are used to obtain the samples after hue transformation, saturation transformation, brightness transformation and contrast transformation.

[0015] Step 2e: Optimize the sample in step 2d The rotation transformation parameters are used to obtain the contrast learning view after hue transformation, saturation transformation, brightness transformation, contrast transformation and rotation transformation.

[0016] Step 3: Compare the learning views in step 2e Input to the pre-built self-supervised contrastive learning network for training; the self-supervised contrastive learning network consists of a backbone network and a multi-layer perceptron, where the backbone network adopts the ResNet-18 structure;

[0017] Step 4: Repeat steps 2 and 3 until a preset number of iterations is reached; at the same time, a trained self-supervised contrastive learning network is obtained.

[0018] Furthermore, the specific process of step 2a is as follows:

[0019] Convert sample x to HSV color space, y h =HSV(x) represents the hue value of sample x, which ranges from [-π,π]; randomly initialize the hue transformation parameter δ h , perform random hue transformation on the sample, as shown in formula (A): h (y h )=y h +δ h (A);

[0020] Where: t h (y h ) represents the hue value after random transformation;

[0021] The samples after random tone transformation are sent to the self-supervised contrastive learning network to calculate the contrast loss, as shown in formula (B):

[0022]

[0023] Where L is the NT-Xent loss function; z i =f(x i ), f(x i ) is the sample x i The self-supervised contrast network is constructed; τ is the temperature coefficient, which is used to control the sharpness of the similarity distribution between views, and Ι [k≠i] is the indicator function, used to exclude its own similarity; z i For sample x i Features of the first view; z k is the characteristic of other samples in the batch; sim(z i ,z j ) is used to calculate the sample x i Cosine similarity of features of two views; z j For sample xi Features of the second view;

[0024] Using the FGSM method, the tone transformation parameters are adjusted according to the gradient of the contrast loss, as shown in formula (C):

[0025]

[0026] in, is the adjusted hue transformation parameter, α represents the perturbation intensity, HSV -1 Indicates converting samples from the HSV color space to the original color space; t h ,t s ,t b ,t c and t g They are hue transformation, saturation transformation, brightness transformation, contrast transformation and rotation transformation;

[0027] The adjusted hue conversion parameters The hue value y of sample x h Add together to get the transformed hue value And limit it to the range of [-π,π], as shown in formula (D):

[0028]

[0029] Get the sample after hue transformation

[0030] Furthermore, the specific process of step 2b is as follows: the sample in step 2a Convert to HSV color space, Represents the saturation value, which ranges from [0,1]; randomly initializes the saturation transformation parameter δ s , perform random saturation transformation on the sample, as shown in formula (E):

[0031] t s (y s )=y s ·δ s (E);

[0032] Where: t s (y s ) is the saturation value after randomly transforming the saturation value of the input sample;

[0033] The samples after hue transformation and random saturation transformation are sent to the self-supervised contrastive learning network to calculate the contrast loss. Then, the FGSM method is used to adjust the saturation transformation parameters according to the gradient of the contrast loss, as shown in formula (F):

[0034]

[0035] in, is the adjusted saturation transform parameter;

[0036] The adjusted saturation conversion parameter The hue value y of the input sample s Multiply to get the transformed saturation value And limit it to the range [0,1], as shown in formula (G):

[0037]

[0038] Get the sample after saturation transformation and hue transformation

[0039] Furthermore, the specific process of step 2c is as follows: the sample in step 2b Send it to the self-supervised contrast learning network to calculate the contrast loss; then use the FGSM method to adjust the brightness transformation parameter δ according to the gradient of the contrast loss b , as shown in formula (H):

[0040]

[0041] in, is the adjusted brightness transformation parameter.

[0042] The adjusted brightness transformation parameters With input sample Add them together to get the samples after hue, saturation and brightness transformation And limit it to the range [0,1], as shown in formula (I):

[0043]

[0044] Furthermore, the specific process of step 2d is as follows: the sample in step 2c Send it to the self-supervised contrast learning network to calculate the contrast loss; then use the FGSM method to adjust the contrast transformation parameter δ according to the gradient of the contrast loss c , as shown in formula (J):

[0045]

[0046] in, is the adjusted brightness transformation parameter.

[0047] Finally, the adjusted contrast transformation parameters With input sample Multiply them together to get the samples used for the transformation And limit it to the range [0,1], as shown in formula (K):

[0048]

[0049] Furthermore, the specific process of step 2e is as follows: the sample in step 2d Send it to the self-supervised contrast learning network to calculate the contrast loss; then use the FGSM method to adjust the rotation transformation parameter θ according to the gradient of the contrast loss, as shown in formula (L):

[0050]

[0051] in, is the adjusted rotation transformation parameter.

[0052] Finally, the adjusted rotation transformation parameters are applied to the input sample. On the top, we get the final transformed view sample As shown in formula (M):

[0053]

[0054] Furthermore, the specific process of step one is as follows: the original remote sensing image is cut into small images, and the small images are divided into three parts, one part is a pre-training set, the second part is a fine-tuning set, and the third part is a simulation test set, wherein the fine-tuning set is selected from the pre-training set.

[0055] The present invention also discloses a method for semantic segmentation of remote sensing images. Based on the above-mentioned method for generating adaptive contrast views for semantic segmentation of remote sensing images, after step 4, the method further includes the following steps:

[0056] Step 5: Backbone network extraction and integration:

[0057] Extracting a backbone network from the trained self-supervised contrastive learning network, embedding the backbone network into a feature extractor in a remote sensing image semantic segmentation network, which also includes a decoding module; the feature extractor is used to extract high-dimensional features in the remote sensing image, and the decoding module is used to map the high-dimensional features back to the spatial resolution of the input image and generate a classification prediction for each pixel;

[0058] Step 6: Remote sensing image semantic segmentation network fine-tuning:

[0059] The image samples in the fine-tuning set in step 1 are input into the remote sensing image semantic segmentation network, and the cross entropy loss function is used to optimize the parameters of the remote sensing image semantic segmentation network, as shown in formula (R):

[0060]

[0061] Get the fine-tuned remote sensing image semantic segmentation network;

[0062] Among them, N represents the number of pixels in the input image, M represents the number of ground object types, and y ic Indicates whether pixel i in the label belongs to category c, p ic Represents the probability that the network i outputs pixel i belonging to category c;

[0063] Step 7: Input the image samples in the simulation test set in step 1 into the fine-tuned remote sensing image semantic segmentation network in step 6, classify each pixel, and generate a complete semantic segmentation map.

[0064] The beneficial effects of the present invention are as follows: 1. By performing hue and saturation transformation in the HSV color space, brightness and contrast changes in the RGB color space, and geometric rotation transformation, a diverse contrast view is generated; since the contrast view covers a wide range of photometric and geometric transformations, the self-supervised contrast learning model can better understand and adapt to different remote sensing image features, thereby improving the classification accuracy in semantic segmentation tasks. 2. By optimizing the enhancement parameters in each iteration, the robustness of the model and its ability to capture features are enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 The semantic segmentation results of the DeepLabV3+ semantic segmentation model on the ISPRS Potsdam dataset after using different pre-training methods;

[0066] Among them: (a) original image; (b) real object classification map; (c) SimCLR pre-training method; (d) Viewmaker pre-training method; (e) the method of the present invention;

[0067] Figure 2 The semantic segmentation results of the DeepLabV3+ semantic segmentation model on the ISPRS Vaihingen dataset after using different pre-training methods;

[0068] Among them: (a) original image; (b) real object classification map; (c) SimCLR pre-training method; (d) Viewmaker pre-training method; (e) the method of the present invention;

[0069] Figure 3 The semantic segmentation results of the DeepLabV3+ semantic segmentation model on the LoveDA dataset after using different pre-training methods;

[0070] Among them: (a) original picture; (b) real object classification map; (c) SimCLR pre-training method; (d) Viewmaker pre-training method; (e) the method of the present invention. DETAILED DESCRIPTION

[0071] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0072] The present invention provides an adaptive contrast view generation method for remote sensing image semantic segmentation, comprising the following steps:

[0073] Step 1: Preprocessing of original remote sensing images:

[0074] Cutting the original remote sensing image into small blocks of images, and dividing the small blocks of images into three parts, one part is a pre-training set, the second part is a fine-tuning set, and the third part is a simulation test set, wherein the fine-tuning set is selected from the pre-training set;

[0075] For different remote sensing image datasets, the cutting process is as follows: the ISPRS Potsdam dataset is cropped using a sliding window with a step size of 256 to obtain 21888 256×256 samples, of which 13824 samples are used for self-supervised contrastive learning pre-training; the ISPRS Vaihingen dataset is cropped using a sliding window with a step size of 128 to obtain 33019 256×256 samples, of which 15268 samples are used for self-supervised contrastive learning pre-training; the LoveDA dataset is cropped using a sliding window with a step size of 256 to obtain 29328 256×256 samples, of which 18496 samples are used for self-supervised contrastive learning pre-training; for each dataset, 1% of the training samples are used for fine-tuning the remote sensing image semantic segmentation network.

[0076] Step 2: Construct a comparative learning view:

[0077] Define hue transformation, saturation transformation, brightness transformation, contrast transformation and rotation transformation as t h ,t s ,t b ,t c and t g .

[0078] Step 2a: Convert the sample x to HSV color space, y h =HSV(x) represents the hue value of sample x, which ranges from [-π,π]; randomly initialize the hue transformation parameter δ h , perform random hue transformation on the sample, as shown in formula (A): h (y h )=y h +δ h(A);

[0079] Where: t h (y h ) represents the hue value of the sample after random transformation;

[0080] The samples after random tone transformation are sent to the self-supervised contrastive learning network to calculate the contrast loss, as shown in formula (B):

[0081]

[0082] Where L is the NT-Xent loss function; z i =f(x i ), f(x i ) is the sample x i The self-supervised contrast network is constructed; τ is the temperature coefficient, which is used to control the sharpness of the similarity distribution between views, and Ι [k≠i] is the indicator function, used to exclude its own similarity; z i For sample x i Features of the first view; z k is the characteristic of other samples in the batch; sim(z i ,z j ) is used to calculate the sample x i Cosine similarity of features of two views; z j For sample x i Features of the second view;

[0083] Using the FGSM method, the tone transformation parameters are adjusted according to the gradient of the contrast loss, as shown in formula (C):

[0084]

[0085] in, is the adjusted hue transformation parameter, α represents the perturbation intensity, HSV -1 Indicates converting samples from the HSV color space to the original color space; t h ,t s ,t b ,t c and t g They are hue transformation, saturation transformation, brightness transformation, contrast transformation and rotation transformation;

[0086] The adjusted hue conversion parameters The hue value y of sample x h Add together to get the transformed hue value And limit it to the range of [-π,π], as shown in formula (D):

[0087]

[0088] Get the sample after hue transformation

[0089] Step 2b: the sample in step 2a Convert to HSV color space, Represents the saturation value, which ranges from [0,1]; randomly initializes the saturation transformation parameter δ s , perform random saturation transformation on the sample, as shown in formula (E):

[0090] t s (y s )=y s ·δ s (E);

[0091] Where: t s (y s ) is the saturation value after randomly transforming the saturation value of the input sample;

[0092] The samples after hue transformation and random saturation transformation are sent to the self-supervised contrastive learning network to calculate the contrast loss. Then, the FGSM method is used to adjust the saturation transformation parameters according to the gradient of the contrast loss, as shown in formula (F):

[0093]

[0094] in, is the adjusted saturation transform parameter;

[0095] The adjusted saturation conversion parameter The hue value y of the input sample s Multiply to get the transformed saturation value And limit it to the range [0,1], as shown in formula (G):

[0096]

[0097] Get the sample after saturation transformation and hue transformation

[0098] Step 2c: the sample in step 2b The self-supervised contrast learning network is sent to calculate the contrast loss; then the FGSM method is used to adjust the brightness transformation parameter δ according to the gradient of the contrast loss. b , as shown in formula (H):

[0099]

[0100] in, is the adjusted brightness transformation parameter.

[0101] The adjusted brightness transformation parameters With input sample Add them together to get the samples after hue, saturation and brightness transformation And limit it to the range [0,1], as shown in formula (I):

[0102]

[0103] Step 2d: the sample in step 2c is Send it to the self-supervised contrast learning network to calculate the contrast loss; then use the FGSM method to adjust the contrast transformation parameter δ according to the gradient of the contrast loss c , as shown in formula (J):

[0104]

[0105] in, is the adjusted brightness transformation parameter.

[0106] Finally, the adjusted contrast transformation parameters With input sample Multiply to get the transformed sample And limit it to the range [0,1], as shown in formula (K):

[0107]

[0108] Step 2e: the sample in step 2d is Send it to the self-supervised contrast learning network to calculate the contrast loss; then use the FGSM method to adjust the rotation transformation parameter θ according to the gradient of the contrast loss, as shown in formula (L):

[0109]

[0110] in, is the adjusted rotation transformation parameter.

[0111] Finally, the adjusted rotation transformation parameters are applied to the input sample. On the top, we get the final transformed view sample As shown in formula (M):

[0112]

[0113] Step 3: Compare the learning view in step 2e Input to a pre-built self-supervised contrastive learning network for training; the self-supervised contrastive learning network consists of a backbone network and a multi-layer perceptron, wherein the backbone network adopts the ResNet-18 structure, and solves the gradient vanishing problem in the deep network by introducing residual connections, and can more effectively learn the feature representation of the image. The multi-layer perceptron is composed of a linear layer, a ReLU activation function, and a fully connected layer connected in sequence, which is responsible for mapping the high-dimensional features output by the backbone network to a low-dimensional space. During the training process, the contrast loss function is used to optimize the network parameters. Specifically, the learning rate is set to 0.03, and the total number of iterations is set to 200 times.

[0114] Step 4: Repeat steps 2 and 3 until the preset number of iterations is reached; at the same time, a trained self-supervised contrastive learning network is obtained. The above steps are executed repeatedly until the preset number of iterations is reached, indicating that the model has converged and achieved the desired performance, and then the training process stops. Through this pre-training process, the self-supervised contrastive learning network can effectively learn highly discriminative feature representations from the input image, providing a good initial model for the remote sensing image semantic segmentation task.

[0115] The present invention also discloses a method for semantic segmentation of remote sensing images. Based on the above-mentioned method for generating adaptive contrast views for semantic segmentation of remote sensing images, the method further includes the following steps after step 4:

[0116] Step 5: Backbone network extraction and integration:

[0117] Extracting a backbone network from the trained self-supervised contrastive learning network, and embedding the backbone network into a feature extractor in a remote sensing image semantic segmentation network, wherein the remote sensing image semantic segmentation network also includes a decoding module; the feature extractor is used to extract high-dimensional features in the remote sensing image, and the decoding module is used to map the high-dimensional features back to the spatial resolution of the input image and generate a classification prediction for each pixel;

[0118] Step 6: Remote sensing image semantic segmentation network fine-tuning:

[0119] Based on the integrated semantic segmentation network constructed in step 4, 1% of the pre-training dataset is used for fine-tuning. The fine-tuning process aims to optimize the performance of the semantic segmentation model through supervised learning methods, making it more accurate in practical applications. During the fine-tuning process, the cross entropy loss function is used to optimize the parameters of the remote sensing image semantic segmentation network, as shown in formula (R):

[0120]

[0121] Among them, N represents the number of pixels in the input image, M represents the number of ground object types, and y icIndicates whether pixel i in the label belongs to category c, p ic Represents the probability that the network output pixel i belongs to category c.

[0122] Step 7. After completing the fine-tuning in step 6, use the fine-tuned remote sensing semantic segmentation network to predict the test image. The segmentation network will classify each pixel and generate a complete semantic segmentation map. These segmentation results will reflect the distribution of ground object categories in the input remote sensing image, providing a basis for further remote sensing image analysis and application.

[0123] The effect of the adaptive contrast view generation method for remote sensing image semantic segmentation of the present invention is further illustrated by the following simulation experiment:

[0124] Simulation conditions:

[0125] The hardware conditions for the simulation of this method are: windows XP, SPI, CPU Pentium(R)4, basic frequency is 2.4GHZ; the software platform is: Pycharm, pytorch;

[0126] The images used in the simulation are from the ISPRS Potsdam remote sensing image dataset, the ISPRS Vaihingen remote sensing image dataset, and the LoveDA remote sensing image dataset. The ISPRS Potsdam image dataset contains 6 types of objects, namely, impervious surface, buildings, low vegetation, trees, cars, and background; the ISPRS Vaihingen dataset contains 6 types of objects, namely, impervious surface, buildings, low vegetation, trees, cars, and background; the LoveDA dataset contains 7 types of objects, namely, background, buildings, roads, water bodies, bare soil, forest land, and cultivated land.

[0127] Simulation content and results:

[0128] Simulation: The method of the present invention is used to perform classification simulation on three data sets respectively. For the ISPRS Potsdam data set, 8064 test images are sent to the remote sensing image semantic segmentation network pre-trained and fine-tuned by the data set to obtain the segmentation result; for the ISPRS Vaihingen data set, 17751 test images are sent to the remote sensing image semantic segmentation network pre-trained and fine-tuned by the data set to obtain the segmentation result; for the LoveDA data set, 10832 test images are sent to the remote sensing image semantic segmentation network pre-trained and fine-tuned by the data set to obtain the segmentation result; Figure 1 , 2 and 3, where:

[0129] Figure 1(a) and (b) are the original image and the real object classification map, respectively. Figures (c) to (e) respectively show the semantic segmentation results of the remote sensing semantic segmentation model on the ISPRSPotsdam dataset using the SimCLR pre-training method, the Viewmaker pre-training method, and the method of the present invention;

[0130] Figure 2 (a) and (b) are the original image and the real object classification map, respectively. Figures (c) to (e) respectively show the semantic segmentation result maps of the remote sensing semantic segmentation model on the ISPRS Vaihingen dataset using the SimCLR pre-training method, the Viewmaker pre-training method and the method of the present invention;

[0131] Figure 3 (a) and (b) are the original image and the real object classification map, respectively. Figures (c) to (e) respectively show the semantic segmentation result maps of the remote sensing semantic segmentation model on the LoveDA dataset using the SimCLR pre-training method, the Viewmaker pre-training method, and the method of the present invention;

[0132] Depend on Figures 1 to 3 By comparing the results of remote sensing image semantic segmentation, we can see that Figure 1 It can be seen from the above that the method of the present invention has significant advantages over other methods in object edge description. Figure 2 It can be concluded from the first row that the method in the present invention is significantly better than other methods in predicting buildings. Figure 3 It shows that other methods will lose some ground object information during prediction, while the method in the present invention can effectively make predictions. From these visualization results, it can be seen that the adaptive contrast learning view generation method of the present invention significantly improves the feature extraction ability of the model in the remote sensing image semantic segmentation task, and the generated semantic segmentation results are more accurate.

[0133] In order to further quantify the performance advantages of the method in the present invention, Table 1, Table 2 and Table 3 list the semantic segmentation index values ​​of the method in the present invention and other seven pre-training methods on the above three datasets. Among them:

[0134] Table 1 is a numerical comparison of segmentation accuracy of the existing technologies MoCo, MoCo V2, SimCLR, BYOL, SimSiam, ACL, Viewmaker and the method of the present invention on the ISPRS Potsdam dataset;

[0135] Table 2 is a numerical comparison of segmentation accuracy of the existing technologies MoCo, MoCo V2, SimCLR, BYOL, SimSiam, ACL, Viewmaker and the method of the present invention on the ISPRS Vaihingen dataset;

[0136] Table 3 is a numerical comparison of the segmentation accuracy of the existing technologies MoCo, MoCo V2, SimCLR, BYOL, SimSiam, ACL, Viewmaker and the method of the present invention on the LoveDA dataset.

[0137] Table 1

[0138]

[0139]

[0140] Table 2

[0141] Training Methods QUR F1 mAcc MoCo 49.33 62.03 60.70 MoCo V2 49.86 62.42 61.17 SimCLR 52.43 64.96 63.41 BYOL 52.54 64.38 62.47 SimSiam 51.97 63.12 62.90 ACL 52.60 63.67 63.61 Viewmaker 52.59 63.84 62.01 Ours (method of the present invention) 53.08 64.98 63.68

[0142] Table 3

[0143] Training Methods QUR F1 mAcc MoCo 32.71 46.98 51.44 MoCo V2 33.60 48.28 50.07 SimCLR 35.39 50.93 51.94 BYOL 35.56 51.10 51.86 SimSiam 33.23 47.06 51.95 ACL 35.00 50.34 52.70 Viewmaker 34.81 48.83 49.31 Ours (method of the present invention) 36.04 51.92 52.82

[0144] The table uses three commonly used indicators in semantic segmentation, namely mIoU, F1 and mAcc, to evaluate the performance of the method in the present invention. The higher the values ​​of these indicators, the better the effect of the method. The experimental results show that in all three data sets, the method in the present invention achieves the highest values ​​in these three indicators. These results show that the method in the present invention has superior performance in the task of semantic segmentation of remote sensing images, significantly better than other existing pre-training methods.

[0145] From the results of the above simulation experiments, it can be concluded that the method in the present invention performs well in the remote sensing image semantic segmentation task. Through adaptive contrast view generation, the method in the present invention can more effectively improve the feature extraction ability and segmentation accuracy of the model, and has high practical value and technical advantages.

Claims

1. An adaptive contrast view generation method for semantic segmentation of remote sensing images, characterized in that: The following steps are included: comparative learning view construction method, Step 1: Preprocessing of original remote sensing images: Preprocess the original remote sensing images to obtain a pre-training set; Step 2: Construct a comparative learning view: Step 2a: Optimize the hue transformation parameters of the sample x in the pre-training set to obtain the sample after hue transformation. is the hue value of the optimized view, HSV -1 Indicates converting samples from the HSV color space to the original color space; Step 2b: Optimize the sample in step 2a The saturation transformation parameters are used to obtain the samples after hue transformation and saturation transformation. is the saturation of the optimized view; Step 2c: Optimize the sample in step 2b The brightness transformation parameters are used to obtain the samples after hue transformation, saturation transformation and brightness transformation. Step 2d: Optimize the sample in step 2c The contrast transformation parameters are used to obtain the samples after hue transformation, saturation transformation, brightness transformation and contrast transformation. Step 2e: Optimize the sample in step 2d The rotation transformation parameters are used to obtain the contrast learning view after hue transformation, saturation transformation, brightness transformation, contrast transformation and rotation transformation. Step 3: Compare the learning view in step 2e Input into a pre-built self-supervised contrastive learning network for training; the self-supervised contrastive learning network is composed of a backbone network and a multi-layer perceptron, wherein the backbone network adopts a ResNet-18 structure; Step 4: Repeat steps 2 and 3 until a preset number of iterations is reached; at the same time, a trained self-supervised contrastive learning network is obtained.

2. The method for generating adaptive contrast views for semantic segmentation of remote sensing images according to claim 1, characterized in that: The specific process of step 2a is as follows: Convert the sample x to HSV color space, y h =HSV(x) represents the hue value of sample x, which ranges from [-π,π]; randomly initialize the hue transformation parameter δ h , perform random hue transformation on the sample, as shown in formula (A): h (y h )=y h +δ h (A); Where: t h (y h ) represents the hue value after random transformation; The samples after random tone transformation are sent to the self-supervised contrastive learning network to calculate the contrast loss, as shown in formula (B): Where L is the NT-Xent loss function; z i =f(x i ), f(x i ) is the sample x i The self-supervised contrast network is constructed; τ is the temperature coefficient, which is used to control the sharpness of the similarity distribution between views, and Ι [k≠i] is the indicator function, used to exclude its own similarity; z i For sample x i Features of the first view; z k is the characteristic of other samples in the batch; sim(z i ,z j ) is used to calculate the sample x i Cosine similarity of features of two views; z j For sample x i Features of the second view; Using the FGSM method, the tone transformation parameters are adjusted according to the gradient of the contrast loss, as shown in formula (C): in, is the adjusted hue transformation parameter, α represents the perturbation intensity, HSV -1 Indicates converting samples from the HSV color space to the original color space; t h ,t s ,t b ,t c and t g They are hue transformation, saturation transformation, brightness transformation, contrast transformation and rotation transformation; The adjusted hue conversion parameters The hue value y of sample x h Add together to get the transformed hue value And limit it to the range of [-π,π], as shown in formula (D): Get the sample after hue transformation 3. The method for generating adaptive contrast views for semantic segmentation of remote sensing images according to claim 1, characterized in that: The specific process of step 2b is as follows: Convert to HSV color space, Represents the saturation value, which ranges from [0,1]; randomly initializes the saturation transformation parameter δ s , perform random saturation transformation on the sample, as shown in formula (E): t s (and s )=and s ·δ s (AND); Where: t s (y s ) is the saturation value after randomly transforming the saturation value of the input sample; The samples after hue transformation and random saturation transformation are sent to the self-supervised contrastive learning network to calculate the contrast loss. Then, the FGSM method is used to adjust the saturation transformation parameters according to the gradient of the contrast loss, as shown in formula (F): in, is the adjusted saturation transform parameter; The adjusted saturation conversion parameter The hue value y of the input sample s Multiply to get the transformed saturation value And limit it to the range [0,1], as shown in formula (G): Get the sample after saturation transformation and hue transformation 4. The method for generating adaptive contrast views for semantic segmentation of remote sensing images according to claim 3, characterized in that: The specific process of step 2c is as follows: The self-supervised contrast learning network is sent to calculate the contrast loss; then the FGSM method is used to adjust the brightness transformation parameter δ according to the gradient of the contrast loss. b , as shown in formula (H): in, is the adjusted brightness transformation parameter. The adjusted brightness transformation parameters With input sample Add together to get the sample after hue, saturation and brightness transformation And limit it to the range [0,1], as shown in formula (I):

5. The method for generating adaptive contrast views for semantic segmentation of remote sensing images according to claim 4, characterized in that: The specific process of step 2d is as follows: Send it to the self-supervised contrast learning network to calculate the contrast loss; then use the FGSM method to adjust the contrast transformation parameter δ according to the gradient of the contrast loss c , as shown in formula (J): in, is the adjusted brightness transformation parameter. Finally, the adjusted contrast transformation parameters With input sample Multiply them together to get the samples used for the transformation And limit it to the range [0,1], as shown in formula (K):

6. The method for generating adaptive contrast views for semantic segmentation of remote sensing images according to claim 5, characterized in that: The specific process of step 2e is as follows: Send it to the self-supervised contrast learning network to calculate the contrast loss; then use the FGSM method to adjust the rotation transformation parameter θ according to the gradient of the contrast loss, as shown in formula (L): in, is the adjusted rotation transformation parameter. Finally, the adjusted rotation transformation parameters are applied to the input sample. On the top, we get the final transformed view sample As shown in formula (M):

7. The method for generating adaptive contrast views for semantic segmentation of remote sensing images according to claim 6, characterized in that: The specific process of step one is as follows: cutting the original remote sensing image into small images, and dividing the small images into three parts, one part is a pre-training set, the second part is a fine-tuning set, and the third part is a simulation test set, wherein the fine-tuning set is selected from the pre-training set.

8. A method for semantic segmentation of remote sensing images, characterized in that: The method for generating adaptive contrast views for semantic segmentation of remote sensing images according to any one of claims 1 to 7 further comprises the following steps after step 4: Step 5: Backbone network extraction and integration: Extracting a backbone network from the trained self-supervised contrastive learning network, and embedding the backbone network into a feature extractor in a remote sensing image semantic segmentation network, wherein the remote sensing image semantic segmentation network also includes a decoding module; the feature extractor is used to extract high-dimensional features in the remote sensing image, and the decoding module is used to map the high-dimensional features back to the spatial resolution of the input image and generate a classification prediction for each pixel; Step 6: Remote sensing image semantic segmentation network fine-tuning: The image samples in the fine-tuning set in step 1 are input into the remote sensing image semantic segmentation network, and the cross entropy loss function is used to optimize the parameters of the remote sensing image semantic segmentation network, as shown in formula (R): Get the fine-tuned remote sensing image semantic segmentation network; Among them, N represents the number of pixels in the input image, M represents the number of ground object types, and y ic Indicates whether pixel i in the label belongs to category c, p ic Represents the probability that the network i outputs pixel i belonging to category c; Step 7: Input the image samples in the simulation test set in step 1 into the fine-tuned remote sensing image semantic segmentation network in step 6, classify each pixel, and generate a complete semantic segmentation map.

Images