A method and system for extracting impervious surfaces from remote sensing images

Abstract

The present invention discloses a method and system for extracting impervious surfaces from remote sensing images. The method comprises: obtaining a semantic segmentation dataset and preprocessing it to obtain preprocessed data; constructing a semantic segmentation model based on the preprocessed data; coupling a positive sample-background sample learning algorithm with the semantic segmentation model to obtain a migration algorithm model; and performing domain matching on the image to be tested based on the migration algorithm model to complete impervious surface extraction. The system comprises: a preprocessing module, a basic model construction module, a coupling module, and an extraction module. By using the present invention, high-precision urban impervious surface extraction can be achieved using training with a small sample dataset. As a method and system for extracting impervious surfaces from remote sensing images, the present invention can be widely applied in the field of image processing.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method and system for extracting impervious surfaces from remote sensing images. Background Art

[0002] In urban remote sensing, artificial surfaces such as urban expressways, buildings, and streets, where surface water cannot quickly penetrate below the surface, are often referred to as impervious surfaces. With rapid urbanization, the rapid growth of impervious surfaces is placing unprecedented pressure on the urban environment, such as the urban heat island effect and urban waterlogging. However, due to the diversity and richness of surface information, extracting large-scale impervious surface information from high-resolution remote sensing imagery, primarily through deep learning, is challenging. Current impervious surface extraction methods suffer from the high data dependency of deep network models and insufficient generalization capabilities. Summary of the Invention

[0003] In order to solve the above technical problems, the purpose of the present invention is to provide a method and system for extracting impervious surfaces from remote sensing images, which can achieve high-precision urban impervious surface extraction under small sample data set training, thereby providing strong support for urban environmental decision-making.

[0004] The first technical solution adopted by the present invention is: a method for extracting impervious surfaces from remote sensing images, comprising the following steps:

[0005] Obtain a semantic segmentation dataset and preprocess it to obtain preprocessed data;

[0006] Build a semantic segmentation model based on preprocessed data;

[0007] The positive-background learning algorithm is coupled with the semantic segmentation model to obtain a transfer algorithm model.

[0008] Based on the migration algorithm model, domain matching is performed on the image to be tested to complete the impervious surface extraction.

[0009] Furthermore, it also includes:

[0010] The migration algorithm model is evaluated using F1-score, pixel accuracy and intersection-over-union ratio as evaluation indicators.

[0011] Furthermore, the step of obtaining a semantic segmentation dataset and preprocessing it to obtain preprocessed data specifically includes:

[0012] Obtain the semantic segmentation dataset and perform normalization processing to obtain normalized data;

[0013] Defining impervious surface features and permeable surface features based on the normalized data;

[0014] The normalized data is subjected to image cropping, screening and histogram matching to obtain preprocessed data.

[0015] Furthermore, the step of constructing a semantic segmentation model based on the preprocessed data specifically includes:

[0016] Divide the preprocessed data into training and test sets;

[0017] Construct a pre-training model based on the U-Net network, wherein the pre-training model includes a convolutional layer, a BN layer, and a maximum pooling layer;

[0018] Train the pre-trained model based on the training set to obtain a semantic segmentation model;

[0019] The semantic segmentation model is tested based on the test set. If the prediction accuracy is greater than the preset value, the trained semantic segmentation model is output.

[0020] Furthermore, the step of training the pre-trained model based on the training set to obtain the semantic segmentation model includes a contraction phase and an expansion phase, which specifically include:

[0021] In the contraction phase, the images in the training set are processed by convolution and function fitting using two convolutional layers with ReLU activation functions. The BN layer is used to normalize the symmetric coefficient distribution of the convolutional layer, and the image is downsampled by the maximum pooling layer.

[0022] In the expansion phase, the images in the training set are deconvolved and feature superimposed. Image semantic information is extracted through multiple convolutional layers. The category information of each pixel in the image is extracted through a convolutional layer with a Sigmoid activation function to obtain the extracted information.

[0023] The extracted information is combined with the true labels in the training set to complete the training and obtain the semantic segmentation model.

[0024] Furthermore, the step of coupling the positive sample-background sample learning algorithm with the semantic segmentation model to obtain a migration algorithm model specifically includes:

[0025] The impervious surface feature is defined as a positive sample and background samples are collected in the image to obtain sample data;

[0026] The background samples include positive samples and negative samples;

[0027] Send the sample data into the semantic segmentation model to simulate the nonlinear segmentation function between positive samples and negative samples;

[0028] Based on the pre-trained classifier, the probability value of each pixel classification of the image is output, and the probability value is corrected using the correction coefficient to obtain the predicted value;

[0029] The error between the predicted value and the true value is calculated through the cross entropy loss function after constraint regularization, and gradient backpropagation is performed to optimize the domain matching parameters and correction coefficients until the loss function converges to obtain the migration algorithm model.

[0030] Furthermore, the loss function formula is expressed as follows:

[0031]

[0032] In the above formula, n represents the number of samples, i represents the number of samples, and s i represents the label of the sample, x i represents the predictor variable, λ represents the L2 regularization parameter, f(x,ω) represents the positive-negative sample segmentation function, c represents the correction coefficient, P max Indicates the theoretical maximum probability value belonging to the positive class.

[0033] The second technical solution adopted by the present invention is: a remote sensing image impervious surface extraction system, comprising:

[0034] The preprocessing module is used to obtain the semantic segmentation dataset and perform preprocessing to obtain preprocessed data;

[0035] Basic model building module, which builds a semantic segmentation model based on preprocessed data;

[0036] A coupling module is used to couple the positive sample-background sample learning algorithm with the semantic segmentation model to obtain a transfer algorithm model;

[0037] The extraction module performs domain matching on the image to be tested based on the migration algorithm model to complete the extraction of impervious surfaces.

[0038] The beneficial effects of the method and system of the present invention are as follows: by coupling the positive sample-background sample learning algorithm with the U-Net semantic segmentation model, the present invention regards impervious surface samples as positive samples, while background samples are randomly selected and the label information is unknown, thereby converting the extraction of impervious surfaces into a classification problem, reducing the cost of obtaining samples while improving accuracy. Based on transfer learning, the pre-trained model is fine-tuned using a small sample data set to achieve high-precision urban impervious surface extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flowchart of the steps of a method for extracting impervious surfaces from remote sensing images of the present invention;

[0040] Figure 2 This is a flowchart illustrating the positive sample-background sample learning migration algorithm according to a specific embodiment of the present invention;

[0041] Figure 3 This is a structural block diagram of a remote sensing image impervious surface extraction system of the present invention. DETAILED DESCRIPTION

[0042] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The step numbers in the following embodiments are provided for ease of description only and do not limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted based on the understanding of those skilled in the art.

[0043] like Figure 1 As shown, the present invention provides a method for extracting impervious surfaces from remote sensing images, the method comprising the following steps:

[0044] S1. Obtain the semantic segmentation dataset and preprocess it to obtain preprocessed data;

[0045] S1.1. Obtain a semantic segmentation dataset and perform normalization processing to obtain normalized data;

[0046] Specifically, the semantic segmentation dataset includes the Potsdam dataset and the EI Cerrito dataset. Due to the differences in units and dimensions between different datasets, in order to eliminate the mutual influence of different dimensions between indicators, the present invention normalizes the data to the maximum and minimum values.

[0047] S1.2. Define impervious surface characteristics and permeable surface characteristics based on the normalized data;

[0048] Specifically, both datasets used in this paper contain multiple feature types, necessitating a unified definition of impervious surfaces across the datasets. This paper ultimately selected buildings and roads as impervious surfaces, while defining low shrubs, trees, and other (difficult-to-identify) features as pervious surfaces.

[0049] S1.3. Perform image cropping, screening, and histogram matching on the normalized data to obtain preprocessed data.

[0050] Specifically, to unify the image sizes of each dataset, the images of the three datasets were cropped to 512×512. Data that did not contain any positive or negative samples were removed to form the required dataset. Due to differences in remote sensing image acquisition time, acquisition equipment, and earth curvature, the data distribution between the training data and the actual application data differs, and directly applying the training model cannot achieve the desired results. Histogram matching is often used as a preprocessing operation for multi-time period remote sensing image stitching and dynamic change detection to reduce the influence of differences in solar altitude angle, earth curvature, atmospheric refraction, etc. in remote sensing images of different time periods. This paper uses the Potsdam dataset as a target reference to modify the multi-band histogram of the EI Cerrito image.

[0051] S2. Build a semantic segmentation model based on preprocessed data;

[0052] Specifically, based on the Potsdam dataset, a dataset containing both positive and negative samples was obtained, and a pre-trained model based on the U-Net semantic segmentation model was constructed. By continuously stacking convolutional layers and skip-linking encoding and decoding blocks, the texture information from shallow convolution layers was integrated with the semantic information from deep convolution layers. The trained model was then saved for later knowledge transfer.

[0053] S2.1. Divide the preprocessed data into training and test sets;

[0054] S2.2. Build a pre-trained model based on the U-Net network, wherein the pre-trained model includes a convolutional layer, a batch normalization layer, and a maximum pooling layer;

[0055] S2.3. Train the pre-trained model based on the training set to obtain a semantic segmentation model;

[0056] Specifically, the training includes a contraction phase and an expansion phase:

[0057] In the contraction phase, the images in the training set are processed by convolution and function fitting using two convolutional layers with ReLU activation functions. The BN layer is used to normalize the symmetric coefficient distribution of the convolutional layer, and the image is downsampled by the maximum pooling layer.

[0058] In the expansion phase, the images in the training set are deconvolved and feature superimposed. Image semantic information is extracted through multiple convolutional layers. The category information of each pixel in the image is extracted through a convolutional layer with a Sigmoid activation function to obtain the extracted information.

[0059] The extracted information is combined with the true labels in the training set to complete the training and obtain the semantic segmentation model.

[0060] The present invention obtains a pre-trained model based on the classic U-Net semantic segmentation model, and connects context information through a skip connection structure, including a contraction stage and an expansion segment. In the contraction stage, the input image passes through two layers of convolutional layers with a window of 3×3 and "same" padding to keep the size of the image unchanged after convolution. Each convolutional layer is equipped with a Relu activation function to fit a highly nonlinear function, and combined with the BN (BatchNormal) layer, the symmetric coefficient distribution of the convolutional layer is normalized to speed up network training and convergence, while preventing gradient explosion or disappearance, effectively alleviating the problem of model overfitting. Finally, the image is downsampled through the MaxPooling maximum pooling operation, and the image size is converted from 512×512 and 256×256 to 128×128. At the same time, the number of model channels is doubled. In the expansion segment, the input data of the previous layer is first deconvolved to restore the corresponding image size in the contraction stage, and is deeply superimposed with the features of the contraction stage. It then passes through three layers of convolutional networks with a window size of 3×3 to continuously reduce the depth of the network and further extract the image semantic information. Finally, a convolutional layer with a window size of 1×1 is used in combination with the Sigmoid activation function to extract the category information of each pixel in the image.

[0061] S2.4. Test the semantic segmentation model based on the test set, determine that the prediction accuracy is greater than the preset value, and output the trained semantic segmentation model.

[0062] S3. Couple the positive sample-background sample learning algorithm with the semantic segmentation model to obtain a transfer algorithm model;

[0063] Specifically, refer to Figure 2 , obtain positive sample-background sample data from the image to be predicted (EI Cerrito dataset), and couple the positive sample-background sample learning (PBLC) algorithm with the semantic segmentation model obtained in step S2 to construct a PBLC_U-Net migration algorithm model, perform domain matching on the image to be predicted, and extract the impervious surface of the image based on the positive sample-background sample data.

[0064] S3.1. Define the impervious surface feature as a positive sample and randomly collect background samples from the image to be predicted to obtain sample data;

[0065] The background samples include positive samples and negative samples, but the category information is unknown;

[0066] S3.2. Send the sample data into the semantic segmentation model to simulate the nonlinear segmentation function between positive samples and negative samples;

[0067] S3.3. Output the probability value of each pixel classification of the image based on the pre-trained classifier, and use the correction coefficient to correct the probability value to obtain the predicted value;

[0068] S3.4. Calculate the error between the predicted value and the true value through the cross-entropy loss function after constraint regularization, perform gradient backpropagation, optimize the domain matching parameters and correction coefficients until the loss function converges, and obtain the migration algorithm model.

[0069] Specifically, the PBLC algorithm is described as follows: In a one-class classification problem, the target sample is defined as a positive sample (s = 1), and the unknown sample is defined as a background sample (s = 0). A correction coefficient c, which is the ratio of labeled positive samples to all positive samples, is introduced, as shown in Formula (1), to correct the error caused by pseudo-negative samples and solve the probability of belonging to the positive class under the condition of feature x, that is, Pr(y = 1|x), where n1 is the number of labeled positive samples and n0 is the number of positive samples in the background samples.

[0070]

[0071] The relationship between the positive-negative sample model and the positive-background sample model is shown in formula (2), where x is the feature information, Pr(y=1|x) is the probability model obtained by training the classifier under positive-negative samples, Pr(s=1|x, η=1) is the probability model obtained by training the classifier under positive-background samples, c is the ratio of the number of labeled positive samples to the total number of positive samples, and η is the case-control sampling method:

[0072]

[0073] It can be derived as formula (3):

[0074]

[0075] At the same time, observing formula (3), we can see that when Pr(y=1|x) is infinitely close to 1, Pr(s=1|x, η=1) is also infinitely close to c. Since the probability prediction value of the significant positive sample Pr(y=1|x) is close to 1, the c value can be estimated by the following formula (4), where k is the cardinality of the significant positive sample dataset PP:

[0076]

[0077] Let Pr(y=1|x)=f(x,w) and Pr(s=1|x,η=1)=g(x,β), where f and g are functions, and w and β are model parameters. According to maximum likelihood estimation, the optimal model parameter β is solved by minimizing the cross entropy loss function. The loss function can be expressed as formula (5), where n is the number of training samples and i is 1, 2, 3, ..., n:

[0078]

[0079] Therefore, according to formula (3), the cross entropy loss function formula (5) can also be expressed as:

[0080]

[0081] Based on the positive sample-background sample data, the optimal parameter w is inferred by minimizing the loss function. Assuming that the prior constraint c is known, the model can accurately estimate the probability Pr(y=1|x). However, in real scenarios, the prior constraint value c is often uncertain. In the PBLC algorithm, based on the theoretical basis of formula (4), we assume that the theoretical maximum probability value of a typical positive sample is Pmax, and let the maximum value of Pr(y=1|x) be infinitely close to Pmax, that is, satisfy the previous conditions of formula (4). Then the cross loss function with the regularization term added is defined as formula (7), where λ is the model regularization parameter:

[0082]

[0083] According to the above algorithm, the present invention marks the pixels of the impermeable surface as positive samples (x, s = 1), where x is the image feature RGB, s = 1 is marked as a positive sample, and randomly collects unlabeled background samples in the image, denoted as (x, s = 0), where the background sample s = 0 includes both positive samples (x, y = 1) and negative samples (x, y = 0), and the label category is unknown. Here, x is also an image feature. In order to construct a PBLC_U-Net-based migration model, the characteristic value x of the positive-background sample is first converted to x' = x*a + b through a linear layer (domain matching), and sent to the pre-trained U-Net network model to simulate the highly nonlinear segmentation function between positive samples and negative samples. The classifier outputs the probability value Pr (y = 1 | x) of each pixel classification, denoted as f. According to formula (3) in the PBLC algorithm, there is a certain relationship between the output probability of the positive-background sample and the output probability of the positive-negative sample, that is, The probability value of the positive-background sample is then calculated as g = Pr(s = 1|x). Finally, the error between the predicted value and the true value is calculated using the regularized cross-entropy loss function, and gradient backpropagation is performed to continuously optimize the domain matching parameters (a and b) and the correction coefficient (c) until the loss function converges, obtaining the final remote sensing image impervious surface segmentation output.

[0084] In an image transfer network based on the PBLC algorithm, the positive sample data of interest and the background sample data are first fed into the x'=x*a+b network layer. The gradient backpropagation algorithm automatically optimizes the a and b parameters to achieve image domain matching between different datasets. The U-Net pre-trained model is then substituted in, the convolutional layer parameters are fixed, and the texture and semantic information of the image are extracted based on the knowledge structure of the pre-trained model. The BatchNormal layer is fine-tuned, and finally the output result of the entire image is obtained through a convolution kernel with a window size of 1*1. This structure is the coupling model PBLC_U-Net of the PBLC algorithm of the present invention and the pre-trained U-Net model.

[0085] S4. Perform domain matching on the image to be tested based on the migration algorithm model to complete the extraction of impervious surfaces.

[0086] S5. Evaluate the migration algorithm model using F1-score, pixel accuracy, and intersection-over-union ratio as evaluation indicators.

[0087] Specifically, the calculation formula of the F1-score indicator is as follows:

[0088]

[0089]

[0090]

[0091] Among them, TP (True Positive) is true positive, which means that the predicted value is consistent with the true value and both are positive samples; FP (False Positive) is false positive, which means that the predicted value is a positive sample and the true value is a negative sample; FN (False Negative) is false negative, which means that the predicted value is a negative sample and the true value is a positive sample; TN (True Negative) is true negative, which means that the predicted value is a negative sample and the true value is also a negative sample.

[0092] The calculation formula for pixel accuracy is as follows:

[0093]

[0094] The calculation formula of intersection-over-union ratio is as follows:

[0095]

[0096] According to the analysis of specific experimental results, it can be seen that the impervious surface extraction boundary of PBLC_U-Net is more accurate, there are fewer misclassifications, and the overall extraction results are closer to the true value.

[0097] The present invention is scalable for identifying other surface types. The present invention focuses on extracting a single type of surface type, and in principle, the present invention can be extended to other surface types such as vegetation, bare soil, etc.

[0098] like Figure 3 As shown, a remote sensing image impervious surface extraction system includes:

[0099] The preprocessing module is used to obtain the semantic segmentation dataset and perform preprocessing to obtain preprocessed data;

[0100] Basic model building module, which builds a semantic segmentation model based on preprocessed data;

[0101] A coupling module is used to couple the positive sample-background sample learning algorithm with the semantic segmentation model to obtain a transfer algorithm model;

[0102] The extraction module performs domain matching on the image to be tested based on the migration algorithm model to complete the extraction of impervious surfaces.

[0103] The contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0104] A device for extracting impervious surfaces from remote sensing images:

[0105] at least one processor;

[0106] at least one memory for storing at least one program;

[0107] When the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned method for extracting impervious surfaces from remote sensing images.

[0108] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0109] A storage medium storing processor-executable instructions, characterized in that the processor-executable instructions are used to implement the above-mentioned method for extracting impervious surfaces from remote sensing images when executed by the processor.

[0110] The contents of the above method embodiments are all applicable to the present storage medium embodiment. The functions specifically implemented by the present storage medium embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0111] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A method for extracting impervious surfaces from remote sensing images, characterized in that: The following steps are involved: Obtain a semantic segmentation dataset and preprocess it to obtain preprocessed data; Build a semantic segmentation model based on preprocessed data; The positive-background learning algorithm is coupled with the semantic segmentation model to obtain a transfer algorithm model. Based on the migration algorithm model, domain matching is performed on the image to be tested to complete the extraction of impervious surfaces; The step of coupling the positive sample-background sample learning algorithm with the semantic segmentation model to obtain a migration algorithm model specifically includes: The impervious surface feature is defined as a positive sample and background samples are randomly collected in the image to obtain sample data; The background samples include positive samples and negative samples; Send the sample data into the semantic segmentation model to simulate the nonlinear segmentation function between positive samples and negative samples; Based on the pre-trained classifier, the probability value of each pixel classification of the image is output, and the probability value is corrected using the correction coefficient to obtain the predicted value; The error between the predicted value and the true value is calculated through the cross entropy loss function after constraint regularization, and gradient backpropagation is performed to optimize the domain matching parameters and correction coefficients until the loss function converges to obtain the migration algorithm model; The positive sample-background sample learning algorithm is specifically as follows: The target sample is defined as a positive sample, and the unknown sample is defined as a background sample. A correction coefficient is introduced, and the expression of the correction coefficient is expressed as follows: Among them, n1 is the number of labeled positive samples, and n0 is the number of positive samples in the background samples; The loss function formula is expressed as follows: In the above formula, n represents the number of samples, i represents the number of samples, and s i represents the label of the sample, x i represents the predictor variable, λ represents the L2 regularization parameter, f(x,ω) represents the positive-negative sample segmentation function, c represents the correction coefficient, P max Indicates the theoretical maximum probability value belonging to the positive category; Based on the positive sample-background sample data, the optimal parameters are inferred by minimizing the loss function.

2. The method for extracting impervious surfaces from remote sensing images according to claim 1, characterized in that: Also includes: The migration algorithm model is evaluated using F1-score, pixel accuracy and intersection-over-union ratio as evaluation indicators.

3. The method for extracting impervious surfaces from remote sensing images according to claim 2, characterized in that: The step of obtaining a semantic segmentation dataset and preprocessing it to obtain preprocessed data specifically includes: Obtain the semantic segmentation dataset and perform normalization processing to obtain normalized data; Defining impervious surface features and permeable surface features based on the normalized data; The normalized data is subjected to image cropping, screening and histogram matching to obtain preprocessed data.

4. The method for extracting impervious surfaces from remote sensing images according to claim 3, characterized in that: The step of building a semantic segmentation model based on preprocessed data specifically includes: Divide the preprocessed data into training and test sets; Construct a pre-training model based on the U-Net network, wherein the pre-training model includes a convolutional layer, a BN layer, and a maximum pooling layer; Train the pre-trained model based on the training set to obtain a semantic segmentation model; The semantic segmentation model is tested based on the test set. If the prediction accuracy is greater than the preset value, the trained semantic segmentation model is output.

5. The method for extracting impervious surfaces from remote sensing images according to claim 4, characterized in that: The step of training the pre-trained model based on the training set to obtain the semantic segmentation model includes a contraction phase and an expansion phase, which specifically includes: In the contraction phase, the images in the training set are processed by convolution and function fitting using two convolutional layers with ReLU activation functions. The BN layer is used to normalize the symmetric coefficient distribution of the convolutional layer, and the image is downsampled by the maximum pooling layer. In the expansion phase, the images in the training set are deconvolved and feature superimposed. Image semantic information is extracted through multiple convolutional layers. The category information of each pixel in the image is extracted through a convolutional layer with a Sigmoid activation function to obtain the extracted information. The extracted information is combined with the true labels in the training set to complete the training and obtain the semantic segmentation model.

6. A remote sensing image impervious surface extraction system, characterized in that: The method for extracting impervious surfaces from remote sensing images according to claim 1 comprises: The preprocessing module is used to obtain the semantic segmentation dataset and perform preprocessing to obtain preprocessed data; Basic model building module, which builds a semantic segmentation model based on preprocessed data; A coupling module is used to couple the positive sample-background sample learning algorithm with the semantic segmentation model to obtain a transfer algorithm model; The extraction module performs domain matching on the image to be tested based on the migration algorithm model to complete the extraction of impervious surfaces.

Images