A glacier change remote sensing detection method based on deep twin neural network

Abstract

The application is suitable for the technical field of remote sensing image processing, and provides a glacier change remote sensing detection method based on a deep twin neural network. Mainly comprising the following steps: step A, pre-processing of double-time-phase glacier remote sensing images to obtain training data for glacier change detection; step B, inputting the double-time-phase training data into the deep twin neural network for training to obtain the trained deep twin neural network; step C, inputting double-time-phase glacier remote sensing image data into the trained deep twin neural network to obtain a glacier change detection binary prediction image, vectorizing the obtained glacier change detection binary prediction image to obtain a glacier change area vector diagram. In the application, the spatial features of the subtle change area can be effectively enhanced, the glacier remote sensing image change detection map is obtained, and the glacier change detection precision is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of remote sensing image change detection, and particularly to a glacier change remote sensing detection method based on a deep twin neural network. BACKGROUND

[0002] In recent years, with the continuous warming of the global climate, the melting of glaciers is accelerating, and the position of glaciers is constantly changing. Glaciers are an important part of the cryosphere, and they are not only one of the important driving factors of climate change, but also reflect the recorder and early warning of climate change, so it is of great significance to quickly and accurately detect the glacier change area.

[0003] At present, the glacier change detection methods based on remote sensing images mainly include simple algebraic operation method, image classification method and structural feature analysis method.

[0004] The change detection method based on simple algebraic operation can be divided into image interpolation method, image ratio method, image regression analysis method and variable vector analysis. The basic principle of the image interpolation method is to perform geometric registration on two images obtained at different times, and then subtract the gray values of the corresponding pixels in the images to obtain a difference image to represent the changes in the target area between the selected two times. The image ratio method is to divide the corresponding pixels of the registered images at different times to obtain a ratio image. The advantage of the change detection technology based on algebraic operation is relatively simple and direct, and the key is to determine the appropriate threshold. Due to the difficulty in selecting the threshold, the current method is to determine the change threshold by human-computer interaction. The disadvantage of this method is that it is difficult to determine the type of change and cannot describe the change information. The regression analysis method first assumes that the two images are linearly related, that is, most of the pixels in the two images do not change much. This method uses the least square method for regression analysis, and then subtracts the predicted value calculated by the regression equation from the true value of the image to obtain the regression difference image of the two images, which can reflect the land cover change information. The regression analysis method solves the difference between the mean and variance of the pixels in the different time images, and the processed remote sensing image data is similar to the relative radiation correction to some extent, so that the adverse effects caused by different atmospheric conditions and solar elevation angles in the multi-temporal image data can be reduced. However, this method requires an accurate regression equation and the selection of appropriate bands, and the precision is not high in practical application. The change vector analysis method first performs difference operation on two images at different times to obtain the change value of each pixel, called change vector. The strength of the change is represented by the Euclidean distance of the change vector, and the content of the change is represented by the direction of the change vector. The change vector analysis method can use more or even all the bands to detect the change pixels, so as to avoid the incomplete information caused by single band comparison, and the change type information can be provided through the direction of the change vector, but with the increase of the number of bands, it is more difficult to determine the change threshold.

[0005] The change detection method based on image classification mainly includes post-classification comparison and multi-temporal image simultaneous classification. The basic principle of post-classification comparison method is to classify two different temporal images separately, and then compare pixel by pixel in the classified region to determine the location and type of change information. The basic principle of multi-temporal image simultaneous classification method is to put two or more temporal remote sensing image data in a database for simultaneous classification. The statistical quantity of the data of the changed class, such as standard deviation, will be large, and the standard deviation of the data of the unchanged class will be small, so the change of the class can be distinguished by comparing the statistical quantity. This method can provide the type information of the change, and can reduce the influence of external factors such as atmosphere on the change detection. The disadvantage is that it needs to select enough high-precision learning samples, which is very difficult for historical data. In addition, the performance of the change detection method based on classification is greatly affected by the classification result.

[0006] The change detection method based on feature structure analysis refers to using different algorithms to extract feature information (such as edge, shape, contour, and texture) from the initial image, then comprehensively analyzing the feature information, and finally obtaining the change detection result image. The extraction and analysis of features are the difficulties of this method. This method needs to extract the primitive features of the image, and uses the obtained edge, shape or texture primitive features for change detection, so the performance of the detection depends on the primitive feature extraction algorithm to a great extent. In summary, the traditional glacier change detection method of remote sensing image has certain shortcomings. At present, there are deep learning-based glacier extraction methods, but there are no deep learning-based glacier change detection methods. Using deep learning technology to realize the change detection of glaciers is an important direction for future development. Glaciers are indicators of climate change and important freshwater resources, so detecting the change of glaciers has valuable scientific value for studying the laws of climate and environmental evolution. Due to the slow change of glaciers and the difficulty of keeping consistent light when shooting double temporal images by satellite, the design of neural network combined with the task of glacier change detection still needs to be explored. SUMMARY

[0007] The technical problem to be solved by the glacier change remote sensing detection method based on deep twin neural network provided by the present application is to design a neural network combined with the task of glacier change detection, so that the network can optimize the model parameters for glacier change in the training process, and solve the problem of poor detection effect of the existing detection network.

[0008] Technical scheme: The glacier change remote sensing detection method based on deep twin neural network comprises the following steps:

[0009] Step A: Preprocessing the double temporal glacier remote sensing image to obtain training data and test data;

[0010] Step B: training the twin neural network model after inputting the training data into the twin neural network model, performing error back propagation using the glacier change detection label and the twin neural network model result image, and obtaining the trained twin neural network model;

[0011] Step C: inputting the test data into the trained twin neural network model, obtaining a glacier change detection binary prediction image, vectorizing the obtained glacier change detection binary prediction image, and obtaining a glacier change area vector image;

[0012] Step : geometric correction and radiation correction are performed on the remote sensing image to obtain corrected double-time-phase glacier remote sensing images (Fig. A4a, A4b);

[0013] Step : the corrected double-time-phase glacier remote sensing images are labeled in the region of interest to obtain labeled double-time-phase glacier remote sensing image label images (Fig. A5a, A5b), and the intersection of the two double-time-phase glacier remote sensing image labels is taken and inverted to obtain a glacier change area label image (Fig. A5c);

[0014] Step : the glacier change area label image is cropped (Fig. A6a, Fig. A6b), and the training data and test data are divided according to the ratio of 8:2;

[0015] Further, the decoding process in step B includes the following steps:

[0016] Step : input the training data in step A into the first layer of the convolutional layer of the encoder (where the convolutional kernel size of the convolutional layer is , and the number is 24), and perform maximum pooling downsampling to obtain a feature map 、 ;

[0017] Step : the feature map in step is further processed by the convolutional layer of the second layer of the encoder (where the convolutional kernel size of the convolutional layer is , the step size is , and the number is ) to generate a feature map ;

[0018] Step : the feature map in step is further processed by the convolutional layer of the third layer of the encoder (where the convolutional kernel size of the convolutional layer is , the step size is , the number is ) Generate feature maps ;

[0019] step :step The feature map After the maximum pooling, the convolution layer of the fourth layer of the encoder + the global information enhancement module (the convolution kernel size of the convolution layer is , the step size is , the number is ), generate feature maps ;

[0020] step :step The feature map After the maximum pooling layer, the convolution layer of the fifth layer of the encoder is used (the convolution kernel size of the convolution layer is , the step size is , the number is ) Generate feature maps ;

[0021] step : Feature map generated by the encoder First, the spatial resolution is improved by the "convolution + upsampling" operation of the decoder, and the feature map after the resolution is improved and the feature map generated by the encoder Splicing to get feature map 、 and the feature map generated by the encoder Splicing to get feature map , the feature map With feature map Subtract pixel by pixel and generate a change feature map after convolution upsampling ;

[0022] step : Feature map generated by the encoder First, the image resolution is improved by the "convolution + upsampling" operation of the decoder, and the feature map after the resolution is improved and the feature map generated by the encoder Splicing to get feature map , and the feature map generated by the encoder Splicing to get feature map , the feature map With feature map Subtract pixel by pixel and generate a change feature map after convolution upsampling ;

[0023] step : Feature map generated by the encoder First, the image resolution is improved by the "convolution + upsampling" operation of the decoder, and the feature map after the resolution is improved and the feature map generated by the encoder Splicing to get feature map , and the feature map generated by the encoder Splicing , the feature map With feature map Subtract pixel by pixel and generate a change feature map after convolution upsampling ;

[0024] step :The feature map , and After weighted summation (weighted formula is D = ,in is the weighting coefficient, which is automatically adjusted by the network model through back propagation. The weighting coefficient change curve is shown in Figure A6c). After the "convolution + upsampling" operation, the convolution kernel size is , the step size is , the number is , generating a binary map of glacier change areas ( Figure A6d );

[0025] step : For the binary map of glacier change areas obtained in step B9, the loss function is calculated using the binary label map of glacier change areas, and the network is converged through error back propagation to obtain the trained network model.

[0026] Further steps The global information enhancement module includes the following steps:

[0027] step :step The feature map After the maximum pooling layer, it passes through the fourth layer of the encoder, and its convolution kernel size is , the step size is , the number is The feature map generated after the convolution layer is passed through a After convolution, use Convolution reduces the number of channels to 80, and then adjusts the feature map to the data format required for calculating self-attention and inputs it into the self-attention network model. The number of self-attention repetitions is 2. When calculating self-attention, each token only calculates attention to the pixels at the same position as other tokens. After output, adjust the feature map format to be consistent with the input format. After convolution, the output feature map is obtained , the width and height of the output feature map are consistent with the width and height of the feature map when input into the global information enhancement module, and the number of channels is ;

[0028] Furthermore, the loss function in step B10 includes the following steps:

[0029] step : Calculates the loss between the predicted result and the true label, where the change detection loss function is calculated using the "focalloss" function. The loss is then backpropagated to optimize the model parameters. The optimizer uses "adaptive moment estimation" optimization. The "focalloss" loss function is calculated as follows:

[0030] ;

[0031] In the formula represents the model prediction value, represents the label, α is a hyperparameter used to balance the importance of positive and negative samples, and is set to 0.9 in this invention. The hyperparameter is used to adjust the rate at which the sample weight is reduced, and is set to 2.0 in the present invention.

[0032] The overall loss is:

[0033] ;

[0034] Where m is the number of positive samples, n is the number of negative samples, and N is the total number of samples. , the loss function curve is as follows Figure 5d shown.

[0035] Furthermore, step C specifically includes the following steps:

[0036] Step C1: The dual-temporal glacier remote sensing images for change detection are simultaneously input into the trained change detection network model to obtain the binary prediction results of the glacier change detection area;

[0037] Step C2: Vectorize the binary prediction results of the glacier change detection area obtained in step C1 to obtain a vector map of the glacier change area. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] FIG1 is a flow chart of the present invention for detecting changes in glacier remote sensing images;

[0039] FIG2 is a diagram of a network model for detecting changes in glacier remote sensing images designed by the present invention;

[0040] FIG3 is a schematic diagram of the structure of the global information enhancement module designed in the present invention;

[0041] Fig. 4a is a glacier remote sensing image after geometric correction and radiation correction in the embodiment of the present application, the shooting time is 2015.08.01, the latitude and longitude is 78°55'N, 11°56'E, and the shooting satellite is: Landsat 8 ;

[0042] Fig. 4b is a glacier remote sensing image after geometric correction and radiation correction in the embodiment of the present application, the shooting time is 2020.07.27, the latitude and longitude is 78°55'N, 11°56'E, and the shooting satellite is: Landsat 8 ;

[0043] Fig. 5a is a glacier remote sensing image label in the previous phase in the embodiment of the present application;

[0044] Fig. 5b is a glacier remote sensing image label in the next phase in the embodiment of the present application;

[0045] Fig. 5c is a glacier remote sensing image change label binary graph in the embodiment of the present application;

[0046] Fig. 5d is a loss function change curve graph in the embodiment of the present application;

[0047] Fig. 6a is a glacier remote sensing image clipping in the previous phase in the embodiment of the present application;

[0048] Fig. 6b is a glacier remote sensing image clipping in the next phase in the embodiment of the present application;

[0049] Fig. 6c is a change line graph of α, β and γ in the embodiment of the present application;

[0050] Fig. 6d is a glacier remote sensing image prediction binary graph in the embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will further illustrate the present application with specific embodiments in the drawings of the embodiments of the present application. It should be understood that these embodiments are only used to illustrate the present application and not used to limit the scope of the present application. After reading the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0052] Figure 1 is a block diagram of a method according to an embodiment of the present invention, which includes the following steps: preprocessing dual-temporal glacier remote sensing images to obtain training data and test data; inputting the training data into a twin neural network model for training, performing error backpropagation using glacier change detection labels and the twin neural network model result graph to obtain a trained twin neural network model; inputting test data into the trained twin neural network model to obtain a binary prediction image for glacier change detection; and vectorizing the obtained binary prediction image for glacier change detection to obtain a vector map of the glacier change area. The method specifically includes the following steps:

[0053] Step A: Preprocess the dual-temporal glacier remote sensing images to obtain training data and test data;

[0054] step : The remote sensing image is geometrically and radiometrically corrected to obtain a corrected dual-temporal glacier remote sensing image (Figures A4a and A4b).

[0055] step : The corrected dual-temporal glacier remote sensing images are used to annotate the regions of interest to obtain labeled dual-temporal glacier remote sensing image labels (Figures A5a and A5b). The labels of the two dual-temporal glacier remote sensing images are then intersected and inverted to obtain a label map of the glacier change area (Figure A5c).

[0056] step : The glacier change area label map is cropped (Figure A6a, Figure A6b), and the training data and test data are divided into 8:2 ratios;

[0057] Step B: Inputting the training data into the twin neural network for training, performing error back propagation using the glacier change detection label and the twin neural network model result graph to obtain a trained twin neural network model;

[0058] step :The training data in step A is input into the first convolution layer of the encoder (where the convolution kernel size of the convolution layer is , the number is 24) and maximum pooling downsampling to obtain the feature map 、 ;

[0059] step :step The feature map After the maximum pooling, the convolution layer of the second layer of the encoder is used (the convolution kernel size of the convolution layer is , the step size is , the quantity is ) Generate feature maps ;

[0060] Step : Step The feature map is generated by using the convolutional layer of the third layer of the encoder after max-pooling (where the convolution kernel size of the convolutional layer is , the step is , and the number is ); ;

[0061] Step : Step The feature map is generated by using the convolutional layer of the fourth layer of the encoder + global information enhancement module after max-pooling (where the convolution kernel size of the convolutional layer is , the step is , and the number is ); ;

[0062] Step : Step The feature map is generated by using the convolutional layer of the fifth layer of the encoder after the max-pooling layer (where the convolution kernel size of the convolutional layer is , the step is , and the number is ); ;

[0063] Step : The feature map generated by the encoder is first improved in spatial resolution by the "convolution + up-sampling" operation of the decoder, and the feature map after the resolution is improved is spliced with the feature map generated by the encoder to obtain a feature map , is spliced with the feature map generated by the encoder to obtain a feature map , the feature map is pixel-by-pixel subtracted from the feature map , and the change feature map is generated after convolution and up-sampling;

[0064] Step : The feature map generated by the encoder is first improved in image resolution by the "convolution + up-sampling" operation of the decoder, and the feature map after the resolution is improved is spliced with the feature map generated by the encoder to obtain a feature map , is spliced with the feature map Splicing to get feature map , the feature map With feature map Subtract pixel by pixel and generate a change feature map after convolution upsampling ;

[0065] step : Feature map generated by the encoder First, the image resolution is improved by the "convolution + upsampling" operation of the decoder, and the feature map after the resolution is improved and the feature map generated by the encoder Splicing to get feature map , and the feature map generated by the encoder Splicing , the feature map With feature map Subtract pixel by pixel and generate a change feature map after convolution upsampling ;

[0066] step :The feature map , and After weighted summation (weighted formula is D = ,in is the weighting coefficient, which is automatically learned and adjusted by the network model through back propagation. The weighting coefficient change curve is shown in Figure A6c). After the "convolution + upsampling" operation, the convolution kernel size is , the step size is , the number is , generating a binary map of glacier change areas ( Figure A6d );

[0067] step : For the binary map of glacier change areas obtained in step B9, the loss function is calculated using the binary label map of glacier change areas, and the network is converged through error back propagation to obtain the trained network model.

[0068] Global information enhancement module such as Figure Three As shown, the following steps are included:

[0069] step :step The feature map After the maximum pooling layer, it passes through the fourth layer of the encoder, and its convolution kernel size is , the step size is , the number is The feature map generated after the convolution layer is passed through a After convolution, use Convolution reduces the number of channels to 80, and then adjusts the feature map to the data format required for calculating self-attention and inputs it into the self-attention network model. The number of self-attention repetitions is 2. When calculating self-attention, each token only calculates attention to the pixels at the same position as other tokens. After output, adjust the feature map format to be consistent with the input format. After convolution, the output feature map is obtained , the width and height of the output feature map are consistent with the width and height of the feature map when input into the global information enhancement module, and the number of channels is ;

[0070] Furthermore, the loss function in step B10 includes the following steps:

[0071] step : Calculates the loss between the predicted result and the true label, where the change detection loss function is calculated using the "focalloss" function. The loss is then backpropagated to optimize the model parameters. The optimizer uses "adaptive moment estimation" optimization. The "focalloss" loss function is calculated as follows:

[0072] ;

[0073] In the formula represents the model prediction value, represents the label, α is a hyperparameter used to balance the importance of positive and negative samples, and is set to 0.9 in this invention. The hyperparameter is used to adjust the rate at which the sample weight is reduced, and is set to 2.0 in the present invention.

[0074] The overall loss is:

[0075] ;

[0076] Where m is the number of positive samples, n is the number of negative samples, and N is the total number of samples. , the loss function curve is as follows Figure 5d shown.

[0077] Step C: Input the dual-temporal glacier remote sensing image that needs to be changed into the trained change detection network to obtain the binary prediction result of the glacier change detection area. Vectorize the binary prediction image of glacier change detection to obtain the binary prediction vector map of glacier change detection.

[0078] Step C1: The dual-temporal glacier remote sensing images that need to be changed are simultaneously input into the trained change detection network model to obtain the binary prediction results of the glacier change detection area.

[0079] Step C2: vectorize the binary prediction result of the glacier change detection region obtained in step C1 to obtain a glacier change region vector map.

Claims

1. A remote sensing detection method for glacier changes based on deep twin neural networks, characterized by The following steps are involved: Step A: Preprocess the dual-temporal glacier remote sensing images to obtain training data for glacier change detection; Step B: Input the dual-phase training data into the deep twin neural network, and obtain the trained deep twin neural network through the "convolution + global information enhancement module"; the step The specific steps include: step :The training data described in step A is input into the first convolution layer of the encoder and the maximum pooling downsampling to obtain the feature map 、 ; step :step The feature map After the maximum pooling, the convolution layer of the second layer of the encoder is used to generate the feature map ; step :step The feature map After the maximum pooling, the convolution layer of the third layer of the encoder is used to generate the feature map ; step :step The feature map After the maximum pooling, the convolution layer of the fourth layer of the encoder + the global information enhancement module are used to generate the feature map ; step :step The feature map After the maximum pooling layer, the convolution layer of the fifth layer of the encoder is used to generate the feature map ; step : Feature map generated by the encoder First, the spatial resolution is improved by the "convolution + upsampling" operation of the decoder, and the feature map after the resolution is improved is converted to and the feature map generated by the encoder Splicing to get feature map 、 and the feature map generated by the encoder Splicing to get feature map , the feature map With feature map Subtract pixel by pixel and generate a change feature map after convolution upsampling ; step : Feature map generated by the encoder First, the image resolution is improved through the "convolution + upsampling" operation of the decoder, and the feature map after the resolution is improved is converted to and the feature map generated by the encoder Splicing to get feature map , and the feature map generated by the encoder Splicing to get feature map , the feature map With feature map Subtract pixel by pixel and generate a change feature map after convolution upsampling ; step : Feature map generated by the encoder First, the image resolution is improved through the "convolution + upsampling" operation of the decoder, and the feature map after the resolution is improved is converted to and the feature map generated by the encoder Splicing to get feature map , and the feature map generated by the encoder Splicing , the feature map With feature map Subtract pixel by pixel and generate a change feature map after convolution upsampling ; step :The feature map , and After weighted summation, a "convolution + upsampling" operation is performed to generate a binary map of the glacier change area; step : For the binary map of glacier change areas obtained in step B9, the loss function is calculated using the binary label map of glacier change areas, and the network is converged through error back propagation to obtain the trained network model; Step C: Input the dual-temporal glacier remote sensing image data into the trained deep twin neural network to obtain a binary prediction image for glacier change detection. Vectorize the obtained binary prediction image for glacier change detection to obtain a vector map of the glacier change area.

2. The method for remote sensing glacier change detection based on deep twin neural network according to claim 1, characterized in that: The step A specifically includes the following steps: step : Performing geometric correction and radiometric correction on the remote sensing image to obtain a corrected dual-temporal glacier remote sensing image; step : The corrected dual-phase glacier remote sensing image is annotated with the region of interest to obtain the annotated dual-phase glacier remote sensing image label map. The labels of the two dual-phase glacier remote sensing images are intersected and inverted to obtain the glacier change area label map; step : The glacier change area label map is cropped and the training data and test data are divided into 8:2 ratios.

3. According to the claim The method for remote sensing detection of glacier changes based on deep twin neural networks is characterized in that: The loss function in step B includes the following steps: step : Calculate the loss between the predicted result and the true label, where the change detection loss function is calculated using the "focalloss" function. Then the loss is backpropagated to optimize the model parameters. The optimizer uses "adaptive moment estimation" optimization. The "focalloss" loss function calculation formula is as follows: ； In the formula represents the model prediction value, Represents the label, α is a hyperparameter used to balance the importance of positive and negative samples, and is set to 0.

9. It is a hyperparameter used to adjust the rate at which sample weights are reduced; The overall loss is: ； Where m is the number of positive samples, n is the number of negative samples, and N is the total number of samples. .

4. According to the claim The method for remote sensing detection of glacier changes based on deep twin neural networks is characterized in that: The step C specifically includes the following steps: Step C1: The dual-temporal glacier remote sensing images for change detection are simultaneously input into the trained change detection network model to obtain the binary prediction results of the glacier change detection area; Step C2: Vectorize the binary prediction results of the glacier change detection area obtained in step C1 to obtain a vector map of the glacier change area.

Images