Remote sensing image change detection method based on fixed-point quantization

Abstract

The present invention proposes a fixed-point quantized remote sensing image change detection method, which mainly solves the problems of the existing technology's lack of consideration of boundary information, the lack of emphasis on change image pairs in the loss function, and slow calculation. Its implementation scheme is as follows: obtain the remote sensing image, perform mean subtraction and normalization preprocessing on it, and divide it into training set and test set; based on the existing convolution layer, design parameters including quantization bit width and batch normalization layer to form a fixed-point quantized convolution layer; use the fixed-point quantized convolution layer to build a remote sensing change detection model; train the fixed-point quantized change detection model; use the trained fixed-point quantized change detection network to obtain image classification results. The present invention effectively improves the accuracy of image change detection results by constructing a new remote sensing change detection model. At the same time, by designing a fixed-point quantized convolution layer, it reduces the time required for network operations. It can be applied to vegetation cover analysis, urban planning, land resource management and disaster assessment.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a remote sensing image change detection method, which can be applied to vegetation cover analysis, urban planning, land resource management and disaster assessment. Background Art

[0002] With the widespread application of high-resolution remote sensing images, remote sensing image change detection has gradually been applied to coverage analysis, surface water changes, urban planning, land resource utilization management, disaster assessment, medical diagnosis and various aspects of civil infrastructure.

[0003] Since satellite sensors have the characteristics of revisiting and can obtain images with meter-level and sub-meter-level spatial resolution, satellite sensors and other earth observation sensors have become the main data source for change detection. They can conduct fine spatial detail investigations of ground objects. These developments have made remote sensing change detection a popular research direction.

[0004] Remote sensing image change detection involves comparing two remote sensing images taken at the same location but different time periods to detect relative differences. Change detection based on remote sensing images is crucial for scenarios such as deep space exploration. Improving the accuracy of remote sensing image change detection is both a challenge and a key aspect of remote sensing image change detection. The F1 score is the primary metric used to measure change detection accuracy; a higher F1 score indicates better change detection results.

[0005] Existing remote sensing image change detection methods are primarily categorized as traditional methods and deep learning-based methods. Traditional change detection methods require manual selection of parameters to control the size of output objects, limiting their performance. Furthermore, traditional methods rely on manual feature annotation of remote sensing images, making it difficult to obtain high-level change information. In recent years, convolutional neural networks (CNNs), one of the most commonly used deep neural networks, have demonstrated their ability to automatically extract complex high-level features to generate robust CD results.

[0006] In their paper “Change Detection Based on Deep SiameseConvolutional Network for Optical Aerial Images”, Y. Zhan et al. proposed a new supervised change detection method for optical aerial images based on a deep Siamese convolutional network. The method directly extracts features from image pairs by learning a Siamese network. Compared with the features manually extracted by traditional change detection methods, the extracted features are more abstract and robust.

[0007] In the paper “A Deep Convolutional Coupling Network for Change Detection Based on Heterogeneous Optical and Radar Images”, J.Liu et al. proposed an unsupervised deep convolutional coupling network change detection method for heterogeneous images. The method uses the complementary characteristics of optical sensors and radar sensors and combines the corresponding optimized coupling functions to unsupervisedly learn network parameters.

[0008] In their paper "DASNet: Dual Attentive Fully Convolutional Siamese Networks for Change Detection in High-Resolution Satellite Images," J. Chen et al. proposed a new method for high-resolution image change detection, namely the Dual Attentive Fully Convolutional Siamese Network. Through the dual attention mechanism, it captures long-range dependencies, obtains more discriminative feature representations, improves the model's recognition performance, and overcomes the problem that existing methods are not resistant to pseudo-changes.

[0009] However, the above methods all focus on the difference information between multi-temporal remote sensing images and lack consideration of boundary information. Therefore, they cannot show good detection performance in certain specific scenarios. At the same time, these methods lack attention to the changing image pairs in their loss functions, and because the network model is large and the number of parameters in the training data is large, the network information is redundant and the calculation is slow. Summary of the Invention

[0010] The purpose of the present invention is to address the deficiencies of the above-mentioned existing technologies and propose a remote sensing image change detection method based on fixed-point quantization, so as to fully consider the boundary information to improve the detection performance and increase the calculation speed through the fixed-point number.

[0011] To achieve the above object, the technical solution adopted by the present invention includes the following steps:

[0012] (1) Obtain remote sensing images and perform preprocessing of them by mean subtraction and linear normalization. The preprocessed data are then divided into a change detection training set and a test set in a ratio of 10:1.

[0013] (2) Construct fixed-point quantized convolutional layer:

[0014] (2a) Set the preset parameters and determine the quantization bit width based on the parameters:

[0015] If there is a quantization bit width parameter in the preset parameters, the preset value is used as the quantization bit width parameter;

[0016] If there is no quantization bit width parameter in the preset parameters, the mean square error of the neural network parameters before and after quantization under different bit widths is calculated, and the bit width with the smallest mean square error is used as the quantization bit width parameter;

[0017] (2b) Determine the parameters of the batch normalization layer through the first forward propagation:

[0018] In the first forward propagation calculation, the input of the convolution layer is quantized using the quantization bit width parameter, and the quantized convolution layer input is convolved with the floating-point value weight to obtain the scale parameter of the batch normalization layer without the need for backpropagation calculation;

[0019] (2c) Calculate the convolutional layer output:

[0020] In the second forward propagation calculation, the quantization bit width parameter is used to quantize the input and weight of the convolution layer, and the quantized convolution layer input is convolved with the quantized convolution layer weight. The result of the convolution calculation is used as the output of the convolution layer, and then the back propagation is performed normally.

[0021] (3) Using fixed-point quantized convolutional layers to build a remote sensing change detection model:

[0022] (3a) constructing a first convolutional module and a second convolutional module, each of which is composed of a first convolutional layer, a first ReLU activation layer, a second convolutional layer, a second ReLU activation layer, and a maximum pooling layer cascaded in sequence;

[0023] (3b) constructing a third convolutional module and a fourth convolutional module, each consisting of a first convolutional layer, a first ReLU activation layer, a second convolutional layer, a second ReLU activation layer, a third convolutional layer, a third ReLU activation layer, and a maximum pooling layer, respectively;

[0024] (3c) building a fifth convolutional module consisting of a first convolutional layer, a first ReLU activation layer, a second convolutional layer, a second ReLU activation layer, a third convolutional layer, and a third ReLU activation layer cascaded in sequence;

[0025] (3d) building a twin network tail convolution module consisting of a first deconvolution layer, a first batch normalization layer, a first ReLU activation layer, a second deconvolution layer, a second batch normalization layer, a second ReLU activation layer, and a convolution layer cascaded in sequence;

[0026] (3e) constructing five boundary extraction modules consisting of the first convolution layer, the first batch normalization layer, the first ReLU activation layer, the second convolution layer, the second batch normalization layer and the second ReLU activation layer in cascade order, and a boundary extraction network tail convolution module;

[0027] (3f) Build a channel fusion convolution module consisting of a deconvolution layer, a batch normalization layer, a ReLU activation layer, and a convolution layer in cascade order;

[0028] (3h) Cascading the first convolution module, the second convolution module, the third convolution module, the fourth convolution module, and the fifth convolution module with the existing image block-level second-order non-local module in sequence to form a twin branch, connecting the two twin branches in parallel, and then connecting the twin network tail convolution module to form a twin network;

[0029] (3i) connecting the first boundary extraction module, the second boundary extraction module, the third boundary extraction module, the fourth boundary extraction module, and the fifth boundary extraction module in parallel, and then connecting the boundary extraction network tail convolution module to form a boundary extraction network;

[0030] (3j) The twin network is connected in parallel with the boundary extraction network, and a channel fusion convolution module is connected after it to form a fixed-point quantization change detection network;

[0031] (4) Training the fixed-point quantized change detection network:

[0032] (4a) Design the network loss function L sum , including the relative loss L of the twin network cl With the boundary loss L edge Two parts, expressed as: L sum =L cl +L edge ;

[0033] (4b) Input the training set into the fixed-point quantized change detection network and use the Adam optimization algorithm to iteratively update the network weight value until the loss function L sum Until convergence, a trained fixed-point quantization change detection network is obtained;

[0034] (5) Input the test set into the trained fixed-point quantization change detection network to obtain the network output results, and divide the output results into two categories: changed and unchanged according to a pre-set threshold.

[0035] Compared with the prior art, the present invention has the following advantages:

[0036] First, since the present invention constructs a remote sensing change detection model consisting of a twin convolutional neural network and a boundary extraction network, it can extract boundary information while extracting the features of the image itself for joint training. This not only effectively improves the accuracy of image change detection results, but also facilitates more comprehensive change detection of images.

[0037] Second, the present invention designs a fixed-point quantized convolution layer, which performs fixed-point quantization on the input, weights, and other parameters in the convolution operation, which can greatly reduce the time required for calculations in the network, allowing effective change detection training and inference on devices with limited computing resources.

[0038] Third, the present invention improves and designs a new relative loss function for the change detection task, so that the network can pay more attention to the image pairs that have changed, effectively improving the training accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flow chart implemented by the present invention;

[0040] Figure 2 Schematic diagram of the fixed-point quantized convolutional layer in the present invention;

[0041] Figure 3 Schematic diagram of the fixed-point quantization change detection model constructed in the present invention;

[0042] Figure 4 yes Figure 3 Schematic diagram of the twin convolutional neural network in;

[0043] Figure 5 yes Figure 3 Schematic diagram of the boundary extraction network in;

[0044] Figure 6 It is a visualization detection result diagram of the simulation experiment of the present invention. DETAILED DESCRIPTION

[0045] The embodiments and effects of the present invention are described in further detail below with reference to the accompanying drawings.

[0046] Reference Figure 1 , the implementation steps for the instance are as follows:

[0047] Step 1. Obtain remote sensing images, preprocess them, and divide them into change detection training set and test set.

[0048] 1.1) Obtain CDD remote sensing image dataset from public database, where the sample set A={a1,a2,…,a i ,…,a n} and its corresponding variation sample set B={b1,b2,…,b i ,…,b n}, a i represents the i-th sample in the sample set, b i represents the i-th sample in the change sample set, and n is the number of images in the sample set;

[0049] 1.2) Perform mean subtraction preprocessing on the data sample set:

[0050] 1.2.1) Based on sample set A and its corresponding variation sample set B, calculate the pixel mean of each channel of the sample set And the pixel mean of each channel of the change sample set

[0051]

[0052]

[0053] Where c = {R, G, B} represents the three channels of the RGB image;

[0054] 1.2.2) Subtract the corresponding mean from the pixel value of each channel of each image in the sample set to obtain two sample sets after mean subtraction:

[0055] A′={a′1,a′2,…,a′ i ,…,a′ n}

[0056] B′={b′1,b′2,…,b′ i ,…,b′ n},

[0057] where a′ i represents the i-th sample of the sample set A′ after mean subtraction, b i represents the i-th sample of the change sample set B′ after mean subtraction;

[0058] 1.3) Normalize the dataset after mean subtraction:

[0059] For the sample set A′ and the changed sample set B′ obtained in 1.2), find the maximum values ​​max(A′) and max(B′) and the minimum values ​​min(A′) and min(B′) of the pixels in the samples, respectively. Then apply the normalization formula to these two sample sets A′ and B′ to obtain the normalized sample set A″ and the changed sample set B″. The formula is as follows:

[0060]

[0061]

[0062] Step 2. Based on the existing convolutional layer, design the parameters of the quantization bit width and batch normalization layer to form a fixed-point quantized convolutional layer.

[0063] Reference Figure 2 , this step is implemented as follows:

[0064] 2.1) Set the preset parameters and determine the quantization bit width according to the preset parameters:

[0065] If there is a quantization bit width parameter in the preset parameters, the preset value is used as the quantization bit width parameter;

[0066] If there is no quantization bit width parameter in the preset parameters, the mean square error of the neural network parameters before and after quantization under different bit widths is calculated, and the bit width with the smallest mean square error is used as the quantization bit width parameter. The mean square error formula is as follows:

[0067]

[0068] Among them, n represents the number of samples, l represents the quantization bit width, Y i represents the sample value, Represents the case where the bit width is l for Y i The sample value obtained by fixed-point quantization, MSE l represents the mean square error calculated under the corresponding quantization bit width l;

[0069] 2.2) Determine the parameters of the batch normalization layer through the first forward propagation:

[0070] In the first forward propagation calculation, the input of the convolution layer is quantized using the quantization bit width parameter, and the quantized convolution layer input is convolved with the floating-point value weight to obtain the scale parameter of the batch normalization layer. No backpropagation calculation is required. The formula used for quantization is as follows:

[0071]

[0072] Among them, l represents the quantization bit width, Y i represents the sample value, round(·) represents the rounding function, Represents the sample value obtained after fixed-point quantization;

[0073] 2.3) Calculate the convolutional layer output:

[0074] In the second forward propagation calculation, the input and weight of the convolution layer are quantized using the quantization bit width parameter. The quantization formula used is the same as 2.2). The quantized convolution layer input is convolved with the quantized convolution layer weight, and the result of the convolution calculation is used as the output of the convolution layer, and then the normal back propagation is performed;

[0075] Step 3. Build a remote sensing change detection model using fixed-point quantized convolutional layers.

[0076] Reference Figure 3 , this step is specifically implemented as follows:

[0077] 3.1) Build a first convolutional module consisting of a first fixed-point quantized convolutional layer, a first ReLU activation layer, a second fixed-point quantized convolutional layer, a second ReLU activation layer, and a maximum pooling layer, in cascade order, where:

[0078] The input channel parameter of the first fixed-point quantized convolutional layer is set to 3, the output channel parameter is set to 64, the convolution kernel size is set to 3×3 pixels, the stride is set to 1 pixel, and the boundary expansion value is set to 1 pixel;

[0079] The second fixed-point quantized convolution layer has an input channel parameter of 64 and an output channel parameter of 64. The convolution kernel size, stride, and boundary expansion value are the same as those of the first fixed-point quantized convolution layer.

[0080] In the maximum pooling layer, the convolution kernel size is 3×3 pixels, the stride is set to 2 pixels, and the boundary expansion value is set to 1 pixel;

[0081] The first ReLU activation layer and the second ReLU activation layer are both implemented using the ReLU function.

[0082] 3.2) Build a second convolutional module with the same structure as the first convolutional module but different parameters, where:

[0083] The input channel parameter of the first fixed-point quantized convolution layer in the second convolution module is set to 64, the output channel parameter is set to 128, the convolution kernel size is set to 3×3 pixels, the step size is set to 1 pixel, and the boundary expansion value is set to 1 pixel;

[0084] The second fixed-point quantized convolution layer in the second convolution module has its input channel parameter set to 128, its output channel parameter set to 128, and the convolution kernel size, step size, and boundary expansion value all the same as those of the first fixed-point quantized convolution layer;

[0085] In the second convolutional module, the maximum pooling layer has a convolution kernel size of 3×3 pixels, a stride of 2 pixels, and a boundary expansion value of 1 pixel.

[0086] The first ReLU activation layer and the second ReLU activation layer in the second convolution module are both implemented using the ReLU function.

[0087] 3.3) Build a third convolutional module consisting of the first fixed-point quantized convolutional layer, the first ReLU activation layer, the second fixed-point quantized convolutional layer, the second ReLU activation layer, the third fixed-point quantized convolutional layer, the third ReLU activation layer, and the maximum pooling layer in cascade order. The parameters of each layer are as follows:

[0088] The input channel parameter of the first fixed-point quantized convolutional layer is set to 128, the output channel parameter is set to 256, the convolution kernel size is set to 3×3 pixels, the stride is set to 1 pixel, and the boundary expansion value is set to 1 pixel;

[0089] The input channel parameter of the second fixed-point quantized convolutional layer is set to 256, the output channel parameter is set to 256, and the convolution kernel size, step size, and boundary expansion value are the same as those of the first fixed-point quantized convolutional layer;

[0090] The input channel parameter of the third fixed-point quantized convolutional layer is set to 256, the output channel parameter is set to 256, and the convolution kernel size, step size, and boundary expansion value are the same as those of the first fixed-point quantized convolutional layer;

[0091] The maximum pooling layer has a convolution kernel size of 3×3 pixels, a stride of 2 pixels, and a boundary expansion value of 1 pixel;

[0092] The first ReLU activation layer, the second ReLU activation layer, and the third ReLU activation layer are all implemented using the ReLU function.

[0093] 3.4) Build the fourth convolutional module. Its structure is the same as the third convolutional module. The parameters of each layer are as follows:

[0094] In the fourth convolution module, the input channel parameter of the first fixed-point quantized convolution layer is set to 256, the output channel parameter is set to 512, the convolution kernel size is set to 3×3 pixels, the step size is set to 1 pixel, and the boundary expansion value is set to 1 pixel;

[0095] The input channel parameter of the second fixed-point quantized convolution layer in the fourth convolution module is set to 512, the output channel parameter is set to 512, and the convolution kernel size, step size, and boundary expansion value are all the same as those of the first fixed-point quantized convolution layer;

[0096] In the fourth convolution module, the input channel parameter of the third fixed-point quantized convolution layer is set to 512, the output channel parameter is set to 512, and the convolution kernel size, step size, and boundary expansion value are all the same as those of the first fixed-point quantized convolution layer;

[0097] In the fourth convolutional module, the maximum pooling layer has a convolution kernel size of 3×3 pixels, a stride of 1 pixel, and a boundary expansion value of 1 pixel.

[0098] The first ReLU activation layer, the second ReLU activation layer and the third ReLU activation layer in the fourth convolution module are all implemented using the ReLU function.

[0099] 3.5) Build a fifth convolutional module consisting of the first fixed-point quantized convolutional layer, the first ReLU activation layer, the second fixed-point quantized convolutional layer, the second ReLU activation layer, the third fixed-point quantized convolutional layer, and the third ReLU activation layer in cascade order. The parameters of each layer are as follows:

[0100] The input channel parameter of the first fixed-point quantized convolutional layer is set to 512, the output channel parameter is set to 512, the convolution kernel size is set to 3×3 pixels, the receptive field is set to 2 pixels, the step size is set to 1 pixel, and the boundary expansion value is set to 2 pixels;

[0101] The input channel parameter of the second fixed-point quantized convolution layer is set to 512, the output channel parameter is set to 512, and the convolution kernel size, step size, receptive field and boundary expansion value are the same as those of the first fixed-point quantized convolution layer;

[0102] The input channel parameter of the third fixed-point quantized convolution layer is set to 512, the output channel parameter is set to 512, and the convolution kernel size, step size, receptive field and boundary expansion value are the same as those of the first fixed-point quantized convolution layer;

[0103] The I ReLU activation layer, the II ReLU activation layer, and the III ReLU activation layer are all implemented using the ReLU function.

[0104] 3.6) Build a twin network tail convolution module consisting of the first deconvolution layer, the first batch normalization layer, the first ReLU activation layer, the second deconvolution layer, the second batch normalization layer, the second ReLU activation layer, and the fixed-point quantization convolution layer in cascade order. The parameters of each layer are as follows:

[0105] The input channel parameter of the first deconvolution layer in the tail convolution module of the twin network is set to 512, the output channel parameter is set to 256, the convolution kernel size is set to 3×3 pixels, the stride is set to 1 pixel, and the boundary expansion value is set to 2 pixels;

[0106] The parameter of the first batch normalization layer in the tail convolution module of the twin network is set to 256;

[0107] The input channel parameter of the second deconvolution layer in the tail convolution module of the twin network is set to 256, the output channel parameter is set to 128, and the convolution kernel size, step size and boundary expansion value are the same as those of the first deconvolution layer;

[0108] The parameter of the second batch normalization layer in the tail convolution module of the twin network is set to 256;

[0109] The input channel parameter of the fixed-point quantized convolution layer in the tail convolution module of the twin network is set to 128, the output channel parameter is set to 128, the convolution kernel size is set to 1×1 pixels, and the stride is set to 1 pixel;

[0110] The first ReLU activation layer and the second ReLU activation layer in the tail convolution module of the twin network are both implemented using the ReLU function.

[0111] 3.7) Build five boundary extraction modules, each consisting of a cascade of the first fixed-point quantized convolutional layer, the first batch normalization layer, the first ReLU activation layer, the second fixed-point quantized convolutional layer, the second batch normalization layer, and the second ReLU activation layer, where:

[0112] In the first boundary extraction module, the input channel parameter of the first fixed-point quantized convolution layer is set to 64, the output channel parameter is set to 64, the convolution kernel size is set to 3×3 pixels, and the step size is set to 1 pixel; the parameter of the first batch normalization layer is set to 64; the input channel parameter of the second fixed-point quantized convolution layer is set to 64, the output channel parameter is set to 1, the convolution kernel size is set to 1×1 pixels, and the step size is set to 1 pixel; the parameter of the second batch normalization layer is set to 1;

[0113] In the second edge extraction module, the input channel parameter of the first fixed-point quantized convolution layer is set to 128, the output channel parameter is set to 32, the convolution kernel size is set to 3×3 pixels, and the step size is set to 1 pixel; the parameter of the first batch normalization layer is set to 32; the input channel parameter of the second fixed-point quantized convolution layer is set to 32, the output channel parameter is set to 1, the convolution kernel size is set to 1×1 pixels, and the step size is set to 1 pixel; the parameter of the second batch normalization layer is set to 1;

[0114] In the third edge extraction module, the input channel parameter of the first fixed-point quantized convolution layer is set to 256, the output channel parameter is set to 16, the convolution kernel size is set to 3×3 pixels, and the step size is set to 1 pixel; the parameter of the first batch normalization layer is set to 16; the input channel parameter of the second fixed-point quantized convolution layer is set to 16, the output channel parameter is set to 1, the convolution kernel size is set to 1×1 pixels, and the step size is set to 1 pixel; the parameter of the second batch normalization layer is set to 1;

[0115] In the fourth edge extraction module, the input channel parameter of the first fixed-point quantized convolution layer is set to 512, the output channel parameter is set to 8, the convolution kernel size is set to 3×3 pixels, and the step size is set to 1 pixel; the parameter of the first batch normalization layer is set to 8; the input channel parameter of the second fixed-point quantized convolution layer is set to 8, the output channel parameter is set to 1, the convolution kernel size is set to 1×1 pixels, and the step size is set to 1 pixel; the parameter of the second batch normalization layer is set to 1;

[0116] In the fifth boundary extraction module, the input channel parameter of the first fixed-point quantized convolution layer is set to 512, the output channel parameter is set to 8, the convolution kernel size is set to 3×3 pixels, and the step size is set to 1 pixel; the parameter of the first batch normalization layer is set to 8; the input channel parameter of the second fixed-point quantized convolution layer is set to 8, the output channel parameter is set to 1, the convolution kernel size is set to 1×1 pixels, and the step size is set to 1 pixel; the parameter of the second batch normalization layer is set to 1;

[0117] The first ReLU activation layer and the second ReLU activation layer of the above five boundary extraction modules are both implemented using the ReLU function.

[0118] 3.8) Build a boundary extraction network tail convolution module with the same structure as the five boundary extraction modules above. The parameters of each layer are as follows:

[0119] In the tail convolution module of the boundary extraction network, the input channel parameter of the first fixed-point quantized convolution layer is set to 5, the output channel parameter is set to 5, and the convolution kernel size is set to 1×1 pixels; the parameter of the first batch normalization layer is set to 5; the input channel parameter of the second fixed-point quantized convolution layer is set to 5, the output channel parameter is set to 64, and the convolution kernel size is set to 1×1 pixels; the parameter of the second batch normalization layer is set to 64;

[0120] The first ReLU activation layer and the second ReLU activation layer of the tail convolution module of the boundary extraction network are both implemented using the ReLU function.

[0121] 3.9) Build a channel-fusion convolution module consisting of a cascade of deconvolution layers, batch normalization layers, ReLU activation layers, and fixed-point quantization convolution layers. The parameters of each layer are as follows:

[0122] The input channel parameter of the deconvolution layer is set to 256, the output channel parameter is set to 32, the convolution kernel size is set to 3×3 pixels, the step size is set to 1 pixel, and the boundary expansion value is set to 2 pixels;

[0123] The parameter of the batch normalization layer is set to 32;

[0124] The ReLU activation layer is implemented using the ReLU function;

[0125] The fixed-point quantized convolution layer has an input channel parameter set to 32, an output channel parameter set to 32, and a convolution kernel size set to 1×1 pixels.

[0126] 3.10) The first convolution module, the second convolution module, the third convolution module, the fourth convolution module, and the fifth convolution module are sequentially cascaded with the existing image block-level second-order non-local module to form a twin branch. The two twin branches are connected in parallel and then connected to the twin network tail convolution module to form a twin network, as shown in FIG. Figure 4 ;

[0127] 3.11) The first boundary extraction module, the second boundary extraction module, the third boundary extraction module, the fourth boundary extraction module and the fifth boundary extraction module are connected in parallel, and the boundary extraction network tail convolution module is connected thereafter to form a boundary extraction network, such as Figure 5 ;

[0128] 3.12) The twin network is connected in parallel with the boundary extraction network, and a channel fusion convolution module is connected to it to form a fixed-point quantization change detection model.

[0129] Step 4. Train the fixed-point quantized change detection model.

[0130] 4.1) Design network loss function L sum :

[0131] 4.1.1) Improve the existing relative loss function to obtain the twin network change loss L cl :

[0132]

[0133] Where N represents the number of pixel pairs, d i =‖a i -b i ‖2 represents the i-th pair of element a i and b i The Euclidean distance between i y is the element matching parameter, i =1 means the i-th pair of elements matches, y i =0 means the i-th pair of elements do not match, m is the pre-set change detection threshold, m>0, w1 and w2 are change weights, 1 <w1<1.5,0.5<w2<1;

[0134] 4.1.2) Improve the existing cross entropy loss function to obtain the designed boundary loss L ci :

[0135]

[0136] Among them, b i represents the truth boundary, p i Represents the predicted boundary change map obtained in the boundary extraction network, w p and w nRepresents the weight of the changed item and the unchanged item;

[0137] 4.1.3) Loss L changed by the twin network cl and boundary loss L edge Get the network loss function L sum :

[0138] L sum =L cl +L edge ;

[0139] 4.2) Input the training set into the fixed-point quantization change detection model and use the Adam optimization algorithm to iteratively update the network weight values:

[0140] 4.2.1) Calculate the current loss function L sum (θ t ) for the parameter θ t Gradient:

[0141]

[0142] Where t is the current t-th iteration;

[0143] 4.2.2) According to the pre-set learning rate η, the first-order momentum m of the previous iteration t-1 and the second-order momentum v t-1 , combined with the calculated current gradient g t Calculate the first-order momentum m at iteration t with the running average coefficients β1 and β2 t With the second-order momentum v t :

[0144] m t =η[β1m t-1 +(1-β1)g t

[0145]

[0146] 4.2.3) According to the first-order momentum m of the t-th iteration t With the second-order momentum v t , the parameter θ t Update to θ t+1 ,

[0147]

[0148] Where ∈ represents the remainder, preventing the divisor from being 0;

[0149] 4.2.4) Repeat steps 4.2.1) to 4.2.3) until the loss function L sum Until convergence, a trained fixed-point quantization change detection network is obtained.

[0150] Step 5. Use the trained fixed-point quantization change detection network to obtain the image classification results.

[0151] 5.1) Input the test set into the trained fixed-point quantization change detection network and obtain the network output C(A,B), where A={a1,a2,…,a i ,…,a n} is the sample set, B={b1,b2,…,b i ,…,b n} is its corresponding change set;

[0152] 5.2) Set the image change threshold τ>0, and convert the network output C(a i ,b i ) is compared with the threshold to obtain the test set images as two categories: changed and unchanged:

[0153] If C(a i ,b i )<τ, then the change detection function f(a i ,b i )=0, representing image pair a i ,b i No change has occurred, that is, the image belongs to the unchanged class;

[0154] If C(a i ,b i )≥τ, then the change detection function f(a i ,b i )=1, representing image pair a i ,b i Change occurs, that is, the image belongs to the change class.

[0155] The effects of the present invention will be further described below in conjunction with simulation experiments:

[0156] 1. Simulation experiment conditions.

[0157] The hardware platform for the simulation experiment of the present invention is: NVIDIA GeForce RTX 3090.

[0158] The software platform for the simulation experiment of the present invention is: Ubuntu 18.04 operating system, based on the PyTorch1.11.0 deep learning framework, and the programming language is Python 3.8.

[0159] The data set used in the simulation experiment of the present invention is the data set constructed in step 1 in the specific implementation of the present invention.

[0160] 2. Simulation experiment content and result analysis.

[0161] The simulation experiment of the present invention uses the method of the present invention and a prior art DASNet method to perform change detection on the pre-processed CDD remote sensing image dataset. The detection results are Figure 6 .in:

[0162] Figure 6 (a) is the real image,

[0163] Figure 6 (b) is the change image corresponding to the real image.

[0164] Figure 6 (c) is the label image corresponding to the real image.

[0165] Figure 6 (d) is the boundary information extracted by the existing DASNet method.

[0166] Figure 6 Column (e) is the boundary information extracted by the present invention.

[0167] The DASNet method is a remote sensing change detection method proposed by J. Chen et al. in “DASNet: Dual Attentive Fully Convolutional Siamese Networks for Change Detection in High-Resolution Satellite Images” (IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 14, pp. 1194-1206, 2021).

[0168] Depend on Figure 6 As can be seen from column (d), the boundary information extracted by DASNet is not smooth and Figure 6 There is a big gap between the labeled images in column (c), mainly because this method does not fully consider the boundary information when building the model, resulting in low detection accuracy.

[0169] It can be seen from column 6(e) that the boundary information extracted by the present invention has good smoothness and is consistent with the Figure 6 The label image in column (c) has a smaller gap and is closer to the boundary of the label image, so the detection result is more ideal.

[0170] The detection results of the above two methods are evaluated using three evaluation indicators: F1_SCORE, mIoU and average iterative training time. The calculation formulas of F1_SCORE and mIoU are as follows:

[0171]

[0172]

[0173]

[0174]

[0175] Among them, TP represents the positive samples predicted by the model as positive, FP represents the negative samples predicted by the model as positive, FN represents the positive samples predicted by the model as negative, k represents the number of categories, and k = 2 in the change detection task.

[0176] The evaluation results are shown in Table 1:

[0177] Table 1. Comparison of evaluation indicators of the two methods in simulation experiments

[0178] F1_SCORE mIoU Average training iteration time The present invention 0.9297 0.9302 819.666477 DASNet 0.9190 0.9045 965.038872

[0179] As can be seen from the data in Table 1, the remote sensing image change detection method of the present invention is higher than the DASNet method in terms of F1_SCORE and mIoU indicators, and lower than the DASNet method in terms of average iterative training time, indicating that the present invention can achieve higher change detection accuracy and shorter training time.

[0180] The above simulation experiments show that the method of the present invention solves the problems in the existing technology of lack of consideration of boundary information, lack of emphasis on changed image pairs in the loss function, and slow calculation. It can not only more effectively extract boundary information in image changes and improve the accuracy of change detection, but also reduce training time.

Claims

1. A remote sensing image change detection method based on fixed-point quantization, characterized in that: The steps include: (1) Obtain remote sensing images and perform preprocessing of them by mean subtraction and linear normalization. The preprocessed data are then divided into a change detection training set and a test set in a ratio of 10:

1. (2) Based on the existing convolutional layer, design parameters including quantization bit width and batch normalization layer to form a fixed-point quantized convolutional layer: (2a) Set the preset parameters and determine the quantization bit width based on the parameters: If there is a quantization bit width parameter in the preset parameters, the preset value is used as the quantization bit width parameter; If there is no quantization bit width parameter in the preset parameters, the mean square error of the neural network parameters before and after quantization under different bit widths is calculated, and the bit width with the smallest mean square error is used as the quantization bit width parameter; (2b) Determine the parameters of the batch normalization layer through the first forward propagation: In the first forward propagation calculation, the input of the convolution layer is quantized using the quantization bit width parameter, and the quantized convolution layer input is convolved with the floating-point value weight to obtain the scale parameter of the batch normalization layer without the need for backpropagation calculation; (2c) Calculate the convolutional layer output: In the second forward propagation calculation, the quantization bit width parameter is used to quantize the input and weight of the convolution layer, and the quantized convolution layer input is convolved with the quantized convolution layer weight. The result of the convolution calculation is used as the output of the convolution layer, and then the back propagation is performed normally. (3) Using fixed-point quantized convolutional layers to build a remote sensing change detection model: (3a) constructing a first convolution module and a second convolution module, each of which is composed of a first fixed-point quantized convolution layer, a first ReLU activation layer, a second fixed-point quantized convolution layer, a second ReLU activation layer and a maximum pooling layer cascaded in sequence; (3b) constructing a third convolution module and a fourth convolution module respectively consisting of a first fixed-point quantized convolution layer, a first ReLU activation layer, a second fixed-point quantized convolution layer, a second ReLU activation layer, a third fixed-point quantized convolution layer, a third ReLU activation layer and a maximum pooling layer cascaded in sequence; (3c) building a fifth convolution module consisting of the first fixed-point quantized convolution layer, the first ReLU activation layer, the second fixed-point quantized convolution layer, the second ReLU activation layer, the third fixed-point quantized convolution layer, and the third ReLU activation layer in cascade order; (3d) building a twin network tail convolution module consisting of a first deconvolution layer, a first batch normalization layer, a first ReLU activation layer, a second deconvolution layer, a second batch normalization layer, a second ReLU activation layer, and a fixed-point quantization convolution layer cascaded in sequence; (3e) constructing five boundary extraction modules and a boundary extraction network tail convolution module, which are respectively composed of the first fixed-point quantization convolution layer, the first batch normalization layer, the first ReLU activation layer, the second fixed-point quantization convolution layer, the second batch normalization layer and the second ReLU activation layer cascaded in sequence; (3f) Build a channel fusion convolution module consisting of a deconvolution layer, a batch normalization layer, a ReLU activation layer, and a fixed-point quantization convolution layer in cascade order; (3h) Cascading the first convolution module, the second convolution module, the third convolution module, the fourth convolution module, and the fifth convolution module with the existing image block-level second-order non-local module in sequence to form a twin branch, connecting the two twin branches in parallel, and then connecting the twin network tail convolution module to form a twin network; (3i) connecting the first boundary extraction module, the second boundary extraction module, the third boundary extraction module, the fourth boundary extraction module, and the fifth boundary extraction module in parallel, and then connecting the boundary extraction network tail convolution module to form a boundary extraction network; (3j) The twin network is connected in parallel with the boundary extraction network, and a channel fusion convolution module is connected after it to form a fixed-point quantization change detection network; (4) Training the fixed-point quantized change detection network: (4a) Design the network loss function L sum , including the relative loss L of the twin network cl With the boundary loss L edge Two parts, expressed as: L sum =L cl +L edge ; (4b) Input the training set into the fixed-point quantized change detection network and use the Adam optimization algorithm to iteratively update the network weight value until the loss function L sum Until convergence, a trained fixed-point quantization change detection network is obtained; (5) Input the test set into the trained fixed-point quantization change detection network to obtain the output of the network, and divide the output into two categories: changed and unchanged by comparing with a pre-set threshold.

2. The method according to claim 1, wherein: In step (1), the remote sensing image is preprocessed by mean subtraction and linear normalization, which is implemented as follows: 1a) For the sample set A = {a1, a2, ..., a n } and its corresponding variation sample set B={b1,b2,…,b n }, calculate the pixel mean of each channel Where c = {R, G, B} represents the three channels of the RGB image, and n represents the number of images in the sample set; 1b) Subtract the corresponding mean from the pixel value of each channel of each image in the sample set to obtain two sample sets A′={a′1,a′2,…,a′ n } and B′={b′1,b′2,…,b′ n }; 1c) For the two sample sets A′ and B′ obtained in 1b), first find the maximum pixel values ​​max(A′) and max(B′) and the minimum pixel values ​​min(A′) and min(B′) of the samples. Then, apply the normalization formula to these two sample sets A′ and B′ to obtain the two normalized sample sets A″ and B″. The formula is as follows:

3. The method according to claim 1, wherein: In step (2a), the mean square error of the neural network parameters before and after quantization at different bit widths is calculated as follows: Among them, n represents the number of samples, l represents the quantization bit width, and Y i represents the sample value, Y i l Represents the case where the bit width is l for Y i The sample value obtained by fixed-point quantization, MSE l Represents the mean square error calculated under the corresponding quantization bit width l.

4. The method according to claim 1, wherein: In steps (2b) and (2c), the quantization bit width parameter is used to quantize the input of the convolution layer. The formula is as follows: Among them, l represents the quantization bit width, Y i represents the sample value, round(·) represents the rounding function, Y i l Represents the sample value obtained after fixed-point quantization.

5. The method according to claim 1, wherein: In the first convolution module and the second convolution module built in step (3a), the parameters of each layer are as follows: In the first convolution module, the input channel parameter of the first fixed-point quantized convolution layer is set to 3, the output channel parameter is set to 64, the convolution kernel size is set to 3×3 pixels, the step size is set to 1 pixel, and the boundary expansion value is set to 1 pixel; the input channel parameter of the second fixed-point quantized convolution layer is set to 64, the output channel parameter is set to 64, the convolution kernel size, step size and boundary expansion value are all the same as those of the first fixed-point quantized convolution layer; the convolution kernel size of the maximum pooling layer is 3×3 pixels, the step size is set to 2 pixels, and the boundary expansion value is set to 1 pixel; In the second convolution module, the input channel parameter of the first fixed-point quantized convolution layer is set to 64, the output channel parameter is set to 128, the convolution kernel size is set to 3×3 pixels, the step size is set to 1 pixel, and the boundary expansion value is set to 1 pixel; the input channel parameter of the second fixed-point quantized convolution layer is set to 128, the output channel parameter is set to 128, the convolution kernel size, step size and boundary expansion value are all the same as those of the first fixed-point quantized convolution layer; the maximum pooling layer has a convolution kernel size of 3×3 pixels, a step size of 2 pixels, and a boundary expansion value of 1 pixel; The first ReLU activation layer and the second ReLU activation layer in the first and second convolution modules are both implemented using the ReLU function.

6. The method according to claim 1, wherein: In the third and fourth convolution modules built in step (3b), the parameters of each layer are as follows: In the third convolution module, the input channel parameter of the first fixed-point quantized convolution layer is set to 128, the output channel parameter is set to 256, the convolution kernel size is set to 3×3 pixels, the step size is set to 1 pixel, and the boundary expansion value is set to 1 pixel; the input channel parameter of the second fixed-point quantized convolution layer is set to 256, the output channel parameter is set to 256, and the convolution kernel size, step size, and boundary expansion value are all the same as those of the first fixed-point quantized convolution layer; The input channel parameter of the third fixed-point quantized convolutional layer is set to 256, the output channel parameter is set to 256, and the convolution kernel size, stride, and boundary expansion value are the same as those of the first convolutional layer. The convolution kernel size of the maximum pooling layer is 3×3 pixels, the stride is set to 2 pixels, and the boundary expansion value is set to 1 pixel. In the fourth convolution module, the input channel parameter of the first fixed-point quantized convolution layer is set to 256, the output channel parameter is set to 512, the convolution kernel size is set to 3×3 pixels, the step size is set to 1 pixel, and the boundary expansion value is set to 1 pixel; the input channel parameter of the second fixed-point quantized convolution layer is set to 512, the output channel parameter is set to 512, and the convolution kernel size, step size, and boundary expansion value are all the same as those of the first convolution layer; The input channel parameter of the third fixed-point quantized convolutional layer is set to 512, the output channel parameter is set to 512, and the convolution kernel size, stride, and boundary expansion value are the same as those of the first convolutional layer. The convolution kernel size of the maximum pooling layer is 3×3 pixels, the stride is set to 1 pixel, and the boundary expansion value is set to 1 pixel. The first ReLU activation layer, the second ReLU activation layer, and the third ReLU activation layer in the third and fourth convolutional modules are all implemented using the ReLU function.

7. The method according to claim 1, wherein: Step (3c) builds the parameters of each layer in the fifth convolutional module as follows: The input channel parameter of the first fixed-point quantized convolution layer is set to 512, the output channel parameter is set to 512, the convolution kernel size is set to 3×3 pixels, the receptive field is set to 2 pixels, the step size is set to 1 pixel, and the boundary expansion value is set to 2 pixels; The input channel parameter of the second fixed-point quantized convolution layer is set to 512, the output channel parameter is set to 512, and the convolution kernel size, step size, receptive field and boundary expansion value are the same as those of the first convolution layer; The input channel parameter of the third fixed-point quantized convolutional layer is set to 512, the output channel parameter is set to 512, and the convolution kernel size, step size, receptive field and boundary expansion value are the same as those of the first convolutional layer; The I-th ReLU activation layer, the II-th ReLU activation layer and the III-th ReLU activation layer in the fifth convolution module are all implemented using the ReLU function.

8. The method according to claim 1, wherein: Step (3d) builds the parameters of each layer in the twin network tail convolution module as follows: The input channel parameter of the first deconvolution layer is set to 512, the output channel parameter is set to 256, the convolution kernel size is set to 3×3 pixels, the step size is set to 1 pixel, and the boundary expansion value is set to 2 pixels; Its first batch normalization layer parameter is set to 256; The input channel parameter of the second deconvolution layer is set to 256, the output channel parameter is set to 128, and the convolution kernel size, step size, and boundary expansion value are the same as those of the first deconvolution layer; The parameter of its second batch normalization layer is set to 256; the input channel parameter of its convolutional layer is set to 128, the output channel parameter is set to 128, the convolution kernel size is set to 1×1 pixels, and the stride is set to 1 pixel; The first ReLU activation layer and the second ReLU activation layer in the tail convolution module of the twin network are both implemented using the ReLU function.

9. The method according to claim 1, wherein: Step (3e) builds five boundary extraction modules and one boundary extraction network tail convolution module. The parameters of each layer are as follows: In the first boundary extraction module, the input channel parameter of the first fixed-point quantized convolution layer is set to 64, the output channel parameter is set to 64, the convolution kernel size is set to 3×3 pixels, and the step size is set to 1 pixel; The parameter of the first batch normalization layer is set to 64; the input channel parameter of the second fixed-point quantized convolutional layer is set to 64, the output channel parameter is set to 1, the convolution kernel size is set to 1×1 pixels, and the stride is set to 1 pixel; The parameter of its second batch normalization layer is set to 1; In the second edge extraction module, the input channel parameter of the first fixed-point quantized convolution layer is set to 128, the output channel parameter is set to 32, the convolution kernel size is set to 3×3 pixels, and the step size is set to 1 pixel; The parameter of the first batch normalization layer is set to 32; the input channel parameter of the second fixed-point quantized convolutional layer is set to 32, the output channel parameter is set to 1, the convolution kernel size is set to 1×1 pixels, and the stride is set to 1 pixel; The parameter of its second batch normalization layer is set to 1; In the third edge extraction module, the input channel parameter of the first fixed-point quantized convolution layer is set to 256, the output channel parameter is set to 16, the convolution kernel size is set to 3×3 pixels, and the step size is set to 1 pixel; The parameter of the first batch normalization layer is set to 16; the input channel parameter of the second fixed-point quantized convolutional layer is set to 16, the output channel parameter is set to 1, the convolution kernel size is set to 1×1 pixels, and the stride is set to 1 pixel; The parameter of its second batch normalization layer is set to 1; In the fourth edge extraction module, the input channel parameter of the first fixed-point quantized convolution layer is set to 512, the output channel parameter is set to 8, the convolution kernel size is set to 3×3 pixels, and the step size is set to 1 pixel; The parameter of its first batch normalization layer is set to 8; the input channel parameter of its second fixed-point quantized convolutional layer is set to 8, the output channel parameter is set to 1, the convolution kernel size is set to 1×1 pixels, and the stride is set to 1 pixel; The parameter of its second batch normalization layer is set to 1; In the fifth edge extraction module, the input channel parameter of the first fixed-point quantized convolution layer is set to 512, the output channel parameter is set to 8, the convolution kernel size is set to 3×3 pixels, and the step size is set to 1 pixel; The parameter of its first batch normalization layer is set to 8; the input channel parameter of its second fixed-point quantized convolutional layer is set to 8, the output channel parameter is set to 1, the convolution kernel size is set to 1×1 pixels, and the stride is set to 1 pixel; The parameter of its second batch normalization layer is set to 1; In the tail convolution module of the boundary extraction network, the input channel parameter of the first fixed-point quantized convolution layer is set to 5, the output channel parameter is set to 5, and the convolution kernel size is set to 1×1 pixels; The parameter of its first batch normalization layer is set to 5; The input channel parameter of the second fixed-point quantized convolutional layer is set to 5, the output channel parameter is set to 64, and the convolution kernel size is set to 1×1 pixels; The parameter of its second batch normalization layer is set to 64; The first and second ReLU activation layers of the above five boundary extraction modules and the tail convolution module of the boundary extraction network are all implemented using ReLU functions.

10. The method according to claim 1, wherein: Step (3f) builds the parameters of each layer in the channel fusion convolution module as follows: The input channel parameter of the deconvolution layer is set to 256, the output channel parameter is set to 32, the convolution kernel size is set to 3×3 pixels, the step size is set to 1 pixel, and the boundary expansion value is set to 2 pixels; The parameter of its batch normalization layer is set to 32; Its ReLU activation layer is implemented using the ReLU function; The input channel parameter of its fixed-point quantized convolution layer is set to 32, the output channel parameter is set to 32, and the convolution kernel size is set to 1×1 pixels.

11. The method according to claim 1, wherein: The relative loss L of the twin network in step (4a) cl and boundary loss L edge They are represented as follows: Where N represents the number of pixel pairs, d i =||a i -b i ||2 represents the i-th pair of elements a i and b i The Euclidean distance between i =1 means the i-th pair of elements matches, y i =0 means the i-th pair of elements does not match, m is the preset change detection threshold, m>0, w1 and w2 are change weights, 1<w1<1.5, 0.5<w2<1, b i represents the truth boundary, p i Represents the predicted boundary change map obtained in the boundary extraction network, w p and w n Represents the weight of the changed item and the unchanged item.

12. The method according to claim 1, wherein: In step (4b), the Adam optimization algorithm is used to iteratively update the network weight values, which is implemented as follows: (4b1) Calculate the current loss function L sum (θ t ) for the parameter θ t Gradient: Where t is the current t-th iteration; (4b2) According to the pre-set learning rate η, the first-order momentum m of the previous iteration t-1 and the second-order momentum v t-1 , combined with the calculated current gradient g t Calculate the first-order momentum m at iteration t with the running average coefficients β1 and β2 t With the second-order momentum v t : m t =η[β1m t-1 +(1-β1)g t (4b3) According to the first-order momentum m of the t-th iteration t With the second-order momentum v t , the parameter θ t Update to θ t+1 , Where ∈ represents the remainder, preventing the divisor from being 0; (4b4) Repeat (4b1) to (4b3) until the loss function L sum Until convergence.

13. The method according to claim 1, wherein: In step (5), the output results are divided into two categories: changed and unchanged by a pre-set threshold, which is implemented as follows: Set the threshold τ>0, and set the network output value C(a i , b i ) is compared with this threshold: If C(a i , b i )<τ, then the change detection function f(a i , b i )=0, representing image pair a i , b i No change has occurred; If C(a i , b i )≥τ, then the change detection function f(a i , b i )=1, representing image pair a i , b i Changes have occurred.

Images