Lightweight remote sensing image change detection method based on binary neural network and large kernel stripe convolution

Abstract

The invention provides a lightweight remote sensing image change detection method based on a binary neural network and large kernel stripe convolution. The implementation steps are as follows: obtaining a training sample set and a test sample set; constructing a lightweight remote sensing image change detection network model based on a binary neural network and large-kernel stripe convolution, and performing iterative training on the lightweight remote sensing image change detection network model; and obtaining a lightweight remote sensing image change detection result. In the process of carrying out iterative training on a lightweight remote sensing image change detection network model and obtaining a change detection result, deep feature extraction is carried out on a dual-tense feature map through ResNet18 networks in the two branch networks, the weight and activation can be quantified as + 1 and-1, 32 times of memory saving and 58 times of CPU acceleration are achieved, and the method has the advantages of being simple in structure, convenient to operate and high in practicability. The defect that in the prior art, the number of parameters is large is overcome, the detection efficiency is effectively improved, and meanwhile the defect of accuracy caused by binary quantization is overcome through the large-kernel stripe convolutional network.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing image processing, and relates to a lightweight remote sensing image change detection method, and specifically to a lightweight remote sensing image change detection method based on binary neural network and large kernel strip convolution, which can be used in the fields of geological disaster monitoring, land cover investigation and urban planning. Background Art

[0002] Remote sensing images are images of the earth's surface information such as surface features, vegetation cover, land use, urban expansion, and water distribution obtained from a long distance by satellite-borne or airborne sensors. Since remote sensing images have the advantage of covering a large area and can quickly scan a wide area, they are widely used in geographic and environmental monitoring. Usually, such images need to be processed by digital image processing technology to extract key information.

[0003] Change detection in remote sensing images refers to processing two images taken at different times at the same location to find the differences between them. Specifically, the remote sensing image change detection system needs to compare two registered remote sensing images and assign a binary label to each pixel, which indicates whether the area corresponding to the pixel has changed during the shooting time period. In the final change detection map, white usually indicates a change, and black indicates no change.

[0004] Image change detection methods can be divided into traditional methods and deep learning-based methods. Traditional methods include detection methods based on image algebraic operations, image transformation, and post-classification. However, traditional change detection methods have many disadvantages, including the need for complex image preprocessing, sensitivity to noise, difficulty in feature extraction, lack of adaptability, complex calculations, difficulty in real-time application, and often limited to a single data source, making the results difficult to interpret. These factors limit the detection accuracy and efficiency. In contrast, deep learning-based methods have end-to-end characteristics and powerful nonlinear mapping capabilities, which can effectively avoid the tedious process of manually designing features in traditional methods.

[0005] In the existing remote sensing image change detection methods based on deep learning, the number of parameters and computational complexity are relatively high, especially when processing high-resolution images or large-scale scenes, which may lead to low computational efficiency. For example, the patent application with the application publication number CN118334532A and the name "Lightweight remote sensing change detection method and system based on dual-temporal remote sensing images" discloses a lightweight remote sensing image change detection method based on dual-temporal remote sensing images. The remote sensing image change detection network used in the invention includes a feature extraction backbone RSShuffle, LightSEPP, Transformer encoder-decoder and prediction head. The remote sensing change detection network is trained by inputting a data set with dual-temporal remote sensing graphics, and the trained remote sensing change detection network outputs a predicted change map of the predicted change area. The invention adopts the method of reducing the network architecture to lightweight the model, which reduces the number of model parameters to a certain extent. However, since full-precision floating-point parameters are still used during calculation, the number of model parameters is still large, which affects the further improvement of detection efficiency. Summary of the invention

[0006] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and propose a lightweight remote sensing image change detection method based on binary neural network and large kernel strip convolution, aiming to improve the detection efficiency while ensuring the detection accuracy.

[0007] To achieve the above object, the technical solution adopted by the present invention includes the following steps:

[0008] (1) Obtain training sample set, verification sample set and test sample set:

[0009] Preprocess N pairs of bi-temporal remote sensing images containing multiple target categories, and annotate the changed areas in the image blocks corresponding to the preprocessed bi-temporal remote sensing images. Then, the preprocessed image blocks corresponding to more than half of the bi-temporal remote sensing images and their labels form a training sample set, and the preprocessed image blocks corresponding to the remaining bi-temporal remote sensing images and their labels form a verification sample set, and the other half form a test sample set, where N ≥ 3000;

[0010] (2) Construct a lightweight remote sensing image change detection network model based on binary neural network and large kernel strip convolution:

[0011] A lightweight remote sensing image change detection network model O is constructed, which includes two branch networks with the same structure and arranged in parallel, and a cascaded subtractor and a depth-separable convolutional detection head; the two branch networks both include a cascaded large-core strip convolutional network and a binary weight-sharing ResNet18 network; the large-core strip convolutional network includes a cascaded channel mapping module, a feature extraction module, and a maximum pooling module;

[0012] (3) Iterative training of lightweight remote sensing image change detection network model:

[0013] The lightweight remote sensing image change detection network model O is iteratively trained through the training sample set and the verification sample set to obtain the trained detection network model O t ;

[0014] (4) Obtain lightweight remote sensing image change detection results:

[0015] The test sample set is used as the trained detection network model O * The input is forward propagated to obtain the change detection result corresponding to each test sample.

[0016] Compared with the prior art, the present invention has the following advantages:

[0017] 1. In the process of iteratively training the lightweight remote sensing image change detection network model and obtaining the change detection results, the present invention uses the ResNet18 network in the two branch networks to perform deep feature extraction on the dual-temporal feature map respectively, and can quantize the weights and activations to +1 and -1, thereby achieving 32 times of memory savings and 58 times of CPU acceleration, avoiding the defect of a large number of parameters caused by the prior art using 32-bit floating-point numbers to store parameters, and effectively improving the detection efficiency.

[0018] 2. The present invention uses a large-core strip convolutional network in two branch networks to perform preliminary feature extraction on the dual-temporal training samples, efficiently captures the context and directional features of the training samples, significantly improves the detection accuracy and sensitivity to subtle changes, avoids the defects of the prior art in that it is difficult to capture large-scale context information, is insufficient in recognizing complex structures and large-scale changes, and has low feature extraction efficiency for linear or slender structures (such as roads and plants), and effectively makes up for the lack of accuracy caused by binary quantization. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart for realizing the present invention;

[0020] Figure 2 It is the overall structure diagram of the remote sensing image change detection network model of the present invention. DETAILED DESCRIPTION

[0021] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0022] Reference Figure 1 , the present invention comprises the following steps:

[0023] Step 1) Obtain training sample set and test sample set:

[0024] Preprocess N pairs of bi-temporal remote sensing images containing multiple target categories, and mark the changed areas in the image blocks corresponding to the preprocessed bi-temporal remote sensing images, then form a training sample set with the preprocessed image blocks corresponding to more than half of the bi-temporal remote sensing images and their labels, form a verification sample set with the preprocessed image blocks corresponding to the remaining bi-temporal remote sensing images and their labels, and form a test sample set with the other half, where N≥3000; in this embodiment, N=3000;

[0025] (1a) Preprocessing N pairs of bi-temporal remote sensing images containing multiple target categories;

[0026] (1a1) Collecting original remote sensing images of buildings. Original dual-temporal remote sensing images refer to remote sensing images of the same area or object collected at different time points. Before data preprocessing, the original images often contain sensor noise, and the radiation intensity of the images may be inconsistent;

[0027] (1a2) remote sensing image preprocessing, filtering each pair of bi-temporal remote sensing images respectively, performing radiometric correction on each pair of bi-temporal remote sensing images after filtering, and then performing spatial position alignment on each pair of bi-temporal remote sensing images after radiometric correction;

[0028] (1a3) cropping the bi-temporal remote sensing image into n pairs of bi-temporal remote sensing image blocks of size H×W. In this embodiment, n=10000, and the cropping size is 256×256;

[0029] (1b) Preprocessing the labeling of the changed regions in the image patches corresponding to the bi-temporal remote sensing images: threshold-based detection, change vector analysis (CVA), and principal component analysis (PCA) are used to automatically identify the changed regions and generate a preliminary label map, which is then fine-tuned using manual adjustments;

[0030] (1c) Based on the set of dual-temporal remote sensing images and change label maps, a dataset is constructed, which includes the first-phase image, the second-phase image and the related label maps;

[0031] (1c1) Organize the images and ensure consistent naming; then create a data set structure to classify and store the first phase images, second phase images, and label images;

[0032] (1c2) The preprocessed image blocks and their labels corresponding to more than half of the bi-temporal remote sensing images form a training sample set, the preprocessed image blocks and their labels corresponding to the remaining bi-temporal remote sensing images form a verification sample set, and the other half form a test sample set. In this embodiment, the data set is randomly divided into 60% training sample data set, 20% verification sample data set and 20% test sample data set;

[0033] (1c3) Generate metadata file record path to ensure that the dataset can be used for subsequent analysis and model training;

[0034] (1d) Input the generated data set into the data enhancement module and set the corresponding data enhancement parameters. To ensure the generalization ability of the model, only random horizontal flipping, random vertical flipping and random blur operations are enabled in the data enhancement process. These enhancement strategies can effectively improve the robustness of the model without introducing too many complex transformations, while avoiding excessive changes to the original image structure. Suppose each pair of original dual-temporal remote sensing images is input1, input2:

[0035] (1d1) Randomly flip the original images input1 and input2 horizontally: When random horizontal flipping is applied, the image is horizontally flipped with a certain probability (50% in this embodiment), which means that the input image has a 50% probability of being mirrored, thereby increasing the diversity of the image; it is enabled by the with_random_hflip=True parameter; for some object categories (such as vehicles, animals, etc.), horizontal flipping does not change their category, so it can effectively improve the model's recognition ability for these objects;

[0036] (1d2) randomly flipping the image processed by random horizontal flipping vertically: When random vertical flipping is applied, the image is vertically flipped with a certain probability (50% in this embodiment), which is enabled by the with_random_vflip=True parameter. In some application scenarios (such as satellite images or certain types of traffic signs), vertical flipping will not change the content of the image, so this can further increase the robustness of the model;

[0037] (1d3) The image that has been randomly flipped vertically is Gaussian blurred to simulate motion blur during shooting, which helps the training model better adapt to different shooting conditions. The image obtained after the above data enhancement is set as

[0038] Step 2) Construct a lightweight remote sensing image change detection network model O based on binary neural network and large kernel strip convolution:

[0039] Build as Figure 2The remote sensing image change detection network model O shown in the figure includes two branch networks with the same structure and arranged in parallel, and a lightweight remote sensing image change detection network model O of a cascaded subtractor and a depth-separable convolution detection head; wherein the two branch networks both include a cascaded large-core strip convolution network and a binary weight-sharing ResNet18 network; the large-core strip convolution network includes a cascaded channel mapping module, a feature extraction module and a maximum pooling module;

[0040] (2a) Constructing a large-core strip convolutional network:

[0041] (2a1) Use the channel mapping module to map the channels and complete downsampling to obtain a low-resolution bi-temporal image I 1_new and I 2_new , the size becomes

[0042] (2a2) The bi-temporal image I 1_new and I 2_new Enter the large kernel convolution module. The large kernel convolution module here is actually a multi-scale convolution parallel module, which contains three key parts:

[0043] (2a21) First, by aggregating local information through depth-wise convolution, the detailed features of the image can be effectively extracted;

[0044] (2a22) Secondly, the module uses multi-branch depth-wise strip convolution to capture multi-scale contextual information, which is particularly important for processing complex structures and changes;

[0045] (2a23) Finally, a convolution with a kernel size of 1 is used to model the relationship between different channels to achieve the fusion and optimization of information from each channel;

[0046] (2a3) Let the feature extracted by depthwise convolution be F i (i=1,2), then:

[0047] F i =Conv(I i_new ),(i=1,2);

[0048] (2a4) The feature F extracted by depth-wise convolution i (i=1,2) is input into the depthwise strip convolution module, where the convolution layer has three branches, each of which uses two depthwise strip convolutions to approximate the large-kernel standard depthwise convolution; the kernel sizes of the three branches are set to 7, 11, and 21 respectively:

[0049]

[0050] Scalej It is a skip connection, which means that in this network structure, the input is directly passed to the subsequent layer through an identity connection without additional transformation to ensure that the information can cross multiple levels without being destroyed by the intermediate layers;

[0051] There are two main reasons why this embodiment uses deep strip convolution: first, strip convolution has the advantage of being lightweight; compared to standard convolution, strip convolution can achieve similar effects by decomposing 7×7 convolution into two groups of convolution operations, 7×1 and 1×7, while significantly reducing the amount of calculation; second, strip convolution is particularly effective in change detection tasks, especially when dealing with slender strip-shaped objects such as people and telephone poles, which can make up for the shortcomings of traditional grid convolution and better extract these strip-shaped features;

[0052] In addition, the convolution kernel size gradually increases, which can capture a wider range of contextual information and help obtain larger-scale global features, thereby improving the accuracy of change detection;

[0053] (2a5) The last step of the convolution module is to weight the output of the 1×1 convolution and the initial input of the module. This residual connection can better preserve the original feature information, enhance the feature transfer capability, and ultimately improve the model's expressiveness and detection accuracy. Let the output of this module be Out i (i=1,2);

[0054] Out i =Conv 1×1 (F i ),i=1,2;

[0055] (2a6) Multiply the retained input features and the large kernel convolution output features element by element, and weight the original features to highlight important feature information:

[0056]

[0057] Through weighted operations, the model can amplify those easily ignored strip-shaped target features in the feature space, while suppressing unimportant or irrelevant information. This mechanism enables the network to focus more on the most critical parts of the image, such as edges, object contours, or strip-shaped objects, thereby improving the detection accuracy and recognition ability of these important features;

[0058] (2a7) constructing a maximum pooling module to fuse the extracted features of different scales to obtain a feature map;

[0059] (2b) Construct a binary quantized ResNet18 and perform 1-bit quantization on it, including the following steps:

[0060] (2b1) For any convolutional layer, the input is defined as The weight is The output of the convolutional layer is To distinguish, let the full-precision weight and full-precision input be ω f ,I f , the binarization weight and binarization input are ω b ,I b , and the possible values ​​are only +1 and -1;

[0061] (2b2) ​​For the full-precision convolution layer, the output result is:

[0062]

[0063] in, It is a standard convolution operation;

[0064] (2b3) Next, we use different methods to quantize the weights and activations. For the weights, we go through the following steps:

[0065] (2b31) Scale the weights and use the weight normalization method to get the full precision weight ω of the input. f After this process, the obtained standardized weight is expressed as ω f '

[0066]

[0067] Among them, is the mean of the weights and is the scaled standard deviation;

[0068] μ=mean(ω f ,dim=[1,2,3],keepdim=True)

[0069]

[0070] First, the weight ω f Perform mean removal processing, by subtracting the mean of the weights, so that the mean of the processed weights is 0, which helps to improve training stability and convergence speed, because mean removal can reduce bias;

[0071] Then calculate the weight ω f The standard deviation of the weights is calculated by adding a small constant 1e-5 to avoid the denominator being 0. This step ensures that there will be no calculation errors due to the variance of the weights being zero during the standardization process.

[0072] Finally, the standard deviation is scaled to a suitable range by dividing it by a constant. This scaling is to ensure that the subsequent training process is within the appropriate numerical range and improve the learning ability of the model;

[0073] (2b32) The weights are clipped to limit them to [Q τ ,-Q τ ], the processing steps are as follows:

[0074] First calculate the expected value of the weight:

[0075]

[0076] where is the K weight ω′ f The number of elements in ;

[0077] Then calculate the quantization threshold Q τ :

[0078] Q τ =-E ω ·log(2-2τ)

[0079] For weight ω′ f Crop it to limit it to [Q τ ,-Q τ ] in the range:

[0080] ω′ f = clamp(ω′ f ,-Q τ ,Q τ )

[0081] Among them, clamp(x,min,max) is a function that limits the input x to between the minimum value min and the maximum value max.

[0082] In this way, unimportant weights can be cut, effectively reducing the amount of model calculations and matrix operation time;

[0083] (2b33) Binarize the processed weights during forward propagation. The processing steps are as follows:

[0084]

[0085] (2b34) During back propagation, due to the non-differentiability of the sign function, ω b The gradient of cannot be calculated directly, so the straight-through estimator (STE) is used to approximate the gradient calculation. The process of calculating the partial derivative of the loss function and parameters during back propagation is as follows:

[0086]

[0087] in, represents the loss function;

[0088] (2b4) For activation, the following steps are required:

[0089] (2b41) During training, activations should be standardized to maintain a stable scale of activations and prevent large differences in data amplitudes. However, during inference and testing, activations should not be standardized. The steps for standardization are as follows:

[0090]

[0091] Where, ∈ is a smoothing term, which is 1e-5 in this embodiment, Var(I f ) represents the variance of the full-precision activation in dimensions [1, 2, 3]:

[0092]

[0093] (2b42) Binarize the processed activations, which is done during the forward propagation. The processing steps are as follows:

[0094]

[0095] (2b43) During back propagation, due to the non-differentiability of the sign function, I b The gradient of cannot be calculated directly, so the polynomial method is used. The process of calculating the partial derivative of the loss function and parameters during back propagation is as follows:

[0096]

[0097] If we split it, we have:

[0098]

[0099] (2b5) After completing the binarization of weights and activations, the convolution operation can be replaced by bitwise operations:

[0100]

[0101] Among them, represents the dot product operation, including XNOR and POPCOUNT;

[0102] (2b6) In previous studies, many binarized networks usually multiply the binarized weights and activations by a scaling factor in order to reduce quantization errors. Although the weights and activations remain binary, they are actually represented by a small number of floating-point numbers. Although this method avoids a significant decrease in accuracy to a certain extent, it does not achieve a complete binarization effect.

[0103] This embodiment improves this method: after completing the binary convolution operation, a scaling factor α is introduced, the output result is multiplied by the factor and returned to the next layer of the network. The convolution layer only has +1 and -1 involved in the operation, which not only improves the input accuracy of the next convolution layer, but also speeds up the entire convolution operation. Through this optimization strategy, the efficiency of the calculation is guaranteed, the accuracy is maintained, and better performance is achieved;

[0104]

[0105] (2b7) Use the quantized convolutional layer as the basic component to construct ResNet18 as the network layer with two weights shared, denoted as CNN_Backbone;

[0106] (2b8) After the image passes through two feature extraction networks ResNet18 with shared weights, two feature maps X with the same dimension are obtained. i (i=1,2):

[0107] X i =CNN_Backbone(I i ′),i=1,2

[0108] (2c) Constructing a detection head based on depthwise separable convolution includes the following steps:

[0109] (2c1) upsampling the change feature map to obtain a change feature map with the same size as the input image;

[0110] (2c2) The change map is input into the prediction head, which consists of a set of depthwise convolutions:

[0111] (2c21) The first step is to perform a depthwise convolution, which only performs convolution operations on each channel separately:

[0112]

[0113] Among them, y i is the output feature, x i,k is the input feature, w i,k It is the convolution kernel of each channel, and the operation is completed through depth convolution;

[0114] (2c22) The second step is to perform point-by-point convolution, using a convolution kernel of size 1 for each output channel to combine information from different channels and complete the task of mapping in_channels to out_channels:

[0115]

[0116] Where M is the number of channels, K i It is a convolution unit with a convolution kernel size of 1;

[0117] (2c3) Using depthwise separable convolution can effectively reduce the computational complexity. The computational complexity of standard convolution is O(C in ×C out ×K 2 ), and this embodiment introduces a depth-separable convolution module, the computational complexity of which is O(C in ×K 2 +C in ×C out ), which greatly reduces the amount of calculation.

[0118] Step 3) Iteratively train the lightweight remote sensing image change detection network model:

[0119] (3a) The number of initialization iterations is t, the maximum number of iterations is T, T ≥ 200, and the convolution kernel weight and bias parameters of the current change detection network model are a t , and let t = 1;

[0120] (3b) The large core strip convolution network in the first branch network performs preliminary feature extraction on the training samples of the first time state, and the ResNet18 network performs deep feature extraction on the feature map obtained by feature extraction to obtain the feature map X of each training sample of the first time state. 1 At the same time, the large core strip convolution network in the second branch network performs preliminary feature extraction on the training samples of the second time state, and the ResNet18 network performs deep feature extraction on the feature map obtained by feature extraction to obtain the feature map X of each training sample of the second time state. 2 ;

[0121] (3b1) a channel mapping module, mapping the channel of each training sample of the first temporal state to a high dimension, and downsampling the mapped image;

[0122] (3b2) Feature extraction module, which extracts features of different scales for each downsampled image and captures rich spatial features of different scales;

[0123] (3b3) The maximum pooling module performs feature fusion on the features of different scales extracted to obtain a feature map;

[0124] (3c) Subtractor for feature map X 1 With X 2 Perform differential operation: The detection head obtains the initial change map X from the differential operation new =X 1 -X 2 Perform detection and obtain the prediction result y of change detection;

[0125] (3d) Through the real label Calculate the cross entropy loss value L of the change detection network model O with the prediction result y ce And dice loss loss value L dice ; and through L ce and L dice Calculate the total loss value Loss of O; then use the adam optimizer and use Loss to optimize a t Update to obtain the change detection network model O of this iteration t ;

[0126] (3d1) The change detection problem is a common binary classification problem, so the cross entropy loss formula for the change detection problem is:

[0127]

[0128] in, is the true label;

[0129] The cross entropy loss function is used to measure the difference between the probability distribution predicted by the model and the true category. The smaller the value, the closer the model prediction is to the true value.

[0130] (3d2) Class imbalance loss (dice loss) is an indicator that measures the similarity between two sample sets. The larger the value, the closer the prediction is to the true label. In order to optimize the network, the loss function returns 1 minus the Dice coefficient, so as to maximize the Dice coefficient by minimizing the loss:

[0131]

[0132] (3d3) by L ce and L dice Calculate the total loss value Loss of O:

[0133] Loss = L ce +λ×L dice

[0134] Among them, λ represents the weighting coefficient;

[0135] (3d4) Update the model parameters:

[0136]

[0137]

[0138] m t =β1m t-1 +(1-β1)g t

[0139]

[0140] Among them, α t Represents the convolution kernel weight and bias parameters of the current round, α t+1 represents the convolution kernel weight and bias parameter of the next round, η represents the learning rate, is the modified first-order momentum estimate, is the exponentially weighted moving average of the modified squared gradient, m t is a momentum estimate, β1 is the exponential decay rate of the first-order momentum, β2 is the exponential decay rate of the second-order moment estimate, and v t is the exponentially weighted moving average of the squared gradient

[0141] (3e) Calculate the change detection network model O of this iteration through the validation set samples t The F1_Score index is used to compare the index with the corresponding change detection network model O t Save it and let the best model O*=O t , and save its corresponding F1_Score;

[0142] (3f) Determine whether t=T is true. If so, obtain the trained detection network model O * Otherwise, let t = t + 1, O = O t , and execute step (3b);

[0143] (3g) After each round of training, the change detection network model O of the current iteration is calculated using the validation set samples. t The F1_Score indicator of the current round is compared with the best F1_Score indicator saved previously; if the F1_Score of the current round is higher than the previous best value, the current model O t Set as the best model O * , and save the best model and its corresponding F1_Score; otherwise, keep the original best model and F1_Score unchanged;

[0144] (3g1) The prediction results are quantitatively analyzed using five commonly used evaluation indicators for change detection: these indicators include: Precision, Intersection over Union (IoU), Recall, F1-score, and overall accuracy, TP (True Positive), TN (True Negative), FP (False Positive), and FN (False Negative); the higher the values ​​of these evaluation indicators, the better the prediction results of the model;

[0145] TP: The number of pixels predicted to be positive and actually positive (i.e., the model correctly detected the actual change area);

[0146]

[0147] TN: The number of pixels predicted to be negative and actually negative (i.e., the model correctly detected areas that did not actually change);

[0148]

[0149] FP: the number of pixels predicted as positive but actually negative (i.e. the model mistakenly detects unchanged areas as changed areas);

[0150]

[0151] FN: The number of pixels predicted as negative but actually positive (i.e., the model failed to detect the actual change area);

[0152]

[0153] Precision: The ratio of pixels that are actually positive among all pixels predicted to be positive. The higher the precision, the lower the false positive rate of the model. The letter P is used to represent this in the present invention.

[0154]

[0155] Recall: The proportion of pixels that are correctly predicted to be positive among all pixels that are actually positive. The higher the recall, the lower the model's missed detection rate. This is represented by the letter R in this paper:

[0156]

[0157] F1_Score (F1 score): The harmonic mean of precision and recall, which is a comprehensive indicator that balances precision and recall:

[0158]

[0159] Overall Accuracy: The ratio of the number of pixels predicted correctly by the model to the total number of pixels. This is represented by the letters OA in the present invention:

[0160]

[0161] Step 4) Obtain lightweight remote sensing image change detection results:

[0162] The test sample set is used as the trained detection network model O *The input is forward propagated to obtain the change detection result corresponding to each test sample.

[0163] The following simulation experiments are used to further illustrate the technical effects of the present invention:

[0164] 1. Simulation conditions and contents:

[0165] This experiment uses three datasets, namely LEVIR-CD Dataset, CDD Dataset, and WHUDataset. These three datasets are introduced one by one below.

[0166] LEVIR-CD (Large-scale Earth Vision Infrared Remote Change Detection Dataset) is a widely used remote sensing change detection dataset that focuses on change detection of high-resolution urban buildings, covering scenes such as building expansion. The dataset contains 637 pairs of remote sensing images with a resolution of 1024x1024. Each pair of images is taken at different time points in the same area to capture changes in buildings.

[0167] CDD Dataset (Change Detection Dataset) is a public dataset for remote sensing change detection. It contains multiple pairs of remote sensing images with a resolution of 256x256. The image types cover a variety of different geographical scenes (such as cities, rural areas, etc.), providing a diverse training and testing platform. The image resolution is 256x256 and the data size is moderate. The image types of this dataset are more diverse, including changes in different geographical environments. It covers a variety of scenes and is more suitable for a wide range of change detection tasks, including vegetation, disasters, urbanization, etc. It is a general change detection dataset.

[0168] WHU Dataset (Wuhan University Building Dataset) is a dataset released by Wuhan University specifically for building extraction and change detection. It includes multiple satellite images with different resolutions, most of which are high-resolution images. It is mainly used to evaluate the change detection task of buildings, focusing on the extraction and change detection of buildings, providing higher resolution and suitable for building detail detection.

[0169] The simulation experiment was conducted on a server with NVIDIA A100 GPU, Ubuntu 18.04 operating system, PyTorch deep learning framework, and Python 3.10 programming language.

[0170] 2. Analysis of simulation results

[0171] The change detection network proposed in the present invention is compared with the existing "lightweight remote sensing change detection method and system based on dual-time remote sensing images". The results are shown in Table 1.

[0172] Experiments have shown that the accuracy of the present invention on three existing public data sets is improved compared with the existing technology. At the same time, the number of parameters is greatly reduced compared with the existing technology, and the number of floating-point operations is compressed 80 times compared with the existing technology, achieving memory savings and CPU acceleration, avoiding the defect of the existing technology of using 32-bit floating-point numbers to store parameters resulting in a large number of parameters, and effectively improving the detection efficiency.

[0173]

Claims

1. A lightweight remote sensing image change detection method based on binary neural network and large kernel strip convolution, characterized in that: The steps include: (1) Obtain training sample set, verification sample set and test sample set: Preprocess N pairs of bi-temporal remote sensing images containing multiple target categories, and annotate the changed areas in the image blocks corresponding to the preprocessed bi-temporal remote sensing images. Then, the preprocessed image blocks corresponding to more than half of the bi-temporal remote sensing images and their labels form a training sample set, and the preprocessed image blocks corresponding to the remaining bi-temporal remote sensing images and their labels form a verification sample set, and the other half form a test sample set, where N ≥ 3000; (2) Construct a lightweight remote sensing image change detection network model based on binary neural network and large kernel strip convolution: A lightweight remote sensing image change detection network model O is constructed, which includes two branch networks with the same structure and arranged in parallel, and a cascaded subtractor and a depth-separable convolutional detection head; the two branch networks both include a cascaded large-core strip convolutional network and a binary weight-sharing ResNet18 network; the large-core strip convolutional network includes a cascaded channel mapping module, a feature extraction module, and a maximum pooling module; (3) Iterative training of lightweight remote sensing image change detection network model: The lightweight remote sensing image change detection network model O is iteratively trained through the training sample set and the verification sample set to obtain the trained detection network model O t ; (4) Obtain lightweight remote sensing image change detection results: The test sample set is used as the trained detection network model O * The input is forward propagated to obtain the change detection result corresponding to each test sample.

2. The method according to claim 1, characterized in that The preprocessing of N pairs of bi-temporal remote sensing images containing multiple target categories described in step (1) is implemented by: Each pair of bi-temporal remote sensing images is filtered separately, and each pair of bi-temporal remote sensing images after filtering is radiometrically corrected. Then, each pair of bi-temporal remote sensing images after radiometric correction are spatially aligned, and each pair of aligned bi-temporal remote sensing images is cropped into multiple image blocks.

3. The method according to claim 1, characterized in that The lightweight remote sensing image change detection network model O described in step (2), wherein: ResNet18 network, including stacked convolutional layers, residual blocks, global average pooling layers, and fully connected layers; The large-core strip convolutional network has a channel mapping module implemented by two convolutional layers; a feature extraction module including multiple parallel large-core strip convolutional layers; and a maximum pooling module including a stacked maximum pooling layer and an activation function PReLU layer. The prediction head consists of stacked depthwise convolutional layers, batch normalization layers, activation function PReLU layers, and point-wise convolutional layers.

4. The method according to claim 1, characterized in that: The iterative training of the lightweight remote sensing image change detection network model described in step (3) is implemented as follows: (3a) The number of initialization iterations is t, the maximum number of iterations is T, T ≥ 200, and the convolution kernel weights and bias parameters of the current change detection network model are a t , and let t = 1; (3b) The large core strip convolution network in the first branch network performs preliminary feature extraction on the training samples of the first time state, and the ResNet18 network performs deep feature extraction on the feature map obtained by feature extraction to obtain the feature map X of each training sample of the first time state. 1 At the same time, the large core strip convolution network in the second branch network performs preliminary feature extraction on the training samples of the second time state, and the ResNet18 network performs deep feature extraction on the feature map obtained by feature extraction to obtain the feature map X of each training sample of the second time state. 2 ; (3c) Subtractor for feature map X 1 With X 2 Perform differential operation; the detection head obtains the preliminary change map X from the differential operation new =X 1 -X 2 Perform detection and obtain the prediction result y of change detection; (3d) Through the real label Calculate the cross entropy loss value L of the change detection network model O with the prediction result y ce And diceloss loss value L dice ; and through L ce and L dice Calculate the total loss value Loss of O; Then use the adam optimizer and use Loss to optimize a t Update to obtain the change detection network model O of this iteration t ; (3e) Calculate the change detection network model O of this iteration through the validation set samples t F1_Score indicator F1 * , and for F1 * And the corresponding change detection network model O t Save and set the best model O*=O t ; (3f) Determine whether t=T is true. If so, obtain the trained detection network model O * , otherwise, let t = t + 1, O = O t , and calculate the change detection network model O of the current iteration through the validation set samples t F1_Score indicator F1 t , judge F1 t ≥F1 * Is it true? If so, change O t As the best model O * , and let F1 * =F1 t Otherwise, keep O * and F1 * No change; then execute step (3b).

5. The method according to claim 4, characterized in that The large-core convolutional network in the first branch network described in step (3b) performs feature extraction on the training sample of the first temporal state, and the implementation steps are: (3b1) The channel mapping module maps the channel of each training sample of the first temporal state to a high dimension and downsamples the mapped image; (3b2) The feature extraction module extracts spatial features of different scales for each downsampled image; (3b3) The maximum pooling module fuses the extracted spatial features of different scales to obtain a feature map.

6. The method according to claim 4, characterized in that The cross entropy loss value L of the change detection network model O described in step (3d) ce And dice loss loss value L dice , and the total loss value Loss of O;, the calculation formulas are: Loss=L ce +λ×L dice Among them, ∈ represents the smoothing term and λ represents the weighting coefficient.

7. The method according to claim 6, characterized in that The step (3d) described in α t Update, the update formula is: m t =β1m t-1 +(1-β1)g t Among them, α t , α t+1 Respectively represent the convolution kernel weights and bias parameters of the tth and t+1th iterations, η represents the learning rate, is the modified first-order momentum estimate, is the exponentially weighted moving average of the modified squared gradient, m t is a momentum estimate, β1 is the exponential decay rate of the first-order momentum, β2 is the exponential decay rate of the second-order moment estimate, and v t is the exponentially weighted moving average of the squared gradients.

Images