Lightweight method based on remote sensing image change detection network

Abstract

The application discloses a kind of light-weighted methods based on remote sensing image change detection network, solve the technical problem that network equipment end deployment memory occupancy is too large, realize including, obtain the remote sensing image pair to be detected;Change detection teacher network is constructed;Change detection teacher-student joint network is constructed and joint loss function is obtained;Quantization student network weight;Lightweight residual learning module is constructed;The fusion feature map of improved student network is obtained;Lightweight remote sensing image change detection network is obtained.The application adopts teacher-student joint knowledge distillation network structure, defines joint loss function in knowledge distillation, constructs lightweight residual learning module to obtain fusion feature map, realizes the light-weight of change detection network.The application effectively reduces the memory occupancy of remote sensing image change detection network, reduces redundant information, improves calculation speed, can more efficiently carry out equipment end deployment, while improving calculation precision and detection efficiency, widely used in image change detection in image processing.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology and mainly relates to model compression of change detection networks. Specifically, it is a lightweight method based on remote sensing image change detection networks, which can effectively reduce the computational load of remote sensing image change detection networks while improving detection accuracy. In practical applications, it can effectively reduce the demand for hardware equipment and reduce inference time, and can be widely used in image change detection in image processing. Background Technology

[0002] Image change detection aims to analyze the differences in imaging of the same area at different points in time. With the continuous collection and development of high-resolution remote sensing images, change detection has been widely applied in many fields, including vegetation cover analysis, urban planning, street view image analysis, and medical diagnosis. Traditional change detection methods can be divided into pixel-based change detection and object-based change detection. In recent years, deep learning technology has overcome the bottleneck of traditional change detection algorithms in feature extraction; convolutional neural networks now possess the ability to automatically extract complex high-level features to generate robust change detection results.

[0003] While change detection algorithms based on deep neural networks perform well, these high-performance deep neural networks typically have a large number of parameters and extremely high computational complexity. This makes these algorithms heavily reliant on high-performance hardware such as GPUs. However, in real-world applications, device computing resources are limited, creating a serious contradiction between complex models and limited computing resources. In 2020, Jie Chen et al. published a paper in IEEE entitled "DASNet: Dual-Attentive Fully Convolutional Siamese Networks for Change Detection in High-Resolution Satellite Images," which disclosed a change detection method based on dual-attention fully convolutional Siamese networks. This method directly measures change by learning implicit metrics, uses spatial attention and channel attention to obtain better feature representations, and uses a specific loss function to balance the influence of changed and invariant regions on the network. Although it performs well in change detection, there are still cases of missed or false detections, and its large neural network size limits its deployment on resource-constrained devices.

[0004] Current mainstream model compression methods include sparsification, numerical quantization, and knowledge distillation. Model sparsification reduces redundant parameters in the network model by removing some nodes, thus reducing the number of parameters and computational cost. Numerical quantization compresses the model by reducing the weights of the FP32 network model to numerical formats such as INT8, accelerating network inference and saving memory. Knowledge distillation can extract a scaled-down model configured for the application from a complex model, reducing the network structure while retaining the knowledge within the network.

[0005] Although existing model compression methods have achieved good results in related fields, they still have shortcomings. Many existing compressed networks can hardly avoid the problem of accuracy reduction compared with the baseline full-precision network; and low-precision networks usually lose effective information; while weight sparsification methods are unstructured sparsity, and their irregular computational characteristics are not conducive to data access and large-scale parallel computing on computing devices, and are not conducive to on-device deployment. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings and problems of the existing technology by proposing a lightweight method for remote sensing image change detection networks that can significantly improve computing speed while effectively ensuring network accuracy. This method is intended to solve the technical problems of excessively large network models and difficult deployment on the device side in the existing technology.

[0007] This invention is a lightweight method based on a remote sensing image change detection network, characterized by employing a network compression method using a teacher-student collaborative knowledge distillation network, comprising the following steps:

[0008] (1) Obtain the remote sensing image pairs to be detected: Select the remote sensing image pairs I1 and I2 with size H×W×C from the remote sensing image change detection dataset, where H, W, and C are the height, width, and number of channels of the remote sensing image pairs I1 and I2 to be detected, respectively, with H>0, W>0, and C>0.

[0009] (2) Constructing a change detection teacher network: Using the BESNet change detection boundary extraction constraint Siamese network as the teacher network, the constructed change detection teacher network is iteratively trained to obtain the teacher network weights W. t ;

[0010] (2a) Change Detection Teacher Network: This change detection teacher network is sequentially connected to a Siamese network, a multi-scale boundary extraction module, and a channel shuffling module. The Siamese network consists of a VGG16 and a second-order nonlocal attention layer. The outputs of the two symmetrical branches of the Siamese network are subtracted to obtain a multi-scale feature map. The obtained multi-scale feature map is used as the input to the multi-scale boundary extraction module for boundary information extraction. Finally, the output F of the Siamese network is processed. cThe output F of the multi-scale boundary extraction module b Input the channel shuffling module and obtain the detection result F from the channel shuffling module. f ;

[0011] (2b) Obtaining the teacher network weight W t The remote sensing images to be detected, I1 and I2, are input into the change detection teacher network for iterative training to obtain the trained teacher network weights W. t ;

[0012] (3) Constructing a joint teacher-student network for change detection and obtaining the joint loss function: First, a Siamese network consisting of VGG16 and a second-order nonlocal attention layer is constructed as the student network. The outputs of the two symmetrical branches of the Siamese network are subtracted to obtain the multi-scale feature maps D1 and D2. a Design the joint loss function L(x,W) for knowledge distillation networks. t W s ), with teacher network weight W t As the initial value, L(x,W) t W s Using I1 and I2 as the loss function, the remote sensing image to be detected is input to the change detection teacher-student joint network for iterative training, resulting in the trained student network weights W. s ;

[0013] (4) Quantify student network weights: Calculate the student network weights W s Low-bit quantization is performed, either 8-bit or 16-bit, to obtain the quantized student network weights W. sq ;

[0014] (5) Constructing a lightweight residual learning module: This lightweight residual learning module takes the feature map D1 of the student network as input, and after downsampling, group convolution and convolution operations, obtains the refined feature map F after output. d ;

[0015] (6) Obtain the fusion feature map of the improved student network: Construct the improved student network and quantize the student network weights W. sq Freeze, update only the bias and lightweight residual learning modules, add the lightweight residual learning modules to the student network as the improved student network, and use the feature map F output by the student network and the Siamese network. c The feature map F output by the lightweight residual learning module d The improved student network fusion feature map F is obtained by fusion. s , feature map F s As the output of the improved student network;

[0016] (7) Obtain a lightweight remote sensing image change detection network: Input the remote sensing image to be detected into I1 and I2 and perform iterative training on the improved student network to obtain a trained lightweight network, thereby achieving the lightweighting of the remote sensing image change detection network.

[0017] This invention solves the technical problems of high network complexity, large memory consumption, low computational efficiency, and difficult deployment on the device side in existing change detection networks. While ensuring detection accuracy, it effectively reduces network complexity and improves computational efficiency.

[0018] Compared with existing technologies, the present invention has the following advantages:

[0019] (1) While ensuring detection accuracy, the complexity of the network is effectively reduced: The present invention adopts a teacher-student joint knowledge distillation structure network, with the change detection boundary extraction constraint Siamese network as the teacher network, which effectively extracts the boundary information of the remote sensing image pairs to be detected. By distilling the extracted boundary information knowledge into the Siamese student network, the memory occupation of the multi-scale boundary extraction module and channel shuffling module in the teacher network is reduced, which reduces the hardware configuration requirements of the change detection network in practical applications and solves the problems of excessive memory occupation and difficult deployment on the device side in the prior art. This invention can effectively reduce the network complexity and ensure detection accuracy while compressing the network.

[0020] (2) Design of joint loss function: This invention designs a joint loss function for the teacher-student joint knowledge distillation structure network. Through the joint loss function containing the mean square error function and the structural similarity error loss function, the student network is guided to continuously learn from the teacher network and correct its own parameters. The structural similarity error compares the brightness, contrast and structure of the image. By considering additional contrast and structural cues, it focuses on the feature importance, correlation and spatial correlation in the feature space, which makes the evaluation of the similarity of the overall image more accurate. Moreover, the structural similarity error loss function calculated purely based on the feature level can be directly added to other loss functions, which is simple to operate.

[0021] (3) Significantly reduced complexity and improved detection efficiency: This invention reduces memory consumption caused by intermediate activation during network training by freezing student network weights, and reduces data processing volume and computational complexity by quantifying student network weights, thereby accelerating the computational speed of network inference. Simultaneously, an improved student network is constructed, and the adaptive capability of the improved student network is enhanced by adding a lightweight residual learning module. Furthermore, the detection accuracy is improved by fusing feature maps. This solves the problems of high processing complexity and significant loss of detection accuracy after network compression in existing remote sensing image change detection algorithms. Therefore, this invention significantly reduces the complexity of data processing during remote sensing image change detection, improves detection efficiency, and further enhances detection accuracy. Attached Figure Description

[0022] Figure 1 This is a flowchart of the present invention;

[0023] Figure 2 This is a simulation image of the detection results using the existing technology of a dual-attention fully convolutional Siamese change detection network method based on remote sensing images;

[0024] Figure 3 This is a simulation diagram of the detection results using the method of this invention. Detailed Implementation

[0025] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0026] Example 1: Existing technologies using dual-attention fully convolutional Siamese change detection networks based on remote sensing images suffer from low detection accuracy, numerous false negatives and missed detections, and an inability to extract sufficiently comprehensive boundary information. The detection accuracy based on remote sensing image datasets is only 89%. Furthermore, the complex neural network structure results in slow computation speed, high memory consumption, and demanding hardware requirements for deployment. This invention addresses these issues by conducting comparative analysis and continuous simulation experiments. Utilizing knowledge distillation techniques, a lightweight method based on remote sensing image change detection networks is creatively proposed.

[0027] This invention is a lightweight method based on remote sensing image change detection networks. It employs a teacher-student collaborative network knowledge distillation technique to distill extracted boundary information knowledge into the Siamese student network, reducing network complexity and memory usage. An improved student network is constructed, and through quantization, weight freezing, and the addition of a lightweight residual learning module, the network training speed and detection accuracy are improved. (See also...) Figure 1 , Figure 1 This is a flowchart of the present invention, which includes the following steps:

[0028] (1) Obtain the remote sensing image pairs to be detected: Select remote sensing image pairs I1 and I2 with a size of H×W×C from the remote sensing image change detection dataset. H, W, and C are the height, width, and number of channels of the remote sensing image pairs I1 and I2, respectively. H>0, W>0, and C>0. In this example, the size of the remote sensing image pairs I1 and I2 is 256×256×3. The image size is determined according to the remote sensing image dataset to be detected, or the image size can be adjusted through image preprocessing.

[0029] (2) Constructing a change detection teacher network: Using the BESNet change detection boundary extraction constraint Siamese network as the teacher network, the constructed change detection teacher network is iteratively trained to obtain the teacher network weights W. t .

[0030] (2a) Change Detection Teacher Network: This change detection teacher network is sequentially connected to a Siamese network, a multi-scale boundary extraction module, and a channel shuffling module. The Siamese network consists of a VGG16 and a second-order nonlocal attention layer. The outputs of the two symmetrical branches of the Siamese network are subtracted to obtain a multi-scale feature map. The obtained multi-scale feature map is used as the input to the multi-scale boundary extraction module for boundary information extraction. Finally, the output F of the Siamese network is processed. c The output F of the multi-scale boundary extraction module b Input the channel shuffling module and obtain the detection result F from the channel shuffling module. f ,in:

[0031]

[0032] in This indicates the operation of the teacher's network, Siam Network. This indicates a multi-scale boundary extraction operation. This indicates a channel rinsing operation.

[0033] (2b) Obtaining the teacher network weight W t The remote sensing images to be detected, I1 and I2, are input into the change detection teacher network for iterative training to obtain the trained teacher network weights W. t The number of iterations is set to 200, and the teacher network weights with the highest detection accuracy are saved during the 200 iterations of training.

[0034] (3) Constructing a joint teacher-student network for change detection and obtaining the joint loss function: First, a Siamese network consisting of VGG16 and a second-order nonlocal attention layer is constructed as the student network. The outputs of the two symmetrical branches of the Siamese network are subtracted to obtain the multi-scale feature maps D1 and D2. a Design the joint loss function L(x,W) for knowledge distillation networks. t W s), with teacher network weight W t As the initial value, L(x,W) t W s The loss function is denoted as I1, and the remote sensing images to be detected are input into the joint network for iterative training to obtain the trained student network weights W. s .

[0035] (4) Quantify student network weights: Calculate the student network weights W s Low-bit quantization is performed, either 8-bit or 16-bit, to obtain the quantized student network weights W. sq .

[0036] (5) Constructing a lightweight residual learning module: This lightweight residual learning module takes the feature map D1 of the student network as input, and after downsampling, group convolution and convolution operations, obtains the refined feature map F after output. d .

[0037] (6) Obtain the fusion feature map of the improved student network: Construct the improved student network and quantize the student network weights W. sq Freeze, update only the bias and lightweight residual learning modules, add the lightweight residual learning modules to the student network as the improved student network, and use the feature map F output by the student network and the Siamese network. c The feature map F output by the lightweight residual learning module d The improved student network fusion feature map F is obtained by fusion. s , feature map F s This is the output of the improved student network.

[0038] (7) Obtain a lightweight remote sensing image change detection network: Input the remote sensing image to be detected into I1 and I2 and perform iterative training on the improved student network to obtain a trained lightweight network, thereby achieving the lightweighting of the remote sensing image change detection network.

[0039] This invention addresses the problems of high network memory consumption and demanding hardware requirements in existing technologies. Instead of using common network pruning and compression methods, it employs knowledge distillation technology to construct a teacher-student joint knowledge distillation network. This effectively preserves the boundary information extracted by the teacher network and distills it into the student network, significantly reducing network complexity and memory consumption. Furthermore, it constructs an improved student network and, through the combined application of quantization, weight freezing, and the addition of a lightweight residual learning module, improves network training speed and enhances network detection accuracy.

[0040] Example 2: A lightweight method based on a remote sensing image change detection network. Similar to Example 1, the method for obtaining the teacher network weight W in step (2b) of this invention is as follows. tThe remote sensing images to be detected, pairs I1 and I2, are input into the boundary extraction constraint Siamese teacher network for iterative training. The steps include:

[0041] (2b1) Initialization: Set the number of training iterations to T, the total number of training iterations to Y, and the average crossover ratio to Miou, and let T = 1. In this example, Y = 200, and Miou... best =0, this step is the initialization setting for the teacher network.

[0042] (2b2) ​​Update network parameters: Input the remote sensing image pairs I1 and I2 to be detected into the change detection teacher network to obtain the output F. f Calculate F f The average crossover ratio Miou with the true value GT, if Miou > Miou best Miou best =Miou, and save the current network parameters, where:

[0043]

[0044] Where N c n represents the number of classes. ij This represents the number of pixels that belong to category i but are predicted to belong to category j.

[0045] (2b3) Loop judgment: Determine whether the iteration number T is less than the set value Y. If so, let T = T + 1 and return to step (2b2) ​​to update the network parameters again; otherwise, when the iteration number T equals the set value Y, execute step (2b4) to end the loop and obtain the teacher network parameters saved after training. This step ensures that iterative training can proceed normally.

[0046] (2b4) Obtain parameters: Obtain the teacher network parameters W saved after training. t .

[0047] This invention obtains the trained teacher network parameters W by iteratively training the teacher network. t This step effectively improves the speed of subsequent network iteration training, enabling the boundary information extracted by the teacher network to be more efficiently preserved and distilled into the subsequent student network. If this step is skipped and the teacher-student joint network is trained directly, the training time will increase significantly, and the efficiency of distilling the boundary information extracted by the teacher network into the student network will be lower.

[0048] Example 3: A lightweight method based on a remote sensing image change detection network. Similar to Examples 1-2, the steps in step (3) of constructing a joint teacher-student change detection network and obtaining the joint loss function include:

[0049] (3a) Constructing the student network: Construct a student network consisting of a convolutional neural network VGG16 and a second-order nonlocal attention layer. Obtain multi-scale feature maps D1 and D2 by subtracting the outputs of the two symmetrical branches of the Siamese network. a The student network output is F s ;

[0050]

[0051] in This refers to the Siamese network operation for student networks.

[0052] (3b) Define the joint loss function: In the joint training of the teacher-student distillation network, the joint loss function is defined as follows:

[0053] L(x,W t W s )=αL t (y,F f )+βL s (y,F c )+γL k (F f ,F c )

[0054] Where L t Let L represent the loss function of the teacher network. s Let L represent the student network loss function. k Let α represent the knowledge distillation loss function, where L is the value of L. t (y,F f The proportionality constant β is L s (y,F c The proportionality constant γ is L k (F f ,F c The proportionality coefficients are α = 0.5, β = 0.2, and γ = 0.3 in this example.

[0055] L k =L MSE +L SSIM

[0056] Where L MSE The student network output F s With teacher network output F f The mean squared error loss function, L SSIM The student network output F s With teacher network output F f The structural similarity error loss function.

[0057] This invention constructs a joint teacher-student network for change detection and obtains a joint loss function. This allows for efficient preservation of boundary information extracted by the teacher network. The joint loss function, which includes a mean squared error function and a structural similarity error loss function, requires the student network to continuously learn from the teacher network and correct its own parameters. The structural similarity error loss function considers additional contrast and structural cues, focusing on the importance, relevance, and spatial relevance of features in the feature space. It compares the brightness, contrast, and structure of the image, resulting in a more accurate evaluation of the overall image similarity. Furthermore, the structural similarity error loss function, which is purely calculated at the feature level, can be directly added to other loss functions, making the operation simple.

[0058] Example 4: A lightweight method based on a remote sensing image change detection network. Similar to Examples 1-3, step (5) involves constructing a lightweight residual learning module. The implementation steps include:

[0059] Building a lightweight residual learning module: Taking the feature map D1 of the student network as input, and performing downsampling, group convolution, and convolution operations, the refined feature map F is obtained as the output. d ,in:

[0060] F d =C 1×1 (G 5×5 (Avgpool(D1)))

[0061] Where Avgpool represents the 2×2 average pooling operation, C 1×1 This indicates a convolution operation with 8 output channels and a kernel size of 1×1, G. 5×5 This indicates a group convolution operation with 32 output channels and a kernel size of 5×5.

[0062] This invention downsamples the input feature map in the lightweight residual learning module by using an average pooling layer, which reduces the resolution of the input feature map, decreases the number of intermediate activations in the lightweight residual learning module, and reduces the memory usage caused by the number of intermediate activations. The efficient group convolution operation reduces the number of channels in the input feature map of the lightweight residual learning module, providing a good trade-off between computational speed and memory usage.

[0063] Example 5: A lightweight method based on a remote sensing image change detection network. Similar to Examples 1-4, step (6) involves obtaining the fusion feature map of the improved student network. The implementation steps include:

[0064] (6a) Constructing the improved student network: The feature map F output by the student network Siamese network is used to construct the improved student network. c The feature map F output by the lightweight residual learning module d The feature map F is obtained by fusion. s, feature map F s As the output of the improved student network, the improved network after feature fusion can effectively improve network accuracy and adaptability to different remote sensing image datasets, where:

[0065]

[0066] in This indicates the Siam Network operation after freezing the quantized weights.

[0067] (6b) Obtain the fusion feature map of the improved student network: Calculate the fusion feature map F s :

[0068]

[0069] Among them, F cn With F dn F c With F d The nth feature map.

[0070] This invention improves the adaptability of the original student network by obtaining the fusion feature map of the improved student network, and improves detection accuracy and efficiency while achieving network compression.

[0071] The following is a more complete example to illustrate this further:

[0072] Example 6: A lightweight method based on a remote sensing image change detection network. The implementation steps are the same as in Examples 1-5, but include:

[0073] (1) Obtain the remote sensing image pairs to be detected: Select remote sensing image pairs I1 and I2 with size H×W×C from the remote sensing image change detection dataset. H, W, and C are the height, width, and number of channels of the remote sensing image pairs I1 and I2 to be detected, respectively. H>0, W>0, and C>0. In this example, the remote sensing image pairs I1 and I2 to be detected are real remote sensing images that change with the seasons obtained from Google Earth. The size is 256×256×3, and the spatial resolution is 3 to 100 cm / px.

[0074] (2) Constructing a change detection teacher network: Using the BESNet change detection boundary extraction constraint Siamese network as the teacher network, the constructed change detection teacher network is iteratively trained to obtain the teacher network weights W. t .

[0075] (2a) Change Detection Teacher Network: This change detection teacher network is sequentially connected to a Siamese network, a multi-scale boundary extraction module, and a channel shuffling module. The Siamese network consists of a VGG16 and a second-order nonlocal attention layer. The outputs of the two symmetrical branches of the Siamese network are subtracted to obtain a multi-scale feature map. The obtained multi-scale feature map is used as the input to the multi-scale boundary extraction module for boundary information extraction. Finally, the output F of the Siamese network is processed. c The output F of the multi-scale boundary extraction module b Input the channel shuffling module and obtain the detection result F from the channel shuffling module. f ,in:

[0076]

[0077] in This indicates the operation of the teacher's network, Siam Network. This indicates a multi-scale boundary extraction operation. This indicates a channel rinsing operation.

[0078] (2b) Obtaining the teacher network weight W t The remote sensing images to be detected, I1 and I2, are input into the change detection teacher network for iterative training to obtain the trained teacher network weights W. t .

[0079] (2b1) Initialization: Set the number of training iterations to T, the total number of training iterations to Y, and the average crossover ratio to Miou, and let T = 1. In this example, Y = 200, and Miou... best =0.

[0080] (2b2) ​​Update network parameters: Input the remote sensing image pairs I1 and I2 to be detected into the change detection teacher network to obtain the output F. f Calculate F f The average crossover ratio Miou with the true value GT, if Miou > Miou best Miou best =Miou, and save the current network parameters, where:

[0081]

[0082] Where N c n represents the number of classes. ij This represents the number of pixels that belong to category i but are predicted to belong to category j.

[0083] (2b3) Loop judgment: Determine whether T is less than Y. If so, let t = T + 1 and execute step (2b2); otherwise, execute step (2b4).

[0084] (2b4) Obtain parameters: Obtain the teacher network parameters saved after training as W. t .

[0085] (3) Constructing a joint teacher-student network for change detection and obtaining the joint loss function: First, a Siamese network consisting of VGG16 and a second-order nonlocal attention layer is constructed as the student network. The outputs of the two symmetrical branches of the Siamese network are subtracted to obtain the multi-scale feature maps D1 and D2. a Design the joint loss function L(x,W) for knowledge distillation networks. t W s ), with teacher network weight W t As the initial value, L(x,W) t W s The loss function is denoted as I1, and the remote sensing images to be detected are input into the joint network for iterative training to obtain the trained student network weights W. s .

[0086] (3a) Constructing the student network: Construct a student network consisting of VGG16 and a second-order nonlocal attention layer. Subtract the outputs of the two symmetrical branches of the Siamese network to obtain multi-scale feature maps D1 and D2. a The student network output is F s ;

[0087]

[0088] in This refers to the Siamese network operation for student networks.

[0089] (3b) Define the joint loss function: In the joint training of the teacher-student distillation network, the joint loss function is defined as follows:

[0090] L(x,W t W s )=αL t (y,F f )+βL s (y,F c )+γL k (F f ,F c )

[0091] Where L t Let L represent the loss function of the teacher network. s Let L represent the student network loss function. k Let α represent the knowledge distillation loss function, where L is the value of L. t (y,F f The proportionality constant β is L s (y,F c The proportionality constant γ is Lk (F f ,F c The proportionality coefficients are α = 0.5, β = 0.2, and γ = 0.3 in this example.

[0092] L k =L MSE +L sSIM

[0093] Where L MSE It is the mean squared error loss function, L sSIM It is the structural similarity error loss function.

[0094] (3c) Obtain the student weight W t The remote sensing images to be detected, I1 and I2, are input into the teacher-student joint network for iterative training to obtain the trained student network weights W. s .

[0095] (4) Quantify student network weights: Calculate the student network weights W s Low-bit quantization is performed, either 8-bit or 16-bit, to obtain the quantized student network weights W. sq .

[0096] (5) Constructing a lightweight residual learning module: This lightweight residual learning module takes the feature map D1 of the student network as input, and after downsampling, group convolution and convolution operations, obtains the refined feature map F after output. d ;

[0097] Building a lightweight residual learning module: Taking the feature map D1 of the student network as input, and performing downsampling, group convolution, and convolution operations, the refined feature map F is obtained as the output. d ,in:

[0098] F d =C 1×1 (G 5×5 (Avgpool(D1)))

[0099] Where Avgpool represents the 2×2 average pooling operation, C 1×1 This indicates a convolution operation with 8 output channels and a kernel size of 1×1, G. 5×5 This indicates a group convolution operation with 32 output channels and a kernel size of 5×5.

[0100] (6) Obtain the fusion feature map of the improved student network: Construct the improved student network and quantize the student network weights W. sq Freeze, update only the bias and lightweight residual learning modules, add the lightweight residual learning modules to the student network as the improved student network, and use the feature map F output by the student network and the Siamese network.c The feature map F output by the lightweight residual learning module d The improved student network fusion feature map F is obtained by fusion. s , feature map F s As the output of the improved student network, among which:

[0101]

[0102]

[0103] in F represents the Siamese network operation after freezing the quantized weights. cn With F dn F c With F d The nth feature map.

[0104] (7) Obtain a lightweight remote sensing image change detection network: Input the remote sensing image to be detected into I1 and I2 and perform iterative training on the improved student network to obtain a trained lightweight network, thereby achieving the lightweighting of the remote sensing image change detection network.

[0105] This invention primarily addresses the technical problem of excessive memory consumption during network device deployment in existing technologies. The steps include: acquiring pairs of remote sensing images to be detected; constructing a change detection teacher network, training it, and obtaining network weights; constructing a change detection student network, using the obtained teacher network weights as initial values, jointly training it, performing knowledge distillation, and obtaining network weights again; reducing the precision of the student network weights; adding a lightweight residual learning module; freezing the student network weights and obtaining a fused feature map; and training the student network separately to obtain a trained lightweight network. This invention effectively reduces the memory consumption of remote sensing image change detection networks, reduces redundant information, improves computation speed, enables more efficient device deployment, and simultaneously improves computational accuracy.

[0106] The effects of the present invention will be further explained below with reference to simulation experiments.

[0107] Example 7: A lightweight method based on remote sensing image change detection network. Similar to Examples 1-6, this example uses existing technology to simulate the dual-attention fully convolutional Siamese change detection network method based on remote sensing images and the method of the present invention.

[0108] Simulation conditions: The simulation experiment was conducted on an NVIDIA 2080Ti GPU with 11GB of memory. The simulation environment was Python 3.6.7, Torch 1.4.0, and CUDA 10.0.

[0109] Simulation Content and Result Analysis: The simulation experiment of this invention uses the method of this invention and the existing technology of dual-attention fully convolutional Siamese change detection network method based on remote sensing images for simulation. Under the above simulation conditions, two simulation experiments of remote sensing image change detection were carried out respectively. Simulation experiment 1 is the existing technology of dual-attention fully convolutional Siamese change detection network method based on remote sensing images, and simulation experiment 2 is the method of this invention.

[0110] Reference Figure 2 The simulation experiment 1, which uses the existing technology of dual-attention fully convolutional Siamese change detection network based on remote sensing images, is described in detail. Figure 2 (a) is a real remote sensing image of a location that changes with the seasons, obtained from Google Earth. Figure 2 (b) A real remote sensing image of the location at another point in time, obtained from Google Earth, showing seasonal variations. Figure 2 (c) is the ground truth for dataset change detection. Figure 2 (d) To utilize existing technologies, a dual-attention fully convolutional Siamese change detection network method based on remote sensing images is used for... Figure 2 (a) and Figure 2 (b) test results, Figure 2 The gray shaded area in (d) represents the detected area of ​​change.

[0111] Reference Figure 3 The simulation experiment 2 conducted using the method of the present invention will be described in detail. Figure 3 (a) is a real remote sensing image of a location that changes with the seasons, obtained from Google Earth. Figure 3 (b) A real remote sensing image of the location at another point in time, obtained from Google Earth, showing seasonal variations. Figure 3 (c) is the ground truth for dataset change detection. Figure 3 (d) is a simulation experiment result diagram using the method of the present invention. Figure 3 The gray shaded area in (d) represents the detected area of ​​change.

[0112] Compare Figure 2 (d) and Figure 3 (d) It can be seen that, compared with the existing dual-attention fully convolutional Siamese change detection network method based on remote sensing images, the present invention detects the boundary of the road, vehicles, and building change areas in the upper left and right sides more clearly and accurately. Figure 2 (d) The detected change areas have blurred lines, failing to accurately capture the edge features of the change areas, resulting in either missed detections or false detections. Figure 2(d) Inconspicuous and meaningless change markers appeared in the lower left and middle areas, which were overly sensitive to noise or minor changes, and incorrectly marked irrelevant areas as changed areas, affecting the accuracy of detection. In addition, the existing dual-attention fully convolutional Siamese change detection network method based on remote sensing images has large memory consumption, slow calculation speed, and high hardware requirements. The present invention has better detection effect, lower memory consumption, and faster calculation speed than the existing dual-attention fully convolutional Siamese change detection network method based on remote sensing images.

[0113] Example 8: A Lightweight Method Based on Remote Sensing Image Change Detection Network. Similar to Examples 1-7, the simulation conditions and content are the same as in Example 7. Example 7 mainly compared the results of remote sensing image change detection. This example analyzes the method from a more precise data perspective, comparing existing technologies such as the dual-attention fully convolutional Siamese change detection network method based on remote sensing images, the constrained Siamese teacher network for change detection boundary extraction, and the method of this invention. To evaluate the detection performance of the three methods, the evaluation metrics are as follows: Precision (P), Recall (R), and F1 score (F1):

[0114]

[0115] Where, n ij n represents the number of pixels belonging to category i but predicted to be category j. ji This represents the number of pixels belonging to category j but predicted as category i. The detection accuracy values ​​calculated using the three methods are shown in the table below.

[0116] Table 1 Comparison of Change Detection Accuracy between the Method of the Present Invention and Existing Techniques

[0117] Existing technology 89.06 93.70 91.32 90.79 42 Teacher Network 95.20 92.39 93.78 93.30 48 This invention 94.59 93.72 94.15 93.33 23

[0118] As shown in Table 1, compared with the existing dual-attention fully convolutional Siamese change detection network method based on remote sensing images, the method of this invention outperforms the existing technology in five aspects: precision, recall, F1 score, mean crossover ratio, and computation speed. Specifically, the precision is improved by 5.53%, the F1 score by 2.83%, the mean crossover ratio by 2.54%, and the computation time is shortened by 45%. Compared with the teacher network, this invention can guarantee the precision and mean crossover ratio of the change detection network, and slightly improve the recall and F1 score. This invention reduces the memory usage of the teacher network by 2× and improves the computation speed by 2.1×. The method of this invention can effectively improve the detection efficiency of remote sensing image change detection network while ensuring detection accuracy.

[0119] In summary, this invention provides a lightweight method for remote sensing image change detection networks, solving the technical problem of excessive memory consumption on network devices. The method includes: acquiring pairs of remote sensing images to be detected; constructing a change detection teacher network; building a joint teacher-student change detection network and obtaining a joint loss function; quantifying student network weights; constructing a lightweight residual learning module; obtaining the fusion feature map of the improved student network; and obtaining a lightweight remote sensing image change detection network. This invention employs a joint teacher-student knowledge distillation network structure, defines the joint loss function in knowledge distillation, and constructs a lightweight residual learning module to obtain the fusion feature map, thus achieving a lightweight change detection network. This invention utilizes a teacher-student joint knowledge distillation network structure, using a change detection boundary extraction constraint Siamese network as the teacher network. This effectively extracts the boundary information of the remote sensing image pairs to be detected and distills the extracted boundary information knowledge into the Siamese student network, significantly reducing the network complexity and computational complexity of the change detection network, shrinking the network structure, and reducing network memory usage. Secondly, through micro-transfer learning, the weights of the student network are frozen to reduce memory consumption caused by intermediate activation during the training process. Then, the weights of the student network are quantized using eight-bit quantization to further improve the computational speed of the student network. The low-precision quantized student network with frozen weights is then fused with a lightweight residual learning network to improve the network's adaptability and computational efficiency. The network is further compressed to reduce redundant information and improve detection accuracy, ultimately resulting in a lightweight remote sensing image change detection network with superior performance.

Claims

1. A lightweight method based on a remote sensing image change detection network, characterized in that, The network compression method using a teacher-student collaborative knowledge distillation network includes the following steps: (1) Obtain the remote sensing image pairs to be detected: Select the remote sensing image pairs I1 and I2 with size H×W×C from the remote sensing image change detection dataset, where H, W, and C are the height, width, and number of channels of the remote sensing image pairs I1 and I2 to be detected, respectively, with H>0, W>0, and C>0. (2) Constructing a change detection teacher network: Using the BESNet change detection boundary extraction constraint Siamese network as the teacher network, the constructed change detection teacher network is iteratively trained to obtain the teacher network weights W. t ; (2a) Change Detection Teacher Network: This change detection teacher network is sequentially connected to a Siamese network, a multi-scale boundary extraction module, and a channel shuffling module. The Siamese network consists of a VGG16 and a second-order nonlocal attention layer. The outputs of the two symmetrical branches of the Siamese network are subtracted to obtain a multi-scale feature map. The obtained multi-scale feature map is used as the input to the multi-scale boundary extraction module for boundary information extraction. Finally, the output F of the Siamese network is processed. c The output F of the multi-scale boundary extraction module b Input the channel shuffling module and obtain the detection result F from the channel shuffling module. f ; (2b) Obtaining the teacher network weight W t The remote sensing images to be detected, I1 and I2, are input into the change detection teacher network for iterative training to obtain the trained teacher network weights W. t ; (3) Constructing a joint teacher-student network for change detection and obtaining the joint loss function: First, a Siamese network consisting of VGG16 and a second-order nonlocal attention layer is constructed as the student network. The outputs of the two symmetrical branches of the Siamese network are subtracted to obtain the multi-scale feature maps D1 and D2. a Design the joint loss function L(x,W) for knowledge distillation networks. t W s ), with teacher network weight W t As the initial value, L(x,W) t W s The loss function is denoted as I1, and the remote sensing images to be detected are input into the joint network for iterative training to obtain the trained student network weights W. s ; (4) Quantify student network weights: Calculate the student network weights W s Low-bit quantization is performed, either 8-bit or 16-bit, to obtain the quantized student network weights W. sq ; (5) Constructing a lightweight residual learning module: This lightweight residual learning module takes the feature map D1 of the student network as input, and after downsampling, group convolution and convolution operations, obtains the refined feature map F after output. d ; (6) Obtain the fusion feature map of the improved student network: Construct the improved student network and quantize the student network weights W. sq Freeze, update only the bias and lightweight residual learning modules, add the lightweight residual learning modules to the student network as the improved student network, and use the feature map F output by the student network Siamese network. c The feature map F output by the lightweight residual learning module d The improved student network fusion feature map F is obtained by fusion. s , feature map F s As the output of the improved student network; (7) Obtain a lightweight remote sensing image change detection network: Input the remote sensing image to be detected into I1 and I2 and perform iterative training on the improved student network to obtain a trained lightweight network, thereby achieving the lightweighting of the remote sensing image change detection network.

2. The lightweight method based on a remote sensing image change detection network according to claim 1, characterized in that, The step (2b) described in obtaining the teacher network weight W t The remote sensing images to be detected, pairs I1 and I2, are input into the boundary extraction constraint Siamese teacher network for iterative training. The steps include: (2b1) Initialization: Set the number of training iterations to T, the total number of training iterations to Y, and the average crossover ratio to Miou, and let T = 1. In this example, Y = 200, and Miou... best =0; (2b2) ​​Update network parameters: Input the remote sensing image pairs I1 and I2 to be detected into the change detection teacher network to obtain the output F. f Calculate F f The average crossover ratio Miou with the true value GT, if Miou > Miou best Miou best =Miou, and save the current network parameters, where: Where N c n represents the number of classes. ij This represents the number of pixels that belong to category i but are predicted to belong to category j; (2b3) Loop judgment: Determine whether T is less than Y. If so, let T = T + 1 and execute step (2b2); otherwise, execute step (2b4). (2b4) Obtain parameters: Obtain the teacher network parameters saved after training as W. t .

3. The lightweight method based on a remote sensing image change detection network according to claim 1, characterized in that, The steps for constructing the change detection teacher-student joint network and obtaining the joint loss function as described in step (3) include: (3a) Constructing the student network: Construct a student network consisting of VGG16 and a second-order nonlocal attention layer. Obtain the multi-scale feature maps D1 and D2 by subtracting the outputs of the two symmetrical branches of the Siamese network. a The student network output is F s ; in This refers to the Siamese network operation for student networks; (3b) Define the joint loss function: In joint training, the loss function is defined as follows: L(x,W t ,W s )=αL t (y,F f )+βL s (y,F c )+γL k (F f ,F c ) Where L t Let L represent the loss function of the teacher network. s Let L represent the student network loss function. k Let α represent the knowledge distillation loss function, where L is the value of L. t (y,F f The proportionality constant β is L s (y,F c The proportionality constant γ is L k (F f ,F c The proportionality coefficient; L k L MSE +L SSIM Where L MSE The student network output F s With teacher network output F f The mean squared error loss function, L SSIM The student network output F s With teacher network output F f The structural similarity error loss function.

4. The lightweight method based on a remote sensing image change detection network according to claim 1, characterized in that, Step (5) involves constructing a lightweight residual learning module, which includes the following steps: Building a lightweight residual learning module: Taking the feature map D1 of the student network as input, and performing downsampling, group convolution, and convolution operations, the refined feature map F is obtained as the output. d ,in: F d =C 1×1 (G 5×5 (Avgpool(D1))) Where Avgpool represents the 2×2 average pooling operation, C 1×1 This indicates a convolution operation with 8 output channels and a kernel size of 1×1, G. 5×5 This indicates a group convolution operation with 32 output channels and a kernel size of 5×5.

5. The lightweight method based on a remote sensing image change detection network according to claim 1, characterized in that, Step (6) involves obtaining the fusion feature map of the improved student network. The steps to achieve this include: (6a) Constructing the improved student network: The feature map F output by the student network Siamese network is used to construct the improved student network. c The feature map F output by the lightweight residual learning module d The feature map F is obtained by fusion. s , feature map F s As the output of the improved student network, among which: in This indicates the Siam Network operation after freezing the quantized weights; (6b) Obtain the fusion feature map of the improved student network: Calculate the fusion feature map F s : Among them, F cn With F dn F c With F d The nth feature map.

Images