Remote sensing data change detection system

Abstract

The invention relates to a remote sensing data change detection system, and the system comprises a single-flow-based lightweight change detection network model Lite-CDNet which comprises a depth feature fusion module DFFM, a multi-scale feature extractor, and a multi-scale axial attention module MAA; after feature fusion is carried out on an input dual-time-phase image through a depth feature fusion module DFFM, multi-scale feature extraction is carried out by using a lightweight network model ShuffleNetv2, and a multi-scale axial attention module MAA is used to help the model to identify changed regional features and improve feature details; a progressive change balance sampling strategy PCBS is used in the training process to promote model convergence. The method can guarantee the detection precision, and solves a problem of data sample imbalance.

Description

Technical Field

[0001] The present application relates to the field of remote sensing data technology, and in particular to a remote sensing data change detection system. Background Art

[0002] Change detection uses multi-temporal remote sensing data to detect and extract changes in objects within the same geographic area. It is widely used in different remote sensing fields, such as land management, disaster assessment, urban expansion surveys, and ecological monitoring.

[0003] In general, in the research of lightweight change detection models, there is currently an end-to-end single-stream change detection model LightCDNet, which takes dual-phase images as input and fuses the two images at the channel level through a feature fusion module. The number of model parameters is reduced by half compared to the dual-stream structure, so the model is relatively lightweight, but the original image features are seriously lost, resulting in low detection accuracy. Regarding the problem of data imbalance, some researchers have proposed a model and discussed some methods to solve the problem of data imbalance, but they have not considered it from the perspective of nonlinear increase.

[0004] In the existing related technologies, in the lightweight design of change detection models, the early fusion strategy can significantly reduce the size of the model. Its operating principle is to ensure that there is no information loss in the input dual-phase image before the image is input into the model, first connect the two images in series in the channel dimension, merge them into a multispectral image, and then use a single-stream network for subsequent change detection tasks. However, early fusion mixes the features of the two images too early and lacks reasonable guidance for the change features, which ultimately leads to low detection accuracy. In order to improve the accuracy of lightweight change detection models while maintaining information integrity, it is necessary to improve the feature guidance ability of early fusion.

[0005] Most other existing change detection models ignore the problem of positive and negative imbalance in data samples and directly learn features from data, resulting in suboptimal detection accuracy. Some methods have attempted to solve the imbalance problem by designing new loss functions, synthesizing CD samples, increasing the number of positive samples, and using transfer learning, but these solutions have increased the computational complexity and complexity of the model to varying degrees, and have not solved the problem of data sample imbalance from the perspective of adjusting model training. Summary of the invention

[0006] The purpose of this application is to provide a remote sensing data change detection system that can ensure detection accuracy and solve the problem of data sample imbalance.

[0007] In order to achieve the above-mentioned object, the present invention provides a remote sensing data change detection system, including: a single-stream based lightweight change detection network model Lite-CDNet, which includes a deep feature fusion module DFFM, a multi-scale feature extractor and a multi-scale axial attention module MAA;

[0008] After the deep feature fusion module DFFM performs feature fusion on the input dual-phase image, the lightweight network model ShuffleNetv2 is used to extract multi-scale features, and the multi-scale axial attention module MAA helps the model identify the changing regional features and improve the feature details;

[0009] A progressively changing balanced sampling strategy (PCBS) is used during training to promote model convergence.

[0010] In an optional embodiment, the deep feature fusion module DFFM merges the dual-phase images into a multi-spectral feature map after feature fusion, which is then sent to the encoder to generate the fused features into feature maps of different scales through three stages. The first layer of each stage is the Shuffle convolution block with downsampling function in ShuffleNetv2, which is used to achieve downsampling of feature mapping.

[0011] In an optional embodiment, each scale feature can be expressed as {f m |m=1,2,3},where f m Represents the image features at stage m, where m represents three different stages, and each scale f is obtained m It will also serve as the input of the MAA module.

[0012] In an optional implementation, the Lite-CDNet model uses a lightweight FPN as a decoder, which resizes the bottom-level features and adds them to the top-level features.

[0013] In an optional embodiment, the deep feature fusion module DFFM and the encoder both use the lightweight network model ShuffleNetv2 as the backbone network. The Shuffle block with a step size of 1 in ShuffleNetv2 allows the split ratio splitratio to be less than 0.5, uses channel cross operation to optimize the data flow, and removes the ReLU activation function of the first layer of convolution.

[0014] In an optional embodiment, the deep feature fusion module DFFM uses a weight-sharing convolution block to extract shallow features from the input initial image. The convolution block includes a conventional 3×3 convolution, a normalization layer and an activation function. The extracted shallow feature map is input into the ShuffleNetv2 model for further feature extraction. First, a shuffle unit with a step size of 2 downsamples it to reduce the feature map size to half of the original size. Then, a shuffle unit with a step size of 1 is used to divide the feature map into two groups along the channel dimension. One group performs identity mapping, and the other group performs feature extraction through three convolution layers with the same input and output channels, including two 1×1 convolution layers and one 3×3 deeply separable convolution layer. The two groups of feature maps are spliced ​​and shuffled through channels to ensure information interaction. Quick connections are used to fuse the shallow features with the features extracted by the ShuffleNetv2 model. After that, after a temporal attention mechanism, the two groups of feature maps are spliced ​​and shuffled through channels. After adjusting the number of channels through a 1×1 convolution block, deep fusion features are obtained.

[0015] In an optional embodiment, the temporal attention mechanism includes an adaptive temporal attention mechanism, which dynamically adjusts the attention weight according to the input features. The adaptive temporal attention mechanism is an adaptive weight learning module, which can automatically calculate the weight of each feature channel based on the input dual-phase feature map.

[0016] In an optional embodiment, the multi-scale axial attention module MAA includes a multi-way convolution structure MCS and an axial attention mechanism AFM. MCS uses expansion coefficients of different sizes to expand the receptive field of the convolution kernel and uses channel superposition operations to promote the exchange of information between different receptive fields. AFM can improve the effectiveness of feature extraction by assigning different weights to feature maps.

[0017] In an optional embodiment, the feature map input to the multi-scale axial attention module MAA is first integrated with multi-scale features through MCS, and then simultaneously refined through AFM to obtain the output result of the multi-scale axial attention module MAA.

[0018] In an optional embodiment, the progressive change balanced sampling strategy PCBS accelerates the convergence of the model by adjusting the sample distribution at different stages of training, by first training images with changes and then gradually adding images without changes for training.

[0019] The remote sensing data change detection system in the present invention uses a single-stream based lightweight change detection network model Lite-CDNet, uses a deep feature fusion module to efficiently fuse the original feature information of the dual-phase image, uses the improved lightweight shuffleNetv2 as the backbone network for multi-scale feature extraction, and adds multi-scale axial attention to improve the detection performance of the single-stream change detection model.

[0020] Combined with the use of progressively changing balanced sampling strategy PCBS during training to promote model convergence, the convergence of the model is accelerated by adjusting the sample distribution at different stages of training.

[0021] The remote sensing data change detection system of the present invention can ensure detection accuracy and solve the problem of data sample imbalance.

[0022] Other features and advantages of the present application will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 This is a schematic diagram of the structure of the lightweight change detection network model Lite-CDNet in the remote sensing data change detection system of this application;

[0025] Figure 2 A visual diagram of the progressive change balance sampling strategy PCBS;

[0026] Figure 3 This is a schematic diagram of the changing structure of the lightweight network model ShuffleNetv2;

[0027] Figure 4 It is a schematic diagram of the deep feature fusion module DFFM;

[0028] Figure 5 It is a schematic diagram of the multi-way convolution structure MCS;

[0029] Figure 6 Schematic diagram of the axial focus mechanism AFM;

[0030] Figure 7 A visualization diagram of the comparative experiment between Lite-CDNet and mainstream models on the LEVIR-CD dataset;

[0031] Figure 8 A visualization diagram of the comparative experiment between Lite-CDNet and mainstream models on the WHU-CD dataset;

[0032] Fig. 9 A visual diagram of the Lite-CDNet training process with and without PCBS added. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0034] In the description of this application, it should be noted that the terms "inside", "outside", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the application is usually placed when in use, which is only for the convenience of describing this application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on this application. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0035] In the description of this application, it should also be noted that, unless otherwise clearly specified and limited, the terms "disposed" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0036] The remote sensing data change detection system in the present application includes: a single-stream based lightweight change detection network model Lite-CDNet, which includes a deep feature fusion module DFFM, a multi-scale feature extractor, and a multi-scale axial attention module MAA;

[0037] After the deep feature fusion module DFFM performs feature fusion on the input dual-phase image, the lightweight network model ShuffleNetv2 is used to extract multi-scale features, and the multi-scale axial attention module MAA helps the model identify the changing regional features and improve the feature details;

[0038] A progressively changing balanced sampling strategy (PCBS) is used during training to promote model convergence.

[0039] The existing lightweight change detection models mainly lack reasonable guidance of change features, which ultimately leads to low detection accuracy. In order to improve the accuracy of lightweight change detection models while maintaining information integrity, it is necessary to improve the feature guidance capability of early fusion.

[0040] At the same time, as shown in Table 1, the number of changed pixels and unchanged pixels in the images of the four datasets CDD-CD, LEVIR-CD, WHU-CD, and SYSU-CD is counted, and it can be observed that the percentage of pixel changes is very low. Most existing change detection models ignore the problem of positive and negative imbalance in data samples and directly learn features from the data, resulting in suboptimal detection accuracy of the model. Some methods have attempted to solve the imbalance problem by designing new loss functions, synthesizing CD samples, increasing the number of positive samples, and using transfer learning, but these solutions have increased the computational complexity and complexity of the model to varying degrees, and have not solved the problem of data sample imbalance from the perspective of adjusting model training.

[0041] Table 1 Statistics of changed pixels and unchanged pixels in each data set

[0042]

[0043] In view of the shortcomings of the prior art, this application proposes a single-stream lightweight change detection network model Lite-CDNet, which uses a deep feature fusion module to efficiently fuse the original feature information of the dual-phase image, uses the improved lightweight shuffleNetv2 as the backbone network for multi-scale feature extraction, and adds multi-scale axial attention to improve the detection performance of the single-stream change detection model. For details, see Figure 1 .

[0044] like Figure 1 As shown in Figure 1, the end-to-end model Lite-CDNet proposed in this application consists of three main parts: Deep Feature Fusion Module (DFFM), Multiscale Feature Extractor and Multiscale Axial Attention (MAA) module. For a dual-phase image I with an input size of W×H×C t1 and I t2 First, DFFM is used to fuse features and merge them into a multi-spectral feature map. Then it is sent to the encoder to generate feature maps of different scales through three stages. The first layer of each stage is the Shuffle convolution block with downsampling function in the improved ShuffleNetv2, which is used to achieve downsampling of feature maps. Each scale feature can be expressed as {f m|m=1,2,3},where f m represents the image features at stage m, where m represents three different stages. m When the model is trained, it will be used as the input of the MAA module. The MAA module can help the model identify the changed regional features and improve the feature details. The model uses lightweight FPN as a decoder. FPN adjusts the size of the underlying features and adds them to the top-level features, thereby effectively integrating features of different scales. Finally, the classification layer outputs the change detection results.

[0045] At the same time, to address the problem of data sample imbalance, a progressive change balanced sampling (PCBS) strategy is proposed, which accelerates the convergence of the model by adjusting the sample distribution at different stages of training. The specific contents are as follows:

[0046] The basic idea of ​​PCBS is to improve the impact of pixel changes on model training. It solves the sample imbalance problem by first training images with changes and then gradually adding images without changes for training. For ease of understanding, the proposed PCBS strategy is visualized as follows: Figure 2 As shown in the figure, (a)-(f) represent the six training processes of Normal Train, Fixed-X, Linear-Y, Non-Linear-Y, Fixed-XLinear-Y, and Fixed-X Non-Linear-Y respectively. These strategies will be introduced in more detail below.

[0047] Figure 2 (a) Normal Train refers to normal training, where the model learns the data in the entire dataset in each epoch and ignores the imbalance problem of the dataset. Figure 2 (b) Fixed-X means that the model is trained with the changed images in the dataset in the first X epochs to help the model learn the changing features. After X epochs, the entire training dataset is used for training to enhance the robustness of the model, taking into account the data imbalance problem. Figure 2 (c) Linear-Y means that the model learns the features of the changed images in the first epoch, and trains by adding unchanged images through a linear function within Y+1 epochs. After Y+1 epochs, the entire training dataset is used for training, taking into account the data imbalance problem. Figure 2 (d) Non-Linear-Y means that the model learns the features of the changed images in the first epoch, and trains by adding unchanged images through a nonlinear function within Y+1 epochs. After Y+1 epochs, the entire training dataset is used for training, taking into account the data imbalance problem. Figure 2(e) Fixed-X Linear-Y means that the model learns the features of the changed images in the first X epochs, and trains by adding unchanged images through a linear function within Y+X epochs. After Y+X epochs, the entire training dataset is used for training, taking into account the data imbalance problem. Figure 2 (f) Fixed-X Non-Linear-Y means that the model learns the features of the changed images in the first X epochs, and trains by adding unchanged images through nonlinear functions within Y+X epochs. After Y+X epochs, the entire training dataset is used for training, taking into account the data imbalance problem.

[0048] To make the model more lightweight, the DFFM module and encoder in Lite-CDNet use the improved ShuffleNetv2 as the backbone network. The specific improvements to ShuffleNetv2 are as follows: Figure 3 shown.

[0049] like Figure 3 As shown in (b), the improved ShuffleNetv2 with a stride of 1 allows the split ratio to be less than 0.5. This improvement improves the flexibility and computational efficiency of the model. By adjusting the split ratio, the model can allocate computing resources more effectively, optimize the data processing flow, and adapt to the needs under different computing resource constraints. The change in split ratio affects the channel reorganization method and data flow of the model, and then adjusts the parameter size and computational overhead of the model, providing a more sophisticated optimization mechanism for ShuffleNetv2 and enhancing its performance and adaptability. The channel crossover operation is used to optimize the data flow and reduce the amount of data processed by subsequent convolutional layers. At the same time, removing the ReLU activation function of the first layer of convolution further reduces the computational and storage costs. These improvements jointly improve the performance and efficiency of ShuffleNetv2, enabling it to maintain high performance under low computing resource conditions, which has obvious advantages over the original ShuffleNetv2.

[0050] For the deep feature fusion module DFFM, in the lightweight design of the change detection model, the early fusion strategy can significantly reduce the size of the model. Its operating principle is to ensure that there is no information loss in the input dual-phase image before the image is input into the model. The two images are first connected in series in the channel dimension, merged into a multispectral image, and then a single-stream network is used for subsequent change detection tasks. However, early fusion mixes the features of the two images too early and lacks reasonable guidance of the change features, which ultimately leads to low detection accuracy. In order to improve the accuracy of the lightweight change detection model while maintaining information integrity, it is necessary to enhance the feature guidance capability of early fusion. Based on this, the present application proposes a deep feature fusion module DFFM, and its structure diagram is shown in the figure. Figure 4 shown.

[0051] like Figure 4 As shown, the DFFM uses a weight-sharing convolutional block to extract shallow features for the input initial image. The convolutional block includes a conventional 3×3 convolution, a normalization layer, and an activation function. The extracted shallow feature map is input into the improved ShuffleNetv2 block for further feature extraction. First, the shuffle unit with a step size of 2 downsamples it to reduce the size of the feature map to half of the original size. Then, the shuffle unit with a step size of 1 is used to divide the feature map into two groups along the channel dimension. One group performs identity mapping, and the other group performs feature extraction through three convolutional layers with the same input and output channels, including two 1×1 convolutional layers and a 3×3 depth-separable convolutional layer. The two groups of feature maps are spliced ​​and channel shuffled to ensure information interaction and effectively reduce the number of model parameters. In order to improve the generalization ability of the model, shortcut connections are used to fuse the shallow features with the features extracted by the ShuffleNetv2 block. After the temporal attention mechanism, the two sets of feature maps are concatenated and shuffled, and the deep fusion features are obtained after adjusting the number of channels through a 1×1 convolution block.

[0052] Existing studies have shown that due to the temporal correlation between dual-phase images, maintaining information in the temporal dimension can improve the accuracy of the change detection model. The traditional temporal attention mechanism relies on fixed parameter settings and cannot adapt to different scenarios and feature distributions. In order to enhance the temporal correlation of the dual-phase feature map in Lite-CDNet, this application uses a simple adaptive temporal attention mechanism, which can dynamically adjust the attention weight according to the input features. The core of the adaptive temporal attention mechanism is an adaptive weight learning module, which can automatically calculate the weight of each feature channel based on the input dual-phase feature map. Its expression is shown in Equation 1 below:

[0053] W adaptive =MLP(GlobalAvgPool([Xt1 , X t2 ])) (1)

[0054] Among them, X t1 , X t2 represents a dual-phase image, W adaptive It is an adaptive weight vector learned from the global average pooling features of two phases through a multi-layer perceptron (MLP). This weight vector will be used to adjust the importance of each channel. This weight is applied to the original attention mechanism to obtain adaptive feature attention, as shown in Formula 2:

[0055]

[0056] Among them, Conv 1×1 represents a 1×1 convolutional layer, AvgPool represents average pooling, and [] represents the connection of two feature maps. Represents element-wise product. X t1 , X t2 After a 1×1 convolution layer, a batch normalization layer, a nonlinear activation layer, and an average pooling layer, the adaptive weight W learned by MLP is adaptive Multiply them together to get adaptive feature attention. In this way, the attention weight of each channel can be dynamically adjusted according to the input features, thereby achieving dynamic feature attention.

[0057] For the multi-scale axial attention module MAA, in order to obtain a more refined feature map, MAA includes a multi-channel convolution structure (MCS) and an axial focus mechanism (AFM). MCS uses expansion coefficients of different sizes to expand the receptive field of the convolution kernel and uses channel superposition operations to promote the exchange of information between different receptive fields. AFM can improve the effectiveness of feature extraction and reduce the computational cost of the model by assigning different weights to feature maps.

[0058] As the network layers continue to deepen, the semantic information of the feature map becomes richer and richer, but each layer of the network will cause a certain degree of feature loss and reduction of detail information. MCS solves this problem by integrating features of different scales. By extracting features at multiple scales and effectively fusing them, MCS can retain more detail information while maintaining sensitivity to semantic information. Its schematic diagram is shown below: Figure 5 shown.

[0059] like Figure 5As shown, MCS has 4 group convolutions, the size of the convolution kernel is 3×3, there are 4 expansion coefficients, and there is space between the elements of the convolution kernel matrix. They can cover a larger input data area without increasing the number of parameters or computational cost, effectively expanding the receptive field, that is, the input area "seen" by each convolution operation, and can integrate multi-scale features. C is the number of group members for each convolution operation, which is 0.5 times the size of the input data. For example, if the input data has 64 channels, then each group in the group convolution will process 32 channels. Such a division enables the network to learn more diverse features from different subsets of the input data, which contributes to a more efficient and powerful feature extraction process. The output of MCS can be expressed as the following formula:

[0060]

[0061]

[0062] Out MCS =conv 1×1 (cat(conv 1×1 (S1)+conv 1×1 (S2))) (5)

[0063] Where I represents the input feature, cat(·) represents the channel dimension concatenation operation, is a convolutional layer with a kernel size of 3 and a dilation factor of m. 1×1 The convolution kernel size is 1 for the normal convolution layer.

[0064] Compared with channel attention CAM, AFM's dimension-by-dimension processing method significantly reduces the amount of calculation, which can greatly improve the calculation speed and efficiency. AFM focuses on one dimension at a time, first one dimension of all columns, and then one dimension of all rows, so that the amount of data processed at each step is reduced, thereby reducing the computational requirements of each step. This method is particularly suitable for high-resolution images or large-scale data sets, and can effectively reduce the pressure on hardware resources.

[0065] like Figure 6 As shown, the receptive field of AFM is W (or H) pixels in the same row (or column) of the target pixel, which means that its receptive field is much smaller than that of CAM. The row attention output of AFM can be expressed as:

[0066]

[0067]

[0068] Among them, Out MCS Represents the output result of multi-way convolution MCS, represents the reshape operation on row attention.

[0069] The column attention output of AFM can be expressed as:

[0070]

[0071]

[0072] Among them, Out MID represents the output result of row attention, represents the reshaping operation on the column attention. Finally, the output of the MAA module can be expressed as the sum of the output of MCS and the output of AFM:

[0073] Out MAA =Out MCS +Out AFM (10)

[0074] The feature map input to the MAA module is first integrated with multi-scale features through MCS, and then refined simultaneously through AFM to obtain the output of the MAA module. This method improves the extraction of context, internal and edge features of the input image. In addition, the MAA module not only attaches importance to the extraction of multi-scale information, but also attaches importance to the extraction of context information, and takes into account the number of module parameters.

[0075] The following describes the technical effects of this application through different data sets and implementation processes.

[0076] The LEVIR-CD dataset contains a total of 637 pairs of extremely high-resolution images with a resolution of 0.5 meters per pixel and a standard size of 1024×1024. The number of pixels that have changed in the dataset is approximately 31,066,643, accounting for 4.65% of the total pixels, while the number of pixels that have not changed is 636,876,269, accounting for 95.35% of the total pixels. It is an extremely unbalanced change detection dataset. In order to make a fair comparison with other research methods, the original image pairs are cropped into small blocks of 256×256, resulting in an image dataset containing 7,120 pairs of training data, 1,024 pairs of verification data, and 2,048 pairs of test data.

[0077] WHU-CD is a public dataset designed for remote sensing building change detection. It contains a pair of high-resolution (0.2 m / pixel) image pairs with an original size of 32507×15345. The total number of changed pixels in this dataset is 21442501, accounting for 4.26% of the total pixels, and the total number of unchanged pixels is 481873979, accounting for 95.74% of the total pixels. It is also an extremely unbalanced change detection dataset. To ensure fair comparison with other methods, the image pairs are cropped to a size of 256×256 pixels, divided according to the default segmentation scheme, and 10% of the images are randomly selected from the training set to form a validation set. The dataset contains 4536 pairs of training data, 504 pairs of validation data, and 2760 pairs of test data.

[0078] The experiment implements the algorithm model based on the open source deep learning library Pytorch and uses a single NVIDIA RTX3090 GPU for training. The Adam optimizer is used, the initial learning rate is 0.0005, and the StepLR with γ of 0.5 is used to update the learning rate, as shown in Formula 11. The batch size is set to 8 and the number of training rounds is set to 100. During the model training process, the most effective training strategy in PCBS is used to promote model convergence, and the best model is saved in the validation set as the final training result.

[0079] 1 new =lr initial ×γ epoch / / stepsize (11)

[0080] In order to verify the lightweight degree and model performance of Lite-CDNet, Lite-CDNet is compared with mainstream change detection methods. The model's parameter quantity Param and MACs value are key indicators for evaluating the lightweight performance of the model. The reduction in the number of parameters is conducive to the simplicity of the model, low storage requirements, and strong generalization ability. The lower MACs value indicates that the computational complexity of the model inference phase is low, which helps to run efficiently on devices with limited resources. By comprehensively analyzing these two indicators, the complexity and computational efficiency of the model can be balanced, providing an important basis for model selection in specific scenarios. In terms of model performance, Precision is used to measure the accuracy of the model's positive example prediction, Recall evaluates the model's ability to correctly predict positive examples, F1Score combines Precision and Recall to provide a comprehensive evaluation, and the IoU indicator is used to evaluate the accuracy of the model in semantic segmentation tasks. Generally speaking, a high Precision indicates that the model is more accurate in positive example prediction, a high Recall indicates that the model captures positive examples better, a higher F1 Score indicates that the model has achieved a balance between Precision and Recall, and a high IoU indicates that the model has a stronger ability to identify and locate targets, thereby making the model perform better.

[0081] Lite-CDNet uses the improved Shufflenetv2 as the backbone network. The comparison results of the number of parameters Param and MACs of Lite-CDNet with those of mainstream models are shown in Table 2.

[0082] Table 2 Comparison of the number of parameters and multiplication-addition-accumulation operations of each model

[0083]

[0084] As shown in Table 2, Lite-CDNet has 0.94M parameters and 1.85G MACs, ranking first. Lite-CDNet has fewer parameters and MACs than other mainstream models, maintaining a relatively small model size. Subsequent experiments further prove that Lite-CDNet can effectively improve the detection accuracy of the model while reducing the model size, and shows excellent performance in resource-constrained environments.

[0085] Table 3 Comparative experimental results of Lite-CDNet on LEVIR-CD dataset and mainstream models

[0086]

[0087] Table 3 shows the comparison results of these models in performing change detection on the LEVIR-CD dataset. Lite-CDNet shows excellent results in both F1 and IoU, two key performance indicators, reaching 91.83% and 84.89% respectively, which are 0.78% and 1.33% higher than TinyCD, and 1.43% and 2.42% higher than the Transformer-based ChangeFormer model. This result shows that Lite-CDNet achieves higher change detection accuracy, and also shows that Lite-CDNet can achieve accurate change detection even compared to the Transformer model with more parameters. This further confirms Lite-CDNet's excellent ability to balance model size and performance.

[0088] In order to more intuitively present the change detection results of various mainstream models on the LEVIR-CD dataset, this application visualizes them as follows Figure 7As shown. In the figure, column (a) represents the input image at time T1, column (b) represents the input image at time T2, column (c) represents the real label image, which provides the ground truth value of change detection and provides a benchmark for subsequent comparative evaluation. Columns (d)-(l) are FC-EF model, FC-Siam-diff model, FC-Siam-conc model, STANet model, BIT model, ChangeFormer model, TinyCD model, LightCDNet model and Lite-CDNet model proposed in this application. Through these visualization results, the performance differences between different models can be intuitively compared.

[0089] from Figure 7 From the content in the dotted box, we can see that Lite-CDNet can detect the edge details of buildings very well. In particular, in the first and third rows of the figure, we can see that the shapes of buildings detected by other methods are slightly rough, and there are even adhesions between buildings. In contrast, the method of this application can obtain accurate images of building changes, which shows that the deep feature fusion module of this application better retains the shallow features of the dual-phase input image, and the residual connection retains more original image information, which can help the model focus on the edge details and feature information of the building.

[0090] Table 4 gives the quantitative comparison results of the model Lite-CDNet proposed in this application and other change detection models on the WHU-CD dataset.

[0091] Table 4 Comparative experimental results of Lite-CDNet on WHU-CD dataset and mainstream models

[0092]

[0093] As can be seen from Table 4, Lite-CDNet also shows excellent performance on the WHU-CD dataset, with its precision Pre, F1 score and IoU score increasing by 1.14%, 0.30% and 0.71% respectively compared with the TinyCD model. This result once again verifies the effectiveness and efficiency of the model. Although Lite-CDNet has an increased number of parameters compared to TinyCD, the overall performance of Lite-CDNet is still better than most other mainstream change detection methods, proving that it has achieved a good balance between lightweight and performance.

[0094] The qualitative analysis on the WHU-CD dataset is as follows Figure 8As shown in the figure, column (a) represents the input image at time T1, column (b) represents the input image at time T2, column (c) represents the real label image, and columns (d) to (k) are FC-EF model, FC-Siam-diff model, FC-Siam-conc model, STANet model, BIT model, TinyCD model, LightCDNet model and Lite-CDNet model proposed in this application.

[0095] from Figure 8 It can be seen that the Lite-CDNet proposed in this application can locate the changing target more accurately and has a strong ability to suppress the interference of irrelevant changes. The models in columns (d)-(j) cannot completely locate the changing buildings, resulting in prediction errors. Lite-CDNet can obtain relatively complete detection results. The main reasons are: First, the proposed deep feature fusion module DFFM better retains and extracts the original image information. Secondly, different levels of temporal difference feature representations are generated to capture rich contextual information at different levels. Finally, under the guidance of multi-scale axial attention MAA, multi-level difference features are gradually aggregated, and different weights are assigned to feature maps in a row-by-row and column-by-column manner to provide a more refined change map for the change detection task.

[0096] In general, the Lite-CDNet model proposed in this application maintains a lightweight structure while ensuring high efficiency and accuracy in change detection tasks.

[0097] From the perspective of model ablation experiments, Lite-CDNet is an improvement on the lightweight change detection network model LightCDNet. It mainly optimizes the deep feature fusion module (DFFM) and introduces multi-scale axial attention (MAA) to refine the change features. During the training process, the progressive change balance sampling strategy (PCBS) is used to promote model convergence. To prove the effectiveness of each module and PCBS, four representative ablation experiments were designed, using the WHU-CD dataset for experiments, and selecting F1 and IoU as evaluation indicators.

[0098] (1) Ablation experiment of each module of Lite-CDNet: LightCDNet with only feature fusion module is used as the baseline model. The experiment includes the impact of adding and removing DFFM module and MAA module on Lite-CDNet. Table 5 shows the experimental results.

[0099] Table 5 Ablation experiment results of each module of Lite-CDNet

[0100]

[0101] As can be seen from Table 5, when any module in the network is removed, the performance of the model decreases to varying degrees, proving that the MAA module and DFFM proposed in this application are effective in improving the performance of model change detection. Removing DFFM alone has a greater impact on model performance than removing the MAA module, which shows that the improved ShuffeNetv2 in DFFM can significantly enhance the network's extraction of specific area features without significantly increasing the computational burden of the model.

[0102] (2) Ablation experiment of PCBS: To verify the effectiveness of the PCBS proposed in this application, this application used different PCBS to guide model training in the Lite-CDNet training phase and tested the results. For the hyperparameters X and Y in PCBS, the experiment was set to X=Y=10. The results are shown in Table 6. Note that due to the limitation of the table header, the abbreviations of the methods in the proposed PCBS are used, where: FX represents Fixed-X, LY represents Linear-Y, NL-Y represents Non-Linear-Y, FX LY represents Fixed-X Linear-Y, and FX NL-Y represents Fixed-X Non-Linear-Y. Table 7 also uses the same abbreviations.

[0103] Table 6 Ablation experiment results of different PCBS strategies

[0104]

[0105] As can be seen from Table 6, the effects of using different PCBS on the experimental results vary, but all PCBS results are higher than the baseline model, among which the Fixed-10 Non-Linear-10 method has the best result. PCBS promotes the convergence of the model by strengthening the model's attention to the changing features, further improving the detection accuracy. Therefore, the PCBS strategy not only promotes the convergence of the model, but also provides an effective strategy for the change detection of the model under the condition of unbalanced data samples.

[0106] In addition, this application also visualizes the changes in the loss function Loss and the precise Press during the training process of Lite-CDNet with and without PCBS, such as Fig. 9 shown.

[0107] like Fig. 9As shown in the figure, Lite-CDNet without PCBS is unstable during the convergence process, especially in the first 20,000 iterations, where large performance fluctuations can be observed. This instability affects the final performance of the model and makes the training process more complex and unpredictable. After the introduction of PCBS, it can be clearly seen that the convergence process of Lite-CDNet becomes smoother and more coherent. This improvement shows that PCBS effectively stabilizes the training process and improves the learning efficiency of the model. PCBS helps Lite-CDNet adapt and identify key change information more quickly by strengthening the model's learning of changed image features, thereby achieving faster and more stable convergence.

[0108] (3) Ablation experiment of hyperparameters in PCBS: In the ablation experiment results of different PCBS strategies in Table 6, the F-10-NL-10 method achieves the best change detection results. Based on this, this experiment further discusses the setting of hyperparameters in the Fixed-X Non-Linear-Y method to find the optimal method, where X and Y represent the number of epochs. As shown in Table 7, the experiment is conducted by increasing ten epochs from 0 to 100 epochs, and the experimental results at 10, 20, 40 and 60 epochs are shown.

[0109] Table 7 Ablation experiment results of hyperparameters in PCBS

[0110]

[0111]

[0112] As can be seen from Table 7, when the Fixed-20 Non-Linear-20 method in PCBS is used, the F1 score of the model reaches 92.04%, which is the best parameter configuration. As X and Y continue to increase, the performance of the model decreases. This shows that PCBS has a positive impact on the improvement of model performance within a certain range, but this impact cannot continue to increase with the increase of X and Y.

[0113] Comparative experiments and ablation experiments were conducted on extremely unbalanced datasets WHU-CD and LEVIR-CD to verify that the proposed PCBS strategy can alleviate the problem of data imbalance by adjusting the training epoch without introducing new synthetic change samples. The proposed lightweight change detection model Lite-CDNet can fully learn the characteristics of the data and achieve a good balance between lightweight and performance.

[0114] It should be noted that, in the absence of conflict, the features in the embodiments of this application may be combined with each other.

[0115] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A remote sensing data change detection system, characterized in that: include: A single-stream lightweight change detection network model Lite-CDNet, which includes a deep feature fusion module DFFM, a multi-scale feature extractor, and a multi-scale axial attention module MAA; After the deep feature fusion module DFFM performs feature fusion on the input dual-phase image, the lightweight network model ShuffleNetv2 is used to extract multi-scale features, and the multi-scale axial attention module MAA helps the model identify the changing regional features and improve the feature details; A progressively changing balanced sampling strategy (PCBS) is used during training to promote model convergence.

2. The remote sensing data change detection system according to claim 1, characterized in that: After the deep feature fusion module DFFM performs feature fusion on the dual-phase images, it merges them into a multispectral feature map, which is then sent to the encoder to generate feature maps of different scales through three stages. The first layer of each stage is the Shuffle convolution block with downsampling function in ShuffleNetv2, which is used to achieve downsampling of feature mapping.

3. The remote sensing data change detection system according to claim 2, characterized in that: Each scale feature can be expressed as {f m |m=1,2,3},where f m Represents the image features at stage m, where m represents three different stages, and each scale f is obtained m It will also serve as the input of the MAA module.

4. The remote sensing data change detection system according to claim 2, characterized in that: The Lite-CDNet model uses a lightweight FPN as a decoder, which resizes the underlying features and adds them to the top-level features.

5. The remote sensing data change detection system according to claim 2, characterized in that: The deep feature fusion module DFFM and the encoder both use the lightweight network model ShuffleNetv2 as the backbone network. The Shuffle block with a step size of 1 in ShuffleNetv2 allows the split ratio to be less than 0.5, uses channel cross operation to optimize the data flow, and removes the ReLU activation function of the first layer of convolution.

6. The remote sensing data change detection system according to claim 2, characterized in that: The deep feature fusion module DFFM uses a weight-sharing convolution block to extract shallow features from the input initial image. The convolution block includes a conventional 3×3 convolution, a normalization layer and an activation function. The extracted shallow feature map is input into the ShuffleNetv2 model for further feature extraction. First, the shuffle unit with a step size of 2 downsamples it to reduce the feature map size to half of the original size. Then, the shuffle unit with a step size of 1 is used to divide the feature map into two groups along the channel dimension. One group performs identity mapping, and the other group performs feature extraction through three convolution layers with the same input and output channels, including two 1×1 convolution layers and one 3×3 depth-separable convolution layer. The two groups of feature maps are concatenated and shuffled to ensure information interaction. The shallow features are fused with the features extracted by the ShuffleNetv2 model using shortcut connections. After the temporal attention mechanism, the two groups of feature maps are concatenated and shuffled. After adjusting the number of channels through a 1×1 convolution block, the deep fusion features are obtained.

7. The remote sensing data change detection system according to claim 6, characterized in that: The temporal attention mechanism includes an adaptive temporal attention mechanism, which dynamically adjusts the attention weight according to the input features. The adaptive temporal attention mechanism is an adaptive weight learning module, which can automatically calculate the weight of each feature channel based on the input dual-phase feature map.

8. The remote sensing data change detection system according to claim 6, characterized in that: The multi-scale axial attention module MAA includes a multi-way convolution structure MCS and an axial attention mechanism AFM. MCS uses expansion coefficients of different sizes to expand the receptive field of the convolution kernel and uses channel superposition operations to promote the exchange of information between different receptive fields. AFM can improve the effectiveness of feature extraction by assigning different weights to feature maps.

9. The remote sensing data change detection system according to claim 8, characterized in that: The feature map input by the multi-scale axial attention module MAA is first integrated through MCS for multi-scale features, and then simultaneously refined through AFM to obtain the output result of the multi-scale axial attention module MAA.

10. The remote sensing data change detection system according to claim 8, characterized in that: The progressive change balanced sampling strategy PCBS accelerates the convergence of the model by adjusting the sample distribution at different stages of training. It first trains images with changes and then gradually adds images without changes for training.

Images