Remote sensing image change detection method and device and electronic equipment

Abstract

The invention provides a remote sensing image change detection method and device and electronic equipment, and belongs to the field of image change detection. The method comprises the following steps: inputting a dual-time-phase remote sensing image into a multi-scale cross-attention network; extracting low-level features of the dual-temporal remote sensing image based on different feature scales, and fusing the low-level features to obtain low-level fused features; based on a cross attention mechanism, performing differential enhancement processing on the low-level fusion features to obtain a reinforced fusion feature map; based on a contextual information link algorithm, performing reconstruction operation on the enhanced fusion feature map to obtain a reconstructed feature map; and carrying out subtraction operation on the reconstructed feature maps in different periods to obtain a final recognition image. According to the method, the domain gap is eliminated by using the unchanged region while the characteristics of the tiny change region in the remote sensing image are emphasized, the final image detection result is jointly expressed, and the integrity and accuracy of identifying the change region of the remote sensing image in different periods are improved.

Description

Technical Field

[0001] The present application belongs to the field of image change detection, and specifically relates to a remote sensing image change detection method, device and electronic equipment. Background Art

[0002] Remote sensing image change detection is a key technology for quickly identifying and evaluating changes in surface features, and is widely used in resource exploration, environmental monitoring, urban planning and management, etc. By comparing remote sensing images at different time points, this technology can reveal the change pattern of the surface and provide an important basis for decision-making in various industries.

[0003] However, when processing high-resolution remote sensing images, it is still challenging to accurately detect all changed areas. On the one hand, due to differences in acquisition conditions (such as weather conditions, lighting angles) and sensor characteristics, even unchanged objects may appear significantly different in images taken at different time points. For example, the texture of the same building under different sensors may show obvious differences, which increases the difficulty of detecting changed areas. On the other hand, for areas with smaller areas and subtle changes, their features are often not obvious enough to be captured by existing change detection algorithms, resulting in the details of the changes being missed in the final detection results.

[0004] In response to the above problems, this application proposes a multi-scale cross-attention network (hereinafter referred to as MSCA-Net). MSCA-Net aims to improve the detection accuracy and completeness of the changed area by integrating multi-scale information and utilizing the cross-attention mechanism. The network can not only effectively reduce the interference caused by environmental and sensor differences, but also more keenly perceive the existence of small changes, thereby ensuring that the changes between images of different periods are fully and accurately identified. This improvement is expected to significantly improve the effect of change detection and provide users with more reliable data support. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a remote sensing image change detection method, device and electronic device, which can solve the problem that image difference detection is interfered with by environmental and sensor differences, and detail changes are difficult to detect and often ignored.

[0006] In order to solve the above technical problems, this application is implemented as follows: In a first aspect, an embodiment of the present application provides a remote sensing image change detection method, the method comprising: Input the dual-temporal remote sensing image into a multi-scale cross-attention network; wherein the multi-scale cross-attention network is based on multi-scale feature encoding, differential feature enhancement algorithm and context information linking; Based on different feature scales, low-level features of dual-phase remote sensing images are extracted, and the low-level features are fused to obtain low-level fused features; wherein the low-level fused features correspond to different phases respectively; Based on the cross-attention mechanism, the low-level fusion features are differentially enhanced to obtain an enhanced fusion feature map; wherein the differential enhancement process is used to reduce the domain gap of the low-level fusion features and enhance the key change information between the low-level fusion features; Based on the context information linking algorithm, a reconstruction operation is performed on the enhanced fusion feature map to obtain a reconstructed feature map; wherein the reconstruction operation is used to enhance key change information in the enhanced fusion feature map based on spatial weights; The final recognition image is obtained by performing subtraction operations on the reconstructed feature maps at different times.

[0007] Compared with the prior art, the above technical solution provided by the present application includes at least the following beneficial effects: The present application first inputs the dual-phase remote sensing image into the multi-scale cross-attention network; secondly, extracts the low-level features of the dual-phase remote sensing image based on different feature scales, fuses the low-level features, and obtains the low-level fusion features to enhance the multi-scale cross-attention network's ability to characterize the features of the slightly changed area; then, based on the cross-attention mechanism, the low-level fusion features are differentially enhanced to obtain an enhanced fusion feature map, so as to guide the network to pay attention to the changed area through the differential features, enhance the network's ability to recognize the changed area, and solve the domain gap problem by sharing the information of the unchanged area; then, based on the context information linking algorithm, the enhanced fusion feature map is reconstructed to obtain a reconstructed feature map, which can integrate the change information of the tiny area into the final feature expression and enhance the network's ability to recognize the tiny change area; finally, the final recognition image is obtained by subtracting the reconstructed feature maps at different times. The present application emphasizes the characteristics of the slightly changed area in the remote sensing image while eliminating the domain gap by using the unchanged area, and jointly expresses the final image detection results, thereby improving the completeness and accuracy of identifying the changed areas of remote sensing images in different periods.

[0008] Preferably, the low-level features of the dual-temporal remote sensing image are extracted based on different feature scales, and the low-level features are fused to obtain low-level fused features. The specific steps include: Up-sampling and down-sampling operations are performed on dual-temporal remote sensing images to obtain multi-scale images; Apply low-level feature extraction to multi-scale images to obtain low-level features; The low-level features are fused based on the concatenation operation to obtain the low-level fused features.

[0009] In this scheme, by learning image features across different scales and integrating multi-scale feature information through concatenation operations, the feature expression capability of the multi-scale cross-attention network can be enhanced. In addition, by adding feature information of larger scales, it is easier for the network to focus on small change areas, thereby improving the accuracy and completeness of the change areas.

[0010] Preferably, the low-level features are fused based on the concatenation operation to obtain the low-level fused features, and the calculation formula is: in, Represents low-level fusion features; represents a dual-temporal remote sensing image, and its subscript represents the scaling factor of upsampling and downsampling operations on the dual-temporal remote sensing image; i Classification labels representing images or features at different times; and Represent upsampling and downsampling operations respectively; Indicates a concatenation operation; It means that the VGG network adopts the structure of the first 7 layers.

[0011] Preferably, based on the cross attention mechanism, the low-level fusion features are differentially enhanced to obtain an enhanced fusion feature map, and the specific steps include: The convolution feature information is obtained by adaptively fusing low-level fusion features through convolution; Based on the cross attention mechanism, the cross attention output is calculated according to the convolution feature information; Add the cross attention output and the convolution feature information according to the time phase to obtain a fused feature combination; wherein the fused feature combination corresponds to different time phases respectively; Combine the tandem fusion features to obtain the tandem fusion features; Multi-layer perceptron is used to process the serial fusion features to obtain deep fusion features; The deep fusion features are deformed to obtain an enhanced fusion feature map.

[0012] In this scheme, the feature differences of different phases are interacted with the initial features through the cross-attention mechanism, which enhances the sensitivity of the multi-scale cross-attention network to change information and improves the ability to identify change areas; Preferably, based on the cross attention mechanism, the cross attention output is calculated according to the convolution feature information, and the specific steps include: The difference features are obtained based on the convolution feature information operation; The calculation formula for the difference feature is: in,D For the difference feature, and Respectively represent the convolution feature information at different stages; The query feature is constructed based on the difference feature; the formula for constructing the query feature is: in, Q represents the query features, W q Represents the query transformation matrix automatically learned by the multi-scale cross-attention network; The key features and value features are constructed based on the convolution feature information. The formula for constructing the key features and value features is: in, K represents the key characteristics, V Represents value characteristics; W k and W v They represent the key transformation matrix and value transformation matrix automatically learned by the multi-scale cross-attention network respectively; i Classification labels representing images or features at different times; Adopting the cross attention mechanism, the cross attention output is obtained according to the query features, key features and value features. The calculation formula for the cross attention output is: in, d is the channel dimension.

[0013] In this scheme, the difference features are integrated into the feature maps of each phase through the cross-attention operation, achieving full interaction and fusion of features from different sources. This enables the multi-scale cross-attention network to accurately identify the changed areas while ignoring the unchanged areas. At the same time, for the unchanged areas in the heterogeneous images, the difference operation and supervision information are forced to unify, thereby solving the domain gap problem in the feature space.

[0014] Preferably, based on the context information linking algorithm, the enhanced fusion feature map is reconstructed to obtain a reconstructed feature map, and the specific steps include: Assign corresponding weights to low-level features; Based on the weights, low-level features are concatenated to obtain a concatenated feature map; Apply convolution operation to the concatenated feature map to obtain a spatial weight feature map; Multiply the spatial weight feature map with the enhanced fusion feature map to obtain the reconstructed feature map.

[0015] In this scheme, the network's ability to identify changed areas and distinguish unchanged areas is enhanced. In addition, the spatial weight feature map fully retains the change information and provides rich semantic information for obtaining a more comprehensive feature expression of the changed area. At the same time, combined with the enhanced fusion feature map, spatial distribution feature information is provided for the unchanged area.

[0016] Preferably, based on the context information linking algorithm, the enhanced fusion feature map is reconstructed to obtain a reconstructed feature map, and the calculation formula is: in, Represents the reconstructed feature map; represents the enhanced fusion feature map; , and Represent low-level features of different scales respectively; i Classification labels representing images or features at different times; a , b and c They represent the weight values ​​corresponding to the low-level features respectively; RELU represents the activation function, BN represents batch normalization, and Conv represents the convolution operation.

[0017] Preferably, the specific steps of the remote sensing image change detection method also include: Constructing a loss function based on image parameters of the final recognized image; wherein the image parameters include the height, width, true pixel value and predicted pixel value of the final recognized image; A multi-scale cross-attention network is trained based on the loss function to obtain a target multi-scale cross-attention network.

[0018] In this scheme, constructing a loss function plays a core role in guiding the optimization direction of the model in the multi-scale cross-attention network. It explicitly defines the difference measure between the predicted detection image and the actual detection image, thereby minimizing such differences through the optimization algorithm to adjust the model parameters and improve the performance to obtain the target multi-scale cross-attention network.

[0019] In a second aspect, an embodiment of the present application provides a remote sensing image change detection device, comprising: A network building module for inputting dual-temporal remote sensing images into a multi-scale cross-attention network; wherein the multi-scale cross-attention network is based on multi-scale feature encoding, differential feature enhancement algorithm and context information linking; The multi-scale feature fusion module is used to extract low-level features of dual-phase remote sensing images based on different feature scales, fuse the low-level features, and obtain low-level fusion features; wherein the low-level fusion features correspond to different phases respectively; The differential feature encoding module is used to perform differential enhancement processing on the low-level fusion features based on the cross-attention mechanism to obtain an enhanced fusion feature map; wherein the differential enhancement processing is used to reduce the domain gap of the low-level fusion features and enhance the key change information between the low-level fusion features; A context information linking module is used to reconstruct the enhanced fusion feature map based on the context information linking algorithm to obtain a reconstructed feature map; wherein the reconstruction operation is used to enhance key change information in the enhanced fusion feature map based on spatial weights; The recognition image output module is used to perform subtraction operations according to the different relative reconstruction feature maps to obtain the final recognition image.

[0020] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0021] It can be understood that the beneficial effects of the technical solutions provided in the second and third aspects can be found in the relevant description of the first aspect, and will not be repeated here.

[0022] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which: Figure 1 is a schematic flow chart of a remote sensing image change detection method shown in some embodiments of the present application; Figure 2 is a block diagram of a remote sensing image change detection device shown in some embodiments of the present application; Figure 3 is a block diagram of an electronic device shown in some embodiments of the present application; The following specific implementation methods will further illustrate the present application in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0025] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here. In addition, the "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally represents that the objects associated with each other are in an "or" relationship.

[0026] In the following, in conjunction with the accompanying drawings, a remote sensing image change detection method provided by an embodiment of the present application is described in detail through specific embodiments and application scenarios.

[0027] Figure 1 This is a flow chart of a remote sensing image change detection method according to the first embodiment of the present application. Figure 1 , the method comprising: Step S101: inputting a dual-temporal remote sensing image into a multi-scale cross-attention network; wherein the multi-scale cross-attention network is based on multi-scale feature encoding, differential feature enhancement algorithm and context information linking; Specifically, the multi-scale cross-attention network consists of a multi-scale feature encoding module (Multi-Scale Feature Enhancement, MSFE), a differential feature enhancement module (Differential Feature Enhancement and Alignment module, DFEA) and a contextual information link module (Contextual Information Linkage, CIL); The MSFE module is used to construct images of different scales and extract features respectively. The multi-scale feature fusion enhances the network's ability to characterize features of areas with small changes. The DFEA module is used to guide the network to focus on the changed area through differential features, enhance the network's ability to identify the changed area, and solve the domain gap problem by sharing information of unchanged areas; The CIL module is used to weightedly integrate and reconstruct the original multi-scale information, integrate the change information of small areas into the final feature expression, and enhance the network's ability to recognize small change areas.

[0028] Step S102: extracting low-level features of the dual-temporal remote sensing image based on different feature scales, fusing the low-level features, and obtaining low-level fused features; wherein the low-level fused features correspond to different temporal phases respectively; Specifically, upsampling and downsampling operations are performed on the dual-temporal remote sensing images to obtain multi-scale images; low-level feature extraction is applied to the multi-scale images to obtain low-level features; and low-level features are fused based on a concatenation operation to obtain low-level fused features.

[0029] In a possible implementation, in order to ensure accurate identification of small change areas, this embodiment designs an MSFE module. First, given a dual-phase remote sensing image ,in, H , W and 3 represent the height, width and number of channels of the image respectively. Based on upsampling and downsampling operations, multi-scale images are constructed from the input image , , , , , Among them, the superscript i =1 or 2 indicates images at different periods, Indicates that the image is reduced by half. represents the original image size, Indicates that the image is doubled in size.

[0030] Furthermore, using VGG-7 The network extracts low-level features from multi-scale images to learn local features of different scales of images, obtains low-level features of multiple scales, and fuses low-level features of three different scales through concatenation operations to obtain low-level fusion features. The calculation formula is as follows: in, represents the low-level fusion features, and Represent upsampling and downsampling operations respectively, Indicates a concatenation operation. express VGG The network adopts the structure of the first 7 layers; represents a dual-temporal remote sensing image, and its subscript represents the scaling factor of upsampling and downsampling operations on the dual-temporal remote sensing image. i ∈{1,2}, represents the classification identification of images or features at different periods.

[0031] The MSFE module can enhance the feature expression ability of the multi-scale cross-attention network by learning image features across different scales and integrating multi-scale feature information through concatenation operations. In addition, by adding feature information of larger scales, the multi-scale cross-attention network can more easily focus on small change areas, thereby improving the accuracy and completeness of the change areas.

[0032] Step S103: Based on the cross attention mechanism, differential enhancement processing is performed on the low-level fusion features to obtain an enhanced fusion feature map; wherein the differential enhancement processing is used to reduce the domain gap of the low-level fusion features and enhance the key change information between the low-level fusion features; Specifically, low-level fusion features are adaptively fused through convolution to obtain convolution feature information; based on the cross-attention mechanism, the cross-attention output is calculated according to the convolution feature information; the cross-attention output is added to the convolution feature information according to the time phase to obtain a fusion feature combination; wherein the fusion feature combinations correspond to different time phases respectively; the fusion feature combinations are concatenated to obtain concatenated fusion features; the concatenated fusion features are processed by a multi-layer perceptron to obtain deep fusion features; the deep fusion features are deformed to obtain an enhanced fusion feature map.

[0033] Further, based on the cross attention mechanism, the cross attention output is calculated according to the convolution feature information, and the specific steps include: obtaining a difference feature based on the convolution feature information; constructing a query feature based on the difference feature; constructing a key feature and a value feature based on the convolution feature information; using the cross attention mechanism, obtaining a cross attention output according to the query feature, the key feature and the value feature; In one possible implementation, due to environmental factors and the influence of different sensors, images at different time periods have significant differences, i.e., the domain gap problem, which leads to large feature differences for the same target and causes recognition errors between changed and unchanged areas. To solve this problem, a DFEA module is proposed.

[0034] First, the concatenated low-level fusion features are and ∈ Apply 3×3 convolution operation to adaptively fuse feature information of different scales to obtain convolution feature information and , which enables the multi-scale cross-attention network to automatically learn features that are more beneficial to the change detection task.

[0035] First, the concatenated and ∈ Use 3×3 convolution operation to adaptively fuse feature information of different scales and , allowing the multi-scale cross-attention network to automatically learn features that are more beneficial to the change detection task.

[0036] Secondly, the convolution feature information of different phases is mapped to the feature space of the same dimension, and the difference features D of different phases are obtained by subtraction operation. The calculation formula is as follows: Among them, D is the difference feature, and Respectively represent the convolution feature information at different stages; Next, in order to make the multi-scale cross-attention network focus on the feature information of the changed area, the cross-attention mechanism is used for feature interaction and fusion, where the calculation formula of the query feature is: and in and Construct the key features of the corresponding phases respectively Sum value characteristics , and its calculation formula is: in, , and , respectively, the query transformation matrix, key transformation matrix, and value transformation matrix automatically learned by the multi-scale cross-attention network. C is the product of the height and width of the input feature image, d are the channel dimensions of query features, key features, and value features, i Indicates the classification of images or features at different stages. The cross-attention operation is used, and the difference features are used as the key guiding information, so that the multi-scale cross-attention network can automatically identify and focus on the feature information of the changed area. The detailed calculation formula is as follows: in, d is the channel dimension.

[0037] It should be noted that the beneficial effects of using the cross-attention mechanism for feature processing are reflected in the following aspects: First, through the cross-attention mechanism, the feature differences of different phases interact with the initial features, which enhances the sensitivity of the multi-scale cross-attention network to change information and improves the ability to identify change areas; Second, through the cross-attention operation, the difference features are integrated into the feature maps of each phase, achieving full interaction and fusion of features from different sources. This enables the multi-scale cross-attention network to accurately identify the changed areas while ignoring the unchanged areas. At the same time, for the unchanged areas in the heterogeneous images, the difference operation and supervision information are forced to unify, thereby solving the domain gap problem in the feature space; Third, the subtraction operation of feature maps of different time phases will be accompanied by the loss of effective information. In order to reduce the impact of information loss on change detection, the cross attention output is combined with the convolution feature information and At the same time, this method also retains the information of the unchanged area, so that the multi-scale cross-attention network can establish the correlation between the feature expressions of data in different time phases through the supervision signal of the unchanged area, thereby achieving feature alignment and further reducing the impact of the domain gap problem.

[0038] Finally, and The concatenation is performed to obtain the concatenated fusion features to integrate the information of the changed features and the unchanged features, and the concatenated fusion features are deeply fused using a multi-layer perceptron (MLP). Subsequently, the deformation operation is applied to obtain the output strong fusion feature maps and .

[0039] Step S104: Based on the context information linking algorithm, a reconstruction operation is performed on the enhanced fused feature map to obtain a reconstructed feature map; wherein the reconstruction operation is used to enhance key change information in the enhanced fused feature map based on the spatial weight; Specifically, low-level features are assigned corresponding weights; low-level features are concatenated based on the weights to obtain a concatenated feature map; a convolution operation is applied to the concatenated feature map to obtain a spatial weight feature map; the spatial weight feature map is multiplied with the enhanced fusion feature map to obtain a reconstructed feature map.

[0040] In a possible implementation, the embodiment of the present application uses a multi-layer DEFA module to guide the network to pay more attention to the feature representation of the changed area, and in order to improve the network's attention to the unchanged information and the details of the slightly changed area, thereby improving the integrity of the change detection results, the embodiment of the present application also proposes a CIL module. This module emphasizes the small area change information, while ensuring the accuracy of detection, it enhances the integrity of the detection results.

[0041] First, the low-level features are concatenated based on different weights to obtain a concatenated feature map, where the large-scale feature map is given a higher weight. Then, the concatenated feature map is concatenated through a 3×3 convolution operation. Reconstruct to obtain a spatial weight feature map that focuses more on small target areas . The spatial weight feature map Enhanced fusion feature map output by multi-level DFEA module Multiply to get the reconstructed feature map , thereby associating the reconstructed spatial weight features with the fused high-level feature information.

[0042] This process enhances the network's ability to identify changed areas and distinguish unchanged areas. In addition, the spatial weight feature map fully retains the change information and provides rich semantic information for obtaining a more comprehensive feature expression of the changed area. At the same time, combined with the enhanced fusion feature map output by the DFEA module, it provides spatial distribution feature information for the unchanged area, and its formula is as follows: in, represents the reconstructed feature map, represents the enhanced fusion feature map, a , b and c Represent the weight values ​​corresponding to the low-level features, , and They represent low-level features, RELU represents activation function, BN represents batch normalization, and Conv represents convolution operation.

[0043] In this scheme, the multi-scale cross-attention network is able to more accurately identify the changed regions while retaining important information of the unchanged regions, thus achieving more accurate and complete change detection.

[0044] Step S105: performing subtraction operations on the reconstructed feature maps according to different periods to obtain a final recognition image.

[0045] The specific steps of the remote sensing image change detection method also include: A loss function is constructed according to image parameters of the final recognized image; the image parameters include the height, width, true pixel value and pixel prediction value of the final recognized image; a multi-scale cross-attention network is trained based on the loss function to obtain a target multi-scale cross-attention network.

[0046] In one possible implementation, this embodiment uses a cross-entropy loss function to minimize the parameters of the multi-scale cross-attention network. Change detection can be regarded as a binary classification problem. Therefore, the cross-entropy function is used as the loss function to quantify the difference between the changes predicted by the model and the actual situation. The specific calculation formula of the loss function is as follows: in, Represents the height and width of the change detection result image, Indicates location The true value of the pixel at This means that the pixel at that position The predicted value of .

[0047] Next, the accuracy evaluation of the multi-scale cross-attention network according to the embodiment of the present application will be introduced.

[0048] The present application embodiment mainly uses three evaluation indicators: F1 score, intersection over union (IoU) and overall accuracy (OA) for evaluation. The definitions of each evaluation indicator are as follows: F1 score: The F1 score is the harmonic mean of precision and recall, and is used to measure the performance of a model in a classification task. It takes into account the accuracy and completeness of the model's predictions for the positive class. The higher the F1 score, the better the model's performance.

[0049] Among them, TP, TN, FP and FN represent the number of true positives (1 Positives), true negatives (1 Negatives), false positives (0 Positives) and false negatives (0 Negatives), respectively.

[0050] TP (1 Positives, True Positives): Indicates the number of samples correctly predicted by the model as positive.

[0051] TN (1 Negatives, true negative examples): indicates the number of negative samples correctly predicted by the model.

[0052] FP (0 ​​Positives, False Positives): Indicates the number of negative samples that the model incorrectly predicts as positive samples.

[0053] FN (0 Negatives, false negatives): indicates the number of positive samples that the model incorrectly predicts as negative samples.

[0054] Intersection over Union (IoU): The intersection over union (IoU) is the ratio of the intersection area to the union area between the model prediction area and the true area. It is an important indicator for measuring the accuracy of model prediction in change detection tasks. The higher the IoU value, the closer the model prediction area is to the true area.

[0055] Overall Accuracy (OA): Overall accuracy is the ratio of the number of pixels correctly predicted by the model to the total number of pixels. It measures the overall performance of the model on the entire dataset. The higher the value of OA, the higher the accuracy of the model.

[0056] These three evaluation indicators together constitute the main criteria for evaluating model performance in this study, which helps to comprehensively and objectively measure the performance of the model in the change detection task.

[0057] The main classification methods compared are: FC-EF (2018), FC-Siam-Di (2018), FC-Siam-Co (2018), STANet (2020), BIT (2022), SNUNet (2022), HANet (2023), VcT (2023) and C2FNet (2024). It should be noted that: The FC-EF, FC-Siam-Di and FC-Siam-Co methods are hybrid convolutional neural networks for hyperspectral image classification, integrating the advantages of 3D convolution and 2D convolution. The 3D convolution part is responsible for extracting joint spatial-spectral feature expressions from continuous spectral bands, while the 2D convolution part further learns more abstract high-level spatial expressions based on 3D convolution. The STANet method uses a deep residual 3D convolutional neural network to extract the spectral-spatial features of the image, and learns a metric space at the same time, so that samples of the same category are more similar in the metric space and the differences between samples of different categories are improved. Finally, the nearest neighbor classifier is used to classify the test samples in the learned metric space. The BIT method is a deep classification model based on relational networks and meta-learning ideas. The feature learning module of the network design extracts deep features from hyperspectral image samples, while the relation learning module performs relation learning by comparing the similarities between different samples, that is, the relationship scores between samples of the same category are high, and the relationship scores between samples of different categories are low. The SNUNet method is a deep cross-domain small-sample learning method that aims to solve the domain shift problem in hyperspectral image classification. It uses the adaptive strategy of conditional adversarial domain to overcome the domain shift problem and achieve domain distribution alignment. In addition, FSL is performed in both the source and target classes, which can not only discover transferable knowledge in the source class, but also learn the discriminative encoding model of the target class. The HANet method learns transferable knowledge from the mini-ImageNet dataset and then fine-tunes it on hyperspectral data to extract more discriminative spectral spatial features and domain knowledge, thereby improving the accuracy of HSI classification. The VcT method designs a self-supervised learning module with geometric transformation based on HFSL. By setting the rotation label as supervision, it extracts low-level features that can better represent different directions, and then performs self-supervised learning and small-sample learning on the base class to learn transferable spatial meta-knowledge. The C2FNe method is a hyperspectral image classification method based on feature decoupling. From the perspective of decoupled expression learning, it aims to solve the domain shift problem in hyperspectral image classification and reduce the bias of source data on expression, so that the model can implicitly focus on the inherent knowledge of the target domain.

[0058] Table 1 Comparison results on three standard datasets, where the bold value is the highest score The remote sensing image change detection method provided by the above embodiment, first, inputs the dual-phase remote sensing image into the multi-scale cross-attention network; secondly, extracts the low-level features of the dual-phase remote sensing image based on different feature scales, fuses the low-level features, and obtains the low-level fusion features to enhance the multi-scale cross-attention network's ability to characterize the features of the slightly changed area; then, based on the cross-attention mechanism, performs differential enhancement processing on the low-level fusion features to obtain an enhanced fusion feature map, so as to guide the network to pay attention to the changed area through differential features, enhance the network's ability to recognize the changed area, and solve the domain gap problem by sharing the information of the unchanged area; then, based on the context information linking algorithm, reconstructs the enhanced fusion feature map to obtain a reconstructed feature map, which can integrate the change information of the tiny area into the final feature expression and enhance the network's ability to recognize the tiny change area; finally, subtracts the reconstructed feature map according to different periods to obtain the final recognized image. This application improves the completeness and accuracy of identifying the changed areas of remote sensing images in different periods by emphasizing the features of the tiny change areas in the remote sensing image while using the unchanged area to jointly express the final image detection results.

[0059] See also Figure 2 , Figure 2 is a schematic diagram of a remote sensing image change detection device according to a second embodiment of the present application. The remote sensing image change detection device 200 includes: Network construction module 201: used to input the dual-temporal remote sensing image into a multi-scale cross-attention network; wherein the multi-scale cross-attention network is based on multi-scale feature encoding, differential feature enhancement algorithm and context information linking; Multi-scale feature fusion module 202: extract low-level features of dual-phase remote sensing images based on different feature scales, fuse the low-level features, and obtain low-level fusion features; wherein the low-level fusion features correspond to different phases respectively; specifically, it includes: Up-sampling and down-sampling operations are performed on the dual-temporal remote sensing images to obtain multi-scale images; low-level feature extraction is applied to the multi-scale images to obtain low-level features; low-level features are fused based on concatenation operations to obtain low-level fused features.

[0060] Differential feature encoding module 203: Based on the cross attention mechanism, differential enhancement processing is performed on the low-level fusion features to obtain an enhanced fusion feature map; wherein the differential enhancement processing is used to reduce the domain gap of the low-level fusion features and enhance the key change information between the low-level fusion features; specifically, it includes: The convolution feature information is obtained by adaptively fusing low-level fusion features through convolution; based on the cross-attention mechanism, the cross-attention output is calculated according to the convolution feature information; the cross-attention output is added to the convolution feature information according to the time phase to obtain a fusion feature combination; wherein the fusion feature combinations correspond to different time phases respectively; the fusion feature combinations are concatenated to obtain concatenated fusion features; the concatenated fusion features are processed by a multi-layer perceptron to obtain deep fusion features; the deep fusion features are deformed to obtain an enhanced fusion feature map.

[0061] Further, based on the cross attention mechanism, the cross attention output is calculated according to the convolution feature information, and the specific steps include: obtaining a difference feature based on the convolution feature information; constructing a query feature based on the difference feature; constructing a key feature and a value feature based on the convolution feature information; using the cross attention mechanism, obtaining a cross attention output according to the query feature, the key feature and the value feature; The context information linking module 204: based on the context information linking algorithm, reconstructs the enhanced fused feature map to obtain a reconstructed feature map; wherein the reconstruction operation is used to enhance the key change information in the enhanced fused feature map based on the spatial weight; specifically includes: Assign corresponding weights to low-level features; concatenate low-level features based on the weights to obtain a concatenated feature map; apply convolution operation to the concatenated feature map to obtain a spatial weighted feature map; multiply the spatial weighted feature map with the enhanced fusion feature map to obtain a reconstructed feature map.

[0062] Recognition image output module 205: used to perform subtraction operations on the reconstructed feature maps according to different periods to obtain the final recognition image.

[0063] A loss function is constructed according to image parameters of the final recognized image; the image parameters include the height, width, true pixel value and pixel prediction value of the final recognized image; a multi-scale cross-attention network is trained based on the loss function to obtain a target multi-scale cross-attention network.

[0064] It should be noted that the remote sensing image change detection method provided in the embodiment of the present application can be executed by a remote sensing image change detection device, or a control module in the remote sensing image change detection device for executing and loading a remote sensing image change detection method. In the embodiment of the present application, a remote sensing image change detection method provided in the embodiment of the present application is described by taking a remote sensing image change detection device executing and loading a remote sensing image change detection method as an example.

[0065] A remote sensing image change detection device in the embodiment of the present application may be a device, or a component, integrated circuit, or chip in a terminal. The device may be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device may be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc., and the non-mobile electronic device may be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0066] A remote sensing image change detection device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0067] The remote sensing image change detection device provided in the embodiment of the present application can realize Figures 1 to 3 In order to avoid repetition, each process implemented by a remote sensing image change detection device in the method embodiment will not be described here.

[0068] Optionally, see Figure 3 The embodiment of the present application also provides an electronic device 300, including a processor 301, a memory 302, and a computer program 303 stored in the memory 302 and executable on the processor 301. When the computer program 303 is executed by the processor 301, each process of the above-mentioned remote sensing image change detection method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0069] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned remote sensing image change detection method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0070] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0071] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned remote sensing image change detection method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0072] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0073] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0074] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0075] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A remote sensing image change detection method, characterized in that: include: Inputting the dual-temporal remote sensing image into a multi-scale cross-attention network; wherein the multi-scale cross-attention network is based on multi-scale feature encoding, differential feature enhancement algorithm and context information linking; Extracting low-level features of the dual-phase remote sensing image based on different feature scales, fusing the low-level features to obtain low-level fused features; wherein the low-level fused features correspond to different phases respectively; Based on the cross attention mechanism, the low-level fusion features are subjected to differential enhancement processing to obtain an enhanced fusion feature map; wherein the differential enhancement processing is used to reduce the domain gap of the low-level fusion features and enhance the key change information between the low-level fusion features; Based on the context information linking algorithm, a reconstruction operation is performed on the enhanced fused feature map to obtain a reconstructed feature map; wherein the reconstruction operation is used to enhance key change information in the enhanced fused feature map based on spatial weights; By performing subtraction operations on the reconstructed feature maps at different times, a final recognition image is obtained.

2. The remote sensing image change detection method according to claim 1, characterized in that: The specific steps of extracting low-level features of the dual-temporal remote sensing image based on different feature scales, fusing the low-level features, and obtaining low-level fused features include: Performing upsampling and downsampling operations on the dual-temporal remote sensing image to obtain a multi-scale image; Applying low-level feature extraction to the multi-scale image to obtain low-level features; The low-level features are fused based on a concatenation operation to obtain low-level fused features.

3. The remote sensing image change detection method according to claim 2, characterized in that: The calculation formula for fusing the low-level features based on the concatenation operation to obtain the low-level fused features is: in, represents the low-level fusion feature; represents the dual-temporal remote sensing image, and its subscript represents the scaling factor of upsampling and downsampling operations on the dual-temporal remote sensing image; i represents the classification identifier of images or features in different periods; and Represent upsampling and downsampling operations respectively; Indicates a concatenation operation; It means that the VGG network adopts the structure of the first 7 layers.

4. The remote sensing image change detection method according to claim 1, characterized in that: The specific steps of performing differential enhancement processing on the low-level fusion features based on the cross attention mechanism to obtain an enhanced fusion feature map include: Adaptively fusing the low-level fusion features through convolution to obtain convolution feature information; Based on the cross attention mechanism, the cross attention output is calculated according to the convolution feature information; Adding the cross attention output to the convolution feature information according to the time phase to obtain a fused feature combination; wherein the fused feature combination corresponds to different time phases respectively; Concatenating the fused feature combinations to obtain concatenated fused features; A multi-layer perceptron is used to process the serial fusion features to obtain deep fusion features; The deep fusion features are deformed to obtain an enhanced fusion feature map.

5. The remote sensing image change detection method according to claim 4, characterized in that: The specific steps of calculating the cross attention output based on the cross attention mechanism and the convolution feature information include: Obtaining difference features based on the convolution feature information calculation; The calculation formula for obtaining the difference feature is: in, D For the difference feature, and Respectively represent the convolution feature information at different periods; A query feature is constructed according to the difference feature; the formula for constructing the query feature is: in, Q represents the query feature, W q A query transformation matrix representing the automatic learning of the multi-scale cross-attention network; The key feature and the value feature are constructed based on the convolution feature information. The formula for constructing the key feature and the value feature is: in, K represents the key feature, V representing said value characteristic; W k and W v Respectively represent the key transformation matrix and the value transformation matrix automatically learned by the multi-scale cross-attention network; i Classification labels representing images or features at different times; Adopting a cross attention mechanism, obtaining the cross attention output according to the query feature, the key feature and the value feature calculation; The operation formula for obtaining the cross attention output is: in, d is the channel dimension.

6. The remote sensing image change detection method according to claim 1, characterized in that: The specific steps of reconstructing the enhanced fusion feature map based on the context information linking algorithm to obtain the reconstructed feature map include: Assigning corresponding weights to the low-level features; Based on the weights, the low-level features are concatenated to obtain a concatenated feature map; Applying a convolution operation to the concatenated feature map to obtain a spatial weight feature map; The spatial weight feature map is multiplied by the enhanced fusion feature map to obtain a reconstructed feature map.

7. The remote sensing image change detection method according to claim 1, characterized in that: The context information linking algorithm is based on which the enhanced fusion feature map is reconstructed to obtain a reconstructed feature map. The calculation formula is: in, represents the reconstructed feature map; represents the enhanced fusion feature map; , and Respectively represent the low-level features of different scales; i represents the classification identification of images or features at different periods; a , b and c They respectively represent the weight values ​​corresponding to the low-level features; RELU represents the activation function, BN represents batch normalization, and Conv represents the convolution operation.

8. The remote sensing image change detection method according to claim 1, characterized in that: The specific steps also include: Constructing a loss function according to image parameters of the final recognized image; wherein the image parameters include the height, width, true pixel value and predicted pixel value of the final recognized image; The multi-scale cross-attention network is trained based on the loss function to obtain a target multi-scale cross-attention network.

9. A remote sensing image change detection device, characterized in that: include: A network construction module, used for inputting a dual-temporal remote sensing image into a multi-scale cross-attention network; wherein the multi-scale cross-attention network is constructed based on multi-scale feature encoding, differential feature enhancement algorithm and context information linking; A multi-scale feature fusion module, used for extracting low-level features of the dual-phase remote sensing image based on different feature scales, fusing the low-level features, and obtaining low-level fused features; wherein the low-level fused features correspond to different phases respectively; A differential feature encoding module, used for performing differential enhancement processing on the low-level fusion features based on a cross-attention mechanism to obtain an enhanced fusion feature map; wherein the differential enhancement processing is used to reduce the domain gap of the low-level fusion features and enhance the key change information between the low-level fusion features; A context information linking module, configured to perform a reconstruction operation on the enhanced fused feature map based on a context information linking algorithm to obtain a reconstructed feature map; wherein the reconstruction operation is configured to enhance key change information in the enhanced fused feature map based on a spatial weight; The recognition image output module is used to obtain a final recognition image by performing a subtraction operation on the reconstructed feature map at different times.

10. An electronic device, characterized in that: include: A memory, a processor, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, is a step of the remote sensing image change detection method as described in any one of claims 1 to 8.

Images