Remote sensing abnormal region and target detection method based on change detection

Abstract

The application provides a remote sensing abnormal area and target detection method based on change detection, a twin attention change detection model is used to train the change detection model to detect the ability of the processing area, and the ability of the change detection model to detect hidden targets and detect change targets is trained. The application constructs a double-branch twin attention change detection network, compares the abnormal area appearing in the remote sensing image with the change in pixel distribution and color difference of the original remote sensing image, and realizes automatic detection of the abnormal area and the change area; the application uses the twin network idea at each level of the down-sampling module, cascades the two time phase features into the subsequent network, processes the two time phase technologies without distinction, and removes the restriction on the input order of the time phase; the application uses the attention mechanism to realize the fusion of the detection results at different levels, uses two different feature fusion methods, uses the attention mechanism to comprehensively compare the two fusion results, and improves the detection ability of the model.

Description

Technical Field

[0001] This invention relates to the field of physical technology, and specifically to a remote sensing anomaly region and target detection method based on change detection. Background Technology

[0002] Satellite-captured remote sensing images contain a wealth of geographic information about the captured area and can reflect the specific geographical locations of various features within the images. The rapid development of remote sensing technology and the increasing openness of global information have made the de-identification of certain information necessary. A growing number of publicly available remote sensing images show traces of various de-identification techniques, including common techniques like image matting (direct removal or whitening), mosaicking, and downsampling—all easily detectable by the human eye. Deep learning-based image inpainting techniques, however, make de-identified areas even more difficult to detect. Therefore, a technique is needed to quickly identify abnormal areas based on comparative images.

[0003] Deep learning-based change detection provides a solution for this technology. The goal of change detection is to learn and detect differences within a group of images, focusing particularly on identifying differences in a specific type of object within an image. Therefore, this approach can be used not only for detecting changed regions but also for identifying differences between objects in two images. Current change detection models are primarily used to detect changes in buildings within the same area, including both new construction and demolition (addition and reduction states).

[0004] When the original image is available for comparison, this technique is highly suitable for detecting anomalous regions and automatically detecting hidden targets in remote sensing imagery. Even though anomalous regions, after being hidden, possess the same semantic information as unprocessed regions and are difficult for the human eye to observe, they exhibit a certain regularity because they are derived through inference and computation by a deep learning network model. This difficult-to-discover regularity can be cracked by change detection networks through learning on large-scale datasets, thereby achieving the detection of anomalous regions and the discovery of hidden targets.

[0005] In their paper "Automatic Detection and Hiding Method for Sensitive Targets in Emergency Remote Sensing Mapping" (Journal of Wuhan University, Information Science Edition, 1671-8860(2020)08-1263-10), Lu Pengjie, Xu Dalu, Ren Fu, et al. proposed an automatic detection and hiding technique for sensitive targets in emergency remote sensing mapping. Taking an aircraft as an example, this technique utilizes target detection technology in deep learning to automatically detect the target, and then uses image inpainting technology in deep learning to hide the detected aircraft. This method achieved good results; however, without comparison, it is difficult to detect the disappearance of the aircraft in the processed image, and the specific processed area is also difficult to identify. Other existing technologies do not address solutions to this situation.

[0006] Furthermore, existing datasets are all for changes in buildings, and there are no datasets for hiding targets, which is also a challenge for this technology.

[0007] Therefore, a method is needed to detect changes in abnormal areas and hidden targets. Summary of the Invention

[0008] This invention addresses the problem of detecting abnormal and changed regions by providing a remote sensing abnormal region and target detection method based on change detection. Using a twin attention change detection model, a dataset designed for image inpainting-based desensitization techniques is created to train the change detection model's ability to detect and process regions, while simultaneously training its ability to detect hidden targets. This enables automatic detection of abnormal regions in remote sensing images. Furthermore, by changing the labels of abnormal regions in the dataset to those of hidden targets, automatic detection of hidden targets using change detection is achieved.

[0009] This invention provides a remote sensing anomaly region and target detection method based on change detection, comprising the following steps:

[0010] S1. Select remote sensing images and corresponding anomaly region label images and change region label images, crop, filter and reduce density to obtain anomaly region detection dataset and change target detection dataset for image restoration and density reduction methods.

[0011] S2. Construct a twin attention change detection network, including a twin network composed of identical downsampling modules, a transmission module, an upsampling network composed of identical upsampling modules, and an ECAM module. The twin network, transmission module, upsampling network, and ECAM module are connected in sequence.

[0012] S3. The abnormal region detection dataset and the changed target detection dataset are randomly sorted and then input into the Siamese network to obtain feature 1 and feature 2 respectively. These features are then sequentially fed into the transmission module, the upsampling network and the ECAM module. After iterative training, two model weight files for abnormal region detection and changed target detection are obtained, and the abnormal region detection model and the changed target detection model are obtained.

[0013] S4. The original image to be detected and the corresponding transformed image are grouped into an image group, cropped, and then detected in the input anomaly detection model and the changed target detection model respectively to obtain the local detection results of the anomaly area and the local detection results of the changed target. The local detection results of the anomaly area and the local detection results of the changed target are then stitched together according to their serial numbers to obtain the anomaly detection result and the changed target detection result. The remote sensing anomaly area and target detection method based on change detection is completed.

[0014] The remote sensing anomaly region and target detection method based on change detection described in this invention, in a preferred embodiment, includes step S1 as follows:

[0015] S11. Randomly select at least 30 remote sensing images from the semantic segmentation dataset that have a balanced ratio of foreground to background, and corresponding abnormal region label images and change region label images.

[0016] S12. Crops the remote sensing image, the anomaly label image, and the change area label image to the same size in sequence to obtain the cropped remote sensing image, the cropped anomaly label image, and the cropped change area label image.

[0017] S13. Perform filtering. The filtering method is to filter out the cropped abnormal region label images, cropped change region label images, and corresponding cropped remote sensing images where the proportion of foreground pixels after cropping is greater than the filtering threshold.

[0018]

[0019] Where threshold is the filtering threshold, N1 is the number of foreground pixels, and N is the total number of pixels in the image;

[0020] S14. The selected cropped remote sensing images, cropped abnormal region label images, and cropped changed region label images are processed using density reduction methods to obtain an abnormal region detection dataset and a changed target detection dataset.

[0021] In the remote sensing anomaly region and target detection method based on change detection described in this invention, as a preferred embodiment, in step S12, the size after cropping is 256×256; in step S12, the screening threshold is 10%.

[0022] The remote sensing anomaly region and target detection method based on change detection described in this invention, in a preferred embodiment, includes step S2 as follows:

[0023] S21. Construct a Siamese network. The Siamese network consists of four identical downsampling modules. Each downsampling module first adds the result of the first convolutional layer to the BRCB module in the form of element-wise addition, and then inputs it into the first ReLU activation layer and the max pooling layer.

[0024] S22. Construct an upsampling network consisting of four identical upsampling modules. The upsampling module replaces the last pooling layer of the downsampling module with a transposed convolutional layer.

[0025] S23. Construct a transmission module, which includes the splicing of downsampling modules at the same level in the channel dimension and the dense connection between upsampling modules with the same output size.

[0026] S24. Construct the ECAM module. First, perform element-wise addition and channel-dimension concatenation on the four different outputs obtained from the upsampling module. Then, input them into the channel attention network layer to fuse the two results using the attention mechanism.

[0027] S25. Connect the twin network, transmission module, upsampling network and ECAM module in sequence to obtain the twin attention change detection network.

[0028] The remote sensing anomaly region and target detection method based on change detection described in this invention, as a preferred embodiment, includes a BRCB module in step S21 comprising a first batch of normalized layers, a second ReLU activation layer, a second convolutional layer, and a second batch of normalized layers; the convolutional layer parameters of the first and second convolutional layers are: a 3×3 kernel with a stride of 1, and a 2×2 kernel with a stride of 2 for the max pooling layer; the first convolutional layer, the BRCB module, and the first ReLU activation layer extract features, and the max pooling layer performs downsampling;

[0029] In step S22, the convolution kernel is 2×2 and the stride is 2;

[0030] In step S24, the last layer of the ECAM module is a convolutional layer with a 1×1 kernel and a stride of 1.

[0031] In a preferred embodiment of the remote sensing anomaly region and target detection method based on change detection described in this invention, in step S24, the ECAM module receives four output results obtained from the upsampling network and processes them through two branches. Branch one concatenates the four outputs along the channel dimension and inputs them into the channel attention layer. Branch two adds the four outputs element-wise and inputs them into the channel attention layer to obtain the extraction result. The result is then copied four times and concatenated along the channel dimension to make the number of channels the same as the output obtained from branch one. The result of branch two is added element-wise to the result of the original four outputs concatenated along the channel dimension and then multiplied element-wise with the result of branch one. Finally, the network's final output is obtained after passing through a convolutional layer.

[0032] The method of the ECAM module is as follows:

[0033]

[0034] Where [x1,x2,x3,x4] are the outputs of the four levels of the upsampling module, CAM is the channel attention mechanism, and repeat... (4) It returns four times.

[0035] In the preferred embodiment of the remote sensing anomaly region and target detection method based on change detection described in this invention, in step S3, the anomaly region detection dataset and the change target detection dataset are trained simultaneously. The training process uses the Adam optimizer, the learning rate is set to 0.0003, and the gradient descent method is used for iterative training.

[0036] The remote sensing anomaly region and target detection method based on change detection described in this invention, in a preferred embodiment, includes step S3 as follows:

[0037] S31. The abnormal region detection dataset and the changed target detection dataset are randomly sorted to shuffle the temporal phases, resulting in new temporal phase one image and new temporal phase two image.

[0038] S32. Input the new temporal image into a twin branch of the twin network to obtain the feature extracted by the four-level twin branches;

[0039] S33. Input the new phase 2 image into another twin branch of the twin network to obtain the feature 2 extracted by the four-level twin branches;

[0040] S34. Concatenate feature one and feature two at different channel dimensions and input them into the dense connection layer of the corresponding transmission module.

[0041] S35. Each level of dense connection layer is concatenated with the corresponding upsampling layer in the channel dimension and input to the next level of upsampling layer;

[0042] S36. Input the results of each upsampling layer into the ECAM module to obtain the detection results of abnormal and changed areas;

[0043] S37. Calculate the sum L of the weighted cross-entropy function and the dice function based on the detection results of abnormal and changed regions, and perform gradient backpropagation.

[0044] S38. Repeat steps S31 to S37 until the training batch reaches the set value or the loss function no longer converges, to obtain two model weight files for anomaly detection and changing target detection, and to obtain the anomaly detection model and the changing target detection model.

[0045] In the preferred embodiment of the remote sensing anomaly region and target detection method based on change detection described in this invention, in step S35, the dense connection layer learns the relationship between features at different levels by merging the color and shape features of the shallow layer and the gradient features and upsampled pixel classification results of the deep layer.

[0046] In step S37, the loss function L is:

[0047]

[0048] in

[0049] N is the total number of pixels in the image. c For pixels of class c, w c For class c weights, y i For the pixels of the original image, y i Y' represents the predicted pixel, Y is the original image, Y' is the predicted image, and softmax is the activation function.

[0050] In the remote sensing anomaly region and target detection method based on change detection described in this invention, as a preferred embodiment, in step S31, both the anomaly region detection dataset and the change target detection dataset are divided into a validation set and a test set. In step S36, the model is saved once every 2 epochs of training, and the test set is tested in the validation set.

[0051] The idea behind this invention is to construct a two-branch twin attention change detection network to automatically detect abnormal regions in remote sensing images. By comparing and learning the changes in pixel distribution and color difference between abnormal regions appearing in remote sensing images and the original remote sensing images, the invention achieves automatic detection of abnormal regions.

[0052] To address the changes in remote sensing images before and after de-densification, this invention employs a Siamese network structure. It uses the same weight parameters to extract features from two temporal remote sensing images within the same input group and concatenates them along the channel dimension. This facilitates subsequent networks in quickly extracting the differences between the two images before and after de-densification, obtaining key feature information. Furthermore, to adapt to non-cooperative working modes, such as situations where image de-densification is not performed or the input order is uncertain, this invention uses the Siamese approach in each downsampling layer to obtain features from both temporal phases and input them into subsequent networks, thereby removing the network's temporal constraints on the input images.

[0053] This invention uses dense connections to enable the interaction of shallow and deep semantic information. By merging shallow color and shape features with deep gradient features and upsampled pixel classification results, it repeatedly compares and learns the connections between features at different levels, thereby improving detection capabilities.

[0054] This invention uses an attention mechanism to fuse detection results from different levels. By using two different feature fusion methods, the attention mechanism is used to comprehensively compare the two fusion results, thereby improving the model's detection capability.

[0055] This invention uses the same network to detect the original targets in abnormal areas of remote sensing images, and after changing the labels of abnormal areas in the dataset to the labels of hidden targets, it achieves automatic detection of changes on hidden targets.

[0056] The specific steps of this invention include the following:

[0057] Step 1: Generate datasets for anomaly region detection and changed target detection targeting image inpainting and de-densification techniques.

[0058] Step 1.1: Randomly select at least 30 high-resolution remote sensing images and their corresponding label images from the semantic segmentation dataset that have a relatively balanced ratio of foreground to background.

[0059] Step 1.2: Crop the selected high-resolution image and the corresponding label image to a size of 256*256.

[0060] Step 1.3: Filter labels and their corresponding remote sensing images where the foreground pixel ratio of the cropped label is greater than 10%;

[0061] Step 1.4: Process the image using a de-encryption method to obtain the de-encrypted image and label.

[0062] Step 2, construct a twin attention change detection network:

[0063] Step 2.1, construct a twin network;

[0064] The Siamese network consists of four identical downsampling modules. Each downsampling module first takes the result of the first convolutional layer and adds it element-wise to the result of the (BRCB, BatchNormal+ReLU+Conv+BatrchNormal) module before inputting it into the ReLU activation layer and the max pooling layer. The parameters of the convolutional layers in the first and BRCB modules are: 3*3 kernel with a stride of 1, and 2*2 kernel with a stride of 2 for the max pooling layer.

[0065] Step 2.2, construct the upsampling network;

[0066] The upsampling network consists of four identical upsampling modules. The upsampling module replaces the last pooling layer of the downsampling module with a transposed convolutional layer, which has a 2*2 kernel and a stride of 2.

[0067] Step 2.3, construct the transmission module;

[0068] The transmission module includes the splicing of downsampling modules at the same level in the channel dimension and its dense connection with upsampling modules of the same output size.

[0069] Step 2.4, construct the ECAM module;

[0070] First, the four different outputs obtained from the upsampling module are element-wise summed and concatenated by channel dimension, and then input into the channel attention network layer respectively. The two results are then fused through the attention mechanism. The last layer is a convolutional layer with a 1×1 kernel and a stride of 1.

[0071] Step 2.5, assemble each network module according to the attached... Figure 2 Connect them;

[0072] Step 3, train the twin attention change detection network:

[0073] The training sets for anomaly detection and change target detection are respectively input into the Siamese attention change detection network. The Adam optimizer is used with a learning rate of 0.0003, and gradient descent is used for iterative training to obtain two model weight files for anomaly detection and change target detection.

[0074] Step 4: Detect abnormal areas and changing targets:

[0075] Step 4.1: Input the original image and the transformed image together into the anomaly detection model to obtain the anomaly detection results.

[0076] Step 4.2: Input both the original image and the transformed image into the changed target detection model to obtain the changed target detection results.

[0077] This is parallel training, where two tasks are trained simultaneously.

[0078] This invention discloses a remote sensing anomaly region and target detection method based on change detection, used to automatically detect regions with abnormal compositions and hidden targets in remote sensing images. The implementation steps are as follows: 1. Generate anomaly region detection and changed target detection datasets for image restoration and de-densification techniques; 2. Construct a Siamese attention change detection network; 3. Train the Siamese attention change detection network; 4. Detect anomaly regions and changed targets. This invention constructs a Siamese attention change detection network to learn the changes in pixel distribution and color difference between anomaly regions in remote sensing images and the original remote sensing images, achieving automatic detection of anomaly regions. This invention uses a Siamese network to quickly discover regions that have changed, extracts features from images before and after de-densification using the same parameters, and compares and learns the change patterns of the de-densified images. This invention uses the Siamese network concept at each level of the downsampling module, cascading two temporal features into subsequent networks, and removes the restriction on the order of temporal input by using a technique that processes two temporal phases indiscriminately. This invention utilizes dense connections to enable interaction between shallow and deep semantic information. By merging shallow color and shape features with deep gradient features and upsampled pixel classification results, it repeatedly compares and learns the relationships between features at different levels, thereby improving detection capabilities. This invention employs an attention mechanism to fuse detection results from different levels. By using two different feature fusion methods and leveraging the attention mechanism to comprehensively compare the two fusion results, it enhances the model's detection capabilities. This invention uses the same network to detect existing targets in abnormal regions of remote sensing images. By changing the labels of abnormal regions in the dataset to those of hidden targets, it achieves automatic detection of changes in hidden targets.

[0079] The present invention has the following advantages:

[0080] (1) This invention constructs a dual-branch twin attention change detection network to realize the automatic detection of abnormal areas in remote sensing images. By comparing and learning the changes in pixel distribution and color difference between the abnormal areas appearing in the remote sensing images and the original remote sensing images, the automatic detection of abnormal areas is realized.

[0081] (2) In response to the changes in remote sensing images before and after densification, this invention uses a twin network structure to extract features from two temporal remote sensing images of the same input image group using the same weight parameters and stitching them together in the channel dimension. This helps the subsequent network to quickly extract the differences between the two images before and after densification and obtain key feature information.

[0082] (3) This invention uses dense connections to realize the interaction between shallow and deep semantic information. By merging the color and shape features of the shallow layer and the gradient features of the deep layer as well as the upsampled pixel classification results, the connection between features at different levels is repeatedly compared and learned to improve the detection capability.

[0083] (4) This invention uses an attention mechanism to achieve the fusion of detection results at different levels. By using two different feature fusion methods, the attention mechanism is used to comprehensively compare the two fusion results to improve the model's detection capability.

[0084] (5) The present invention uses the same network to detect the original targets in the abnormal area of ​​remote sensing image, and after changing the labels of the abnormal areas in the dataset to the labels of the hidden targets, it realizes the automatic detection of changes on the hidden targets. Attached Figure Description

[0085] Figure 1 This is a flowchart of a remote sensing anomaly region and target detection method based on change detection;

[0086] Figure 2 This is a schematic diagram of the twin attention change detection network structure, a remote sensing anomaly region and target detection method based on change detection.

[0087] Figure 3 This is a schematic diagram of the ECAM module structure, a remote sensing anomaly region and target detection method based on change detection.

[0088] Figure 4a This is the original remote sensing image from Example 1 of a remote sensing anomaly region and target detection method based on change detection.

[0089] Figure 4b Example 1: A remote sensing image segmentation label map and inflation map corresponding to a remote sensing image, which is an example of a remote sensing anomaly region and target detection method based on change detection.

[0090] Figure 4c Example 1: Semantic segmentation label map corresponding to a remote sensing image for a remote sensing anomaly region and target detection method based on change detection;

[0091] Figure 4d This is an example of a remote sensing anomaly region and target detection method based on change detection, shown in Implementation 1, after the data reduction process.

[0092] Figure 4e This is an example of an anomaly detection result diagram for a remote sensing anomaly region and target detection method based on change detection, as shown in Example 1.

[0093] Figure 4f This is a diagram showing the result of changing target detection in Example 1 of a remote sensing anomaly region and target detection method based on change detection. Detailed Implementation

[0094] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0095] Example 1

[0096] like Figures 1-3 As shown, a remote sensing anomaly region and target detection method based on change detection includes the following steps:

[0097] Step 1: Generate datasets for anomaly region detection and changed target detection targeting image inpainting and de-densification techniques.

[0098] Step 1.1: Randomly select 30 high-resolution remote sensing images and their corresponding label images from the Massachusetts building dataset that have a relatively balanced ratio between the foreground and background.

[0099] Step 1.2: Crop the selected high-resolution image and the corresponding label image to a size of 256×256;

[0100] Step 1.3: Filter labels and their corresponding remote sensing images where the foreground pixel ratio of the cropped label is greater than 10%;

[0101] This approach considers that most targets in remote sensing images are non-densification regions, meaning the background pixel ratio is too large. During training, this can cause the model to prioritize background region segmentation accuracy. Therefore, a threshold is set to filter out images with excessively large background pixel ratios. Let N0 represent the number of background pixels, N1 represent the foreground (i.e., the number of pixels in the changing region), and N be the total number of pixels in the image. The remaining images should satisfy the following conditions:

[0102]

[0103] Step 1.4: Process the image using a de-encryption method to obtain the de-encrypted image and label.

[0104] Step 2, construct a twin attention change detection network:

[0105] Reference Figure 2 The twin attention change detection network constructed in this invention will be described in further detail.

[0106] Step 2.1, construct the twin network branches;

[0107] The Siamese network consists of four identical downsampling modules. Each downsampling module first element-wise adds the result of the first convolutional layer to the result of the BRCB (BatchNormal+ReLU+Conv+BatchNormal) module, and then inputs it into the ReLU activation layer and the max pooling layer. The parameters of the convolutional layers in the first convolutional layer and the BRCB module are: 3×3 kernel with a stride of 1, and 2×2 kernel with a stride of 2 for the max pooling layer. The purpose of the max pooling layer here is to achieve downsampling, while the previous network structures are for feature extraction.

[0108] Siamese networks can conveniently extract different feature information between images in the same group, and quickly extract remote sensing images processed by de-diffusing techniques. To meet the requirements of non-cooperative target tasks, this network uses identical Siamese downsampling layers, that is, it uses the same downsampling module for both temporal inputs, which improves the ability to extract different features between the two temporal phases, and at the same time enhances the network's adaptability to different input orders.

[0109] Step 2.2, construct the upsampling network;

[0110] The upsampling network consists of four identical upsampling modules. Each upsampling module replaces the last pooling layer of the downsampling module with a transposed convolutional layer, which has a 2×2 kernel and a stride of 2. Similar to the downsampling module, except for the transposed convolutional layer restoring image resolution, the preceding network structure performs feature extraction.

[0111] The upsampling network analyzes the relationships between pixels at different locations in the image based on existing features and provides preliminary detection results.

[0112] Step 2.3, construct the transmission module;

[0113] The transmission module includes the splicing of downsampling modules at the same level in the channel dimension and its dense connection with upsampling modules of the same output size.

[0114] Step 2.4, construct the ECAM module;

[0115] First, the four different outputs obtained from the upsampling module are element-wise summed and concatenated by channel dimension, and then input into the channel attention network layer respectively. The two results are then fused through the attention mechanism. Finally, a convolutional layer with a 1×1 kernel and a stride of 1 is added after this module.

[0116] Here, according to the appendix Figure 3 A detailed explanation of this module:

[0117] The ECAM module receives four outputs from the upsampled network and processes them through two branches. Branch one concatenates the four outputs along the channel dimension and inputs the result into the channel attention layer. Branch two element-wise adds the four outputs and inputs the result into the channel attention layer to obtain the extraction result. This result is then copied four times and concatenated along the channel dimension to ensure the number of channels matches the output from branch one. The result from branch two is then element-wise added to the original concatenated four outputs, and then element-wise multiplied with the result from branch one. Finally, the result is passed through a convolutional layer to obtain the network's final output.

[0118] This section employs two fusion methods, fusing outputs from four levels. Branch one focuses on the dimensional relationships between different levels, while branch two emphasizes the positional relationships between pixels across different levels. Comparing the two fusion results using an attention mechanism improves the detection accuracy of the de-dense regions. The formula for this module is:

[0119]

[0120] Where [x1,x2,x3,x4] are the outputs of the four levels of the upsampling module, and CAM is the channel attention mechanism.

[0121] Step 2.5, assemble each network module according to the attached... Figure 2 Connect them;

[0122] Step 3, train the twin attention change detection network:

[0123] The training sets for anomaly detection and changing target detection were respectively input into the Siamese attention-based change detection network. Both datasets were trained using the Adam optimizer with a learning rate of 0.0003, employing gradient descent for iterative training to obtain two model weight files for anomaly detection and changing target detection.

[0124] The training set is randomly sorted to shuffle the order of phases 1 and 2, in order to train the network's ability to detect non-cooperative targets with random phase input order.

[0125] Step 3.1: Input the temporal image into the twin branch to obtain the first feature extracted by the twin branch at four levels.

[0126] Step 3.2: Input the temporal phase 2 image into the twin branch to obtain the feature 2 extracted by the twin branch at four levels.

[0127] Step 3.4: Concatenate feature 1 and feature 2 at different channel dimensions and input them into the corresponding dense connection layer.

[0128] Step 3.5: Each level of dense connection layer is concatenated with the corresponding upsampling layer in the channel dimension and input into the next level of upsampling layer.

[0129] Step 3.6: Input the results of each upsampling layer into the ECAM module to obtain the final detection results.

[0130] Step 3.7: Calculate the sum of the weighted cross-entropy function and the Dice function based on the detection results, and perform gradient backpropagation.

[0131] The formula for calculating the loss function is:

[0132]

[0133] in,

[0134] As a weight for cross-entropy, it balances the proportion of different pixels and reduces the model's shift towards background detection caused by excessive background.

[0135] Step 3.8: Repeat steps 3.1 to 3.7 until the training batch reaches the set value or the loss function no longer converges.

[0136] Step 4: Detect abnormal areas and changing targets:

[0137] Step 4.1: Input the original image and the transformed image together into the anomaly detection model to obtain the anomaly detection results.

[0138] Step 4.2: Input both the original image and the transformed image into the changed target detection model to obtain the changed target detection results.

[0139] Step 4.3: The abnormal region detection results and the changed target detection results of the cropped remote sensing image are stitched together according to their serial numbers.

[0140] The effects of this invention can be further illustrated by the following simulation experiments.

[0141] Simulation experimental conditions:

[0142] The hardware platform for the simulation experiment of this invention is as follows: the processor is an Intel i9-10940X with a main frequency of 3.30GHz, the memory is 64G, the graphics card is an NVIDIA GeForce RTX 2080Ti with 12GB of video memory.

[0143] The software platform for the simulation experiment of this invention is: Windows 10 operating system and Python 3.6.

[0144] The simulation experiment of this invention uses 30 randomly selected images from the Massachusetts remote sensing building dataset. This dataset is composed of aerial images of the Boston area. The original dataset contains 151 aerial images with a resolution of 1500×1500. The labels are mainly for buildings and include buildings of various shapes.

[0145] Simulation experiment content and result analysis:

[0146] The simulation experiment of this invention is carried out according to the following steps.

[0147] The image inpainting method chosen is based on deep learning-based image restoration networks.

[0148] The simulation experiment involved processing the original image, reducing its density, and then detecting the modified areas and key changed targets.

[0149] Step A involves cropping randomly selected images from the Massachusetts dataset to a size of 256×256 as the standard image size for this experiment, with a foreground ratio threshold of 10%.

[0150] Step B involves de-encrypting the training set to obtain the de-encrypted images corresponding to the original images, which are then used as image groups. The final training set contains 3960 images, the validation set 144 images, and the test set 360 images.

[0151] Step C: Input the image groups and labels from the training set into the Siamese attention change detection network. During training, the image groups are randomly ordered, and the temporal order is shuffled. The model is saved every 2 epochs and tested on the validation set.

[0152] Step D involves inputting the image set from the test set into the trained twin attention change detection network to obtain the detection results of abnormal regions and changed targets.

[0153] The following is combined Figures 4a-4f The simulation diagrams further illustrate the effects of the present invention.

[0154] Figure 4a The images are the original remote sensing images from the test set. Figure 4b This is the result of dilation of the segmentation label map corresponding to the remote sensing image. Figure 4c This is the semantic segmentation label map corresponding to the remote sensing image. Figure 4d This is the effect after the declassification process. Figure 4e This is the result of anomaly detection. Figure 4f The result is for detecting changing targets.

[0155] Depend on Figure 4a , Figure 4b As a twin attention change detection network for image group input, the labels and detection results for anomaly region detection are as follows: Figure 4b , Figure 4e The labels of the changing targets and the target detection results are Figure 4c , Figure 4f By comparing the results pairwise, it can be seen that the detection results of such abnormal regions are very close to the abnormal region labels, while the detection results of the changing targets, although not reaching extremely high similarity, are still very good.

[0156] Using the formulas below, we calculate three evaluation metrics: Recall, Acc, and Intersection over Union (IoU) to evaluate the segmentation results. All the calculation results are plotted in Table 1.

[0157] The formula for recall is as follows:

[0158]

[0159] Where TP is the number of samples that are actually positive and whose predicted results are also positive, and FN is the number of samples that are actually positive and whose predicted results are negative.

[0160] The formula for accuracy (Acc) is as follows:

[0161]

[0162] Wherein, FP is the number of samples that are actually negative but are predicted to be positive, and TN is the number of samples that are actually negative and are also predicted to be negative.

[0163] The formula for the Intersection over Union (IoU) is as follows:

[0164]

[0165] Table 1. Quantitative analysis of the method of the present invention in anomaly region detection and changing target detection in simulation experiments.

[0166]

[0167] In Table 1, higher values ​​for Recall, Acc, and IoU indicate more accurate segmentation results. Table 1 shows that the present invention achieves a recall of 98.96%, a precision of 99.16%, and an IoU of 99.02% for anomaly region detection, indicating that it can achieve near 100% anomaly region detection when the modified region is known. For changing target detection, the present invention achieves a recall of 95.98%, a precision of 96.67%, and an IoU of 83.93%, indicating that it can achieve excellent detection results for changing targets. This demonstrates the feasibility of using the same model to simultaneously achieve anomaly region detection and changing target detection, effectively realizing the detection of anomalies and changing targets after de-identification processing.

[0168] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A remote sensing anomaly region and target detection method based on change detection, characterized in that: Includes the following steps: S1. Select remote sensing images and corresponding anomaly region label images and change region label images, crop, filter and reduce density to obtain anomaly region detection dataset and change target detection dataset for image restoration and density reduction methods. S2. Construct a twin attention change detection network, including a twin network composed of identical downsampling modules, a transmission module, an upsampling network composed of identical upsampling modules, and an ECAM module, wherein the twin network, the transmission module, the upsampling network, and the ECAM module are connected in sequence; S3. The abnormal region detection dataset and the changed target detection dataset are randomly sorted and then input into the Siamese network to obtain feature one and feature two respectively. These features are then sequentially input into the transmission module, the upsampling network and the ECAM module. After iterative training, two model weight files for abnormal region detection and changed target detection are obtained, and the abnormal region detection model and the changed target detection model are obtained. S4. The original image to be detected and the corresponding changed image are grouped into an image group, cropped, and then detected in the anomaly detection model and the changed target detection model respectively to obtain the local detection results of the anomaly area and the local detection results of the changed target. The local detection results of the anomaly area and the local detection results of the changed target are then stitched together according to their serial numbers to obtain the anomaly detection result and the changed target detection result. The remote sensing anomaly area and target detection method based on change detection is completed.

2. The remote sensing anomaly region and target detection method based on change detection according to claim 1, characterized in that: Step S1 includes: S11. Randomly select at least 30 remote sensing images from the semantic segmentation dataset that have a balanced foreground-to-background ratio and the corresponding abnormal region label images and change region label images; S12. The remote sensing image, the abnormal area label image, and the change area label image are cropped to the same size in sequence to obtain the cropped remote sensing image, the cropped abnormal area label image, and the cropped change area label image. S13. Perform filtering. The filtering method is to filter out the cropped abnormal region label image, the cropped change region label image, and the corresponding cropped remote sensing image where the proportion of foreground pixels after cropping is greater than the filtering threshold. Where threshold is the filtering threshold, N1 is the number of foreground pixels, and N is the total number of pixels in the image; S14. The selected cropped remote sensing images, cropped abnormal region label images, and cropped changed region label images are processed using a density reduction method to obtain the abnormal region detection dataset and the changed target detection dataset.

3. The remote sensing anomaly region and target detection method based on change detection according to claim 2, characterized in that: In step S12, the cropped size is 256×256; in step S12, the filtering threshold is 10%.

4. The remote sensing anomaly region and target detection method based on change detection according to claim 1, characterized in that: Step S2 includes: S21. Construct the Siamese network, which consists of four identical downsampling modules. Each downsampling module first adds the result of the first convolutional layer to the BRCB module in the form of element-wise addition, and then inputs it into the first ReLU activation layer and the max pooling layer. S22. Construct the upsampling network, which consists of four identical upsampling modules, wherein the last pooling layer of the downsampling module is replaced by a transposed convolutional layer in the upsampling module. S23. Construct the transmission module, which includes the splicing of downsampling modules at the same level in the channel dimension and the dense connection with upsampling modules of the same output size; S24. Construct the ECAM module. First, perform element-wise addition and channel-dimensional concatenation on the four different outputs obtained by the upsampling module. Then, input them into the channel attention network layer to fuse the two results using the attention mechanism. S25. The twin network, the transmission module, the upsampling network, and the ECAM module are connected sequentially to obtain the twin attention change detection network.

5. The remote sensing anomaly region and target detection method based on change detection according to claim 4, characterized in that: The BRCB module in step S21 includes a first batch of normalization layers, a second ReLU activation layer, a second convolutional layer, and a second batch of normalization layers. The convolutional layer parameters of the first convolutional layer and the second convolutional layer are: a 3×3 kernel with a stride of 1, and a 2×2 kernel with a stride of 2 for the max pooling layer. The first convolutional layer, the BRCB module, and the first ReLU activation layer extract features, and the max pooling layer performs downsampling. In step S22, the convolution kernel is 2×2 and the stride is 2; In step S24, the last layer of the ECAM module is a convolutional layer with a 1×1 kernel and a stride of 1.

6. The remote sensing anomaly region and target detection method based on change detection according to claim 4, characterized in that: In step S24, the ECAM module receives the four output results obtained by the upsampling network and processes them through two branches. Branch one concatenates the four outputs along the channel dimension and inputs them into the channel attention layer. Branch two adds the four outputs element-wise and inputs them into the channel attention layer to obtain the extraction result. The result is then copied four times and concatenated along the channel dimension to make the number of channels the same as the output obtained by branch one. The result of branch two is added element-wise to the result of concatenating the original four outputs along the channel dimension and then multiplied element-wise with the result of branch one. Finally, the network's final output is obtained through a convolutional layer. The method of the ECAM module is as follows: Where [x1,x2,x3,x4] are the outputs of the four levels of the upsampling module, CAM is the channel attention mechanism, and repeat... (4) It returns four times.

7. The remote sensing anomaly region and target detection method based on change detection according to claim 1, characterized in that: In step S3, the abnormal region detection dataset and the changed target detection dataset are trained simultaneously. The training process uses the Adam optimizer, the learning rate is set to 0.0003, and gradient descent is used for iterative training.

8. The remote sensing anomaly region and target detection method based on change detection according to claim 1, characterized in that: Step S3 includes: S31. The abnormal region detection dataset and the changed target detection dataset are randomly sorted to obtain a new time phase one image and a new time phase two image. S32. Input the new temporal image into a twin branch of the twin network to obtain feature 1 extracted by the four-level twin branches; S33. Input the new temporal phase 2 image into another twin branch of the twin network to obtain feature 2 extracted by the four-level twin branches; S34. Feature 1 and Feature 2 are spliced ​​together in the channel dimensions at different levels and input into the dense connection layer of the corresponding transmission module. S35. Each level of dense connection layer is concatenated with the corresponding upsampling layer in the channel dimension and input to the next level of upsampling layer; S36. The results of each upsampling layer are input into the ECAM module to obtain the detection results of abnormal and changed regions; S37. Calculate the sum L of the weighted cross-entropy function and the dice function based on the detection results of abnormal and changed regions, and perform gradient backpropagation. S38. Repeat steps S31 to S37 until the training batch reaches the set value or the loss function no longer converges, to obtain two model weight files for anomaly detection and changing target detection, and to obtain the anomaly detection model and the changing target detection model.

9. The remote sensing anomaly region and target detection method based on change detection according to claim 8, characterized in that: In step S35, the dense connection layer learns the relationship between features at different levels by merging the color and shape features of the shallow layer and the gradient features and upsampled pixel classification results of the deep layer. In step S37, the loss function L is: in N is the total number of pixels in the image. c For pixels of class c, w c For class c weights, y i For the pixels of the original image, y i Y' represents the predicted pixel, Y is the original image, Y' is the predicted image, and softmax is the activation function.

10. A remote sensing anomaly region and target detection method based on change detection according to claim 8, characterized in that: In step S31, both the abnormal region detection dataset and the changed target detection dataset are divided into a validation set and a test set. In step S36, the model is saved every 2 epochs of training, and the test set is tested in the validation set.

Images