Edge collaborative change detection method and device based on edge perception graph guidance

Abstract

The invention discloses an edge collaborative change detection method and device based on edge perception graph guidance, and belongs to the technical field of remote sensing image processing, and the method comprises the steps: selecting a change detection data set with front and back time phase images and label data from a remote sensing satellite image data set, and carrying out the preprocessing of the front and back time phase images; based on a Gaussian inverse distance weighting mechanism, obtaining a change edge and influence information of the edge on internal pixels of a change area from the label data, and generating an edge perception graph; extracting multi-scale hierarchical features from the change detection data set; performing edge feature enhancement and fusion processing on the multi-scale layered features to obtain edge sensing features; fusing the multi-scale layered features; performing information interaction on the edge sensing features and the fused change features to obtain edge collaborative change features; and carrying out network training on the edge collaborative change characteristics by using a mixed loss function, and outputting a change detection result. According to the method, the collaboration problem of edge learning and change detection tasks is solved.

Description

Technical Field

[0001] The invention relates to an edge collaborative change detection method and device based on edge perception map guidance, belonging to the technical field of remote sensing image processing. Background Art

[0002] At present, with high-resolution remote sensing satellite images as data sources, change detection based on deep learning methods has become one of the key means to quickly grasp the surface coverage status of a large area, and it is also a research hotspot in the field of remote sensing. High-resolution remote sensing satellite images contain rich and clear appearance information such as edges, shapes, and shadows, which provide sufficient basis for distinguishing changing objects. As an important component of the complex appearance information of the image, edge information depicts the location and shape of the object distribution. Therefore, the integrity of the edge is extremely important for improving the accuracy of change detection results and practical applications.

[0003] Prior to this, research in the field of remote sensing image processing has shown that the reasonable addition of edge-related prior knowledge can assist the network to learn more edge features, thereby improving the accuracy of tasks such as target detection and semantic segmentation. Similarly, in the field of remote sensing change detection, many researchers have helped improve the accuracy of change detection by adding edge priors and introducing edge learning tasks. For example, the edge-aware decoder designed in EATder calculates the edge loss by comparing the generated edge map with the edge prior; ERFM uses the canny operator to obtain the edge prior to train the edge extraction branch, thereby enhancing the expression of edge features; EGPNet adopts a dual-branch structure to add the features obtained by the edge branch to the multi-scale hierarchical features. For building change detection tasks with densely distributed targets, small areas, but regular edges, adding edge priors is more applicable. In this regard, existing networks such as EGDE-Net, IEEF, and EGFDFN obtain edge prior knowledge by extracting building edges from labels and using it for network training or loss calculation, thereby guiding the model to increase its attention to edges, and have achieved certain results.

[0004] However, existing networks only emphasize the importance of edge features, but ignore the potential impact of edges on the internal features of the change region. Moreover, in some edge-coordinated change detection models, there is a lack of effective information interaction between change detection and edge learning tasks, which makes the effect of edge coordination poor. Summary of the invention

[0005] The present invention proposes an edge collaborative change detection method and device based on edge perception graph guidance, which can solve the collaborative problem of edge learning and change detection tasks and achieve more accurate edge learning.

[0006] The technical solution adopted by the present invention to solve the technical problem is: In a first aspect, an embodiment of the present invention provides an edge collaborative change detection method based on edge perception map guidance, comprising the following steps: Step S1, selecting a change detection dataset having before and after phase images and label data from a high-resolution remote sensing satellite image dataset, and preprocessing the before and after phase images in the change detection dataset; Step S2, based on the Gaussian inverse distance weighting mechanism, obtain the change edge and the influence information of the edge on the internal pixels of the change area from the label data in the change detection data set, and generate an edge perception map; Step S3, extracting multi-scale hierarchical features from the preprocessed change detection dataset; Step S4, performing edge feature enhancement and fusion processing on the multi-scale hierarchical features to obtain edge perception features; Step S5, fusing the multi-scale hierarchical features to obtain fused change features; Step S6, performing information interaction between the edge perception feature and the fused change feature to obtain the edge coordinated change feature; Step S7, using a hybrid loss function consisting of edge perception loss and change detection loss, network training is performed on the change features of edge collaboration, and the change detection results are output.

[0007] As a possible implementation of this embodiment, the preprocessing of the before and after phase images in the change detection data set includes: One or more data enhancement methods including random addition of fog effect, random addition of noise, random blur, random grayscale and contrast-limited adaptive histogram equalization are used to enhance the before and after phase image data.

[0008] As a possible implementation of this embodiment, step S2, based on the Gaussian inverse distance weighted mechanism, obtains the change edge and the influence information of the edge on the internal pixels of the change area from the label data in the change detection data set, and generates an edge perception map, including: Step S21, using the canny operator to extract the change edge from the label data in the change detection data set: (1) Step S22, calculate the distance from the internal pixel of the change area to the nearest edge point, and weight the pixel point according to the Gaussian function: (2) In the formula, is the pixel value inside the weighted change area, Indicates the lowest weight of pixels in the change area, is the Euclidean distance between the point and the nearest edge point of the changing object, is the attenuation coefficient; Step S23, weighting the changed edge points: (3).

[0009] As a possible implementation of this embodiment, step S3, extracting multi-scale hierarchical features from the preprocessed change detection data set, includes: The size is The anterior and posterior phase images are input into the residual network Resnet18, and the residual network is used to extract multi-scale hierarchical features from the anterior and posterior dual phase images. After passing through 4 basic blocks, the output resolution is The hierarchical features and hierarchical features ,in , .

[0010] As a possible implementation of this embodiment, step S4, performing edge feature enhancement and fusion processing on multi-scale hierarchical features to obtain edge perception features, includes: Step S41: extract the hierarchical features and hierarchical features Subtract the absolute value to obtain the difference feature: (4) Step S42: Differential features Perform 1×1 convolution Conv, standardize BatchNorm and Relu activation operations to extract features : (5) Step S43, feature Perform three parallel directional convolutions to obtain the features : (6) (7) The convolution kernel sizes of the three parallel directional convolutions are 1×3, 3×1, and 3×3, respectively, wherein the 1×3 and 3×1 directional convolutions are used to extract edge features in the horizontal and vertical directions, respectively, and the 3×3 convolution is used to obtain global spatial information; Step S44, feature Perform batch normalization and activation processing to obtain hierarchical features after directional edge highlighting : (8) Step S45: layering edge features Perform upward edge feature fusion layer by layer to obtain the fused edge feature representation: (9) in, Indicates a 2x upsampling operation, Indicates fusion to The edge features of the layer, Used to extract the spatial information of features. After convolution, the number of channels is adjusted to half of the original number. Contains information about 4 layers of hierarchical edge features, which are upsampled, the number of channels adjusted, and normalized to obtain edge-aware features .

[0011] As a possible implementation of this embodiment, the step S5 of fusing the multi-scale hierarchical features to obtain the fused change features includes: Step S51: hierarchical features and hierarchical features Upsample to the same size as the original image: (10) Step S52, connect the upsampled bi-temporal features of the same layer: (11) Step S53: Fusing the features of each layer to obtain the change feature : (12) in, There are 4 layers of hierarchical change characteristics .

[0012] As a possible implementation of this embodiment, the step S6, performing information interaction between the edge perception feature and the fused change feature to obtain the edge coordinated change feature, includes: Step S61: edge-sensing features With changing characteristics Conduct information interaction and obtain interactive features : (13) Step S62, performing channel-dimensional feature enhancement on the interactive features: (14) (15).

[0013] As a possible implementation of this embodiment, step S7 uses a hybrid loss function composed of edge perception loss and change detection loss to perform network training on the change characteristics of edge collaboration and output a change detection result, including: Step S71, calculating edge perception loss: (16) (17) (18) in, For Huber loss, is the structural similarity loss, and Represent the weights of Huber loss and structural similarity loss respectively, It is a hyperparameter that determines the emphasis of Huber loss on MAE and MSE. and Represent edge perception prediction value and label value respectively, and Represent the means of the predicted values ​​and label values, respectively. and Represent the variance of the predicted value and the label value respectively, represents the covariance of the predicted value and the label value, and represents a small constant used to avoid the problem of zero denominator; Step S72, calculate the change detection loss: (19) (20) (twenty one) in, is the focal loss, is the Dice loss, and Represent the weights of Focal loss and Dice loss respectively, and Represent the change detection prediction value and label value respectively, and is the weight parameter specified in the Focal loss; Step S73, calculate a hybrid loss function consisting of edge perception loss and change detection loss: (twenty two) in, and Represent the edge-aware loss and change detection loss weights respectively; Step S74, using a mixed loss function to perform network training on the change features of edge collaboration and output the change detection results.

[0014] In a second aspect, an embodiment of the present invention provides an edge collaborative change detection device based on edge perception map guidance, comprising: The data set acquisition module is used to select a change detection data set with before and after phase images and label data from a high-resolution remote sensing satellite image data set, and pre-process the before and after phase images in the change detection data set; The edge perception map generation module is used to obtain the change edge and the influence of the edge on the internal pixels of the change area from the label data in the change detection data set based on the Gaussian inverse distance weighting mechanism, and generate the edge perception map; A hierarchical feature extraction module, used to extract multi-scale hierarchical features from the preprocessed change detection dataset; The edge feature processing module is used to perform edge feature enhancement and fusion processing on multi-scale hierarchical features to obtain edge perception features; Hierarchical feature fusion module, used to fuse multi-scale hierarchical features to obtain fused change features; The edge collaboration module is used to exchange information between the edge perception features and the fused change features to obtain the edge collaborative change features; The network training module is used to use a hybrid loss function consisting of edge perception loss and change detection loss to perform network training on the change features of edge collaboration and output change detection results.

[0015] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory through the bus, and the processor executes the machine-readable instructions to perform any step of the above-mentioned edge collaborative change detection method guided by the edge perception graph.

[0016] In a fourth aspect, an embodiment of the present invention provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned edge collaborative change detection methods guided by edge perception graphs are executed.

[0017] The beneficial effects of the technical solution of the embodiment of the present invention are as follows: The technical solution of the embodiment of the present invention is an edge collaborative change detection method based on edge perception map guidance, which not only solves the coordination problem of edge learning and change detection tasks, but also proposes an edge perception map based on Gaussian function and inverse distance weighting mechanism, thereby modeling the influence of the changing edge on the internal pixels of the changing area, and realizing more accurate edge learning. The present invention achieves the purpose of accurately outputting the fine boundary representation of the changing object and obtains satisfactory change detection results.

[0018] The present invention performs edge perception and change detection tasks in a unified model, and improves the performance of change detection with the help of edge perception; an edge perception map is formed through an inverse distance weighted mechanism based on a Gaussian function, simulating the influence of the edge on the pixels inside the change area, while reducing the influence of edge detection errors on the change detection results. In addition, the present invention uses edge collaboration to strengthen the information interaction between edge perception and change detection tasks, and enhances the synergistic effect of the model on the two tasks.

[0019] An edge collaborative change detection device based on edge perception map guidance according to the technical solution of an embodiment of the present invention has the same beneficial effects as an edge collaborative change detection method based on edge perception map guidance according to the technical solution of an embodiment of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flow chart of an edge collaborative change detection method based on edge perception graph guidance according to an exemplary embodiment; Figure 2 is a schematic diagram of the structure of an edge collaborative change detection device based on edge perception graph guidance according to an exemplary embodiment; Figure 3 It is a specific implementation schematic diagram of the present invention for detecting a change region in a public data set; Figure 4 It is a partial visualization diagram of the change detection results of the method of the present invention on the CDD dataset. DETAILED DESCRIPTION

[0021] In order to more clearly illustrate the technical features of the solution of the present invention, the present invention is described in detail below through specific implementation methods and in conjunction with the accompanying drawings.

[0022] In view of the problems that the existing edge collaborative change detection model ignores the potential impact of edges on the internal features of the change area, and the edge learning and change detection tasks lack effective information interaction, resulting in poor edge collaborative effect, the present invention proposes an edge collaborative change detection method guided by Gaussian inverse distance weighted edge perception map.

[0023] like Figure 1As shown, an edge collaborative change detection method based on edge perception map guidance provided by an embodiment of the present invention includes the following steps: Step S1, selecting a change detection dataset having before and after phase images and label data from a high-resolution remote sensing satellite image dataset, and preprocessing the before and after phase images in the change detection dataset; Step S2, based on the Gaussian inverse distance weighting mechanism, obtain the change edge and the influence information of the edge on the internal pixels of the change area from the label data in the change detection data set, and generate an edge perception map; Step S3, extracting multi-scale hierarchical features from the preprocessed change detection dataset; Step S4, performing edge feature enhancement and fusion processing on the multi-scale hierarchical features to obtain edge perception features; Step S5, fusing the multi-scale hierarchical features to obtain fused change features; Step S6, performing information interaction between the edge perception feature and the fused change feature to obtain the edge coordinated change feature; Step S7, using a hybrid loss function consisting of edge perception loss and change detection loss, network training is performed on the change features of edge collaboration, and the change detection results are output.

[0024] As a possible implementation of this embodiment, the preprocessing of the before and after phase images in the change detection data set includes: One or more data enhancement methods including random addition of fog effect, random addition of noise, random blur, random grayscale and contrast-limited adaptive histogram equalization are used to enhance the before and after phase image data.

[0025] As a possible implementation of this embodiment, step S2, based on the Gaussian inverse distance weighted mechanism, obtains the change edge and the influence information of the edge on the internal pixels of the change area from the label data in the change detection data set, and generates an edge perception map, including: Step S21, using the canny operator to extract the change edge from the label data in the change detection data set: (1) Step S22, calculate the distance from the internal pixel of the change area to the nearest edge point, and weight the pixel point according to the Gaussian function: (2) In the formula, is the pixel value inside the weighted change area, Indicates the lowest weight of pixels in the change area, is the Euclidean distance between the point and the nearest edge point of the changing object, is the attenuation coefficient; Step S23, weighting the changed edge points: (3).

[0026] As a possible implementation of this embodiment, step S3, extracting multi-scale hierarchical features from the preprocessed change detection data set, includes: The size is The anterior and posterior phase images are input into the residual network Resnet18, and the residual network is used to extract multi-scale hierarchical features from the anterior and posterior dual phase images. After passing through 4 basic blocks, the output resolution is The hierarchical features and hierarchical features ,in , .

[0027] As a possible implementation of this embodiment, step S4, performing edge feature enhancement and fusion processing on multi-scale hierarchical features to obtain edge perception features, includes: Step S41: extract the hierarchical features and hierarchical features Subtract the absolute value to obtain the difference feature: (4) Step S42: Differential features Perform 1×1 convolution Conv, standardize BatchNorm and Relu activation operations to extract features : (5) Step S43, feature Perform three parallel directional convolutions to obtain the features : (6) (7) The convolution kernel sizes of the three parallel directional convolutions are 1×3, 3×1, and 3×3, respectively, wherein the 1×3 and 3×1 directional convolutions are used to extract edge features in the horizontal and vertical directions, respectively, and the 3×3 convolution is used to obtain global spatial information; Step S44, feature Perform batch normalization and activation processing to obtain hierarchical features after directional edge highlighting : (8) Step S45: layering edge features Perform upward edge feature fusion layer by layer to obtain the fused edge feature representation: (9) in, Indicates a 2x upsampling operation, Indicates fusion to The edge features of the layer, Used to extract the spatial information of features. After convolution, the number of channels is adjusted to half of the original number. Contains information about 4 layers of hierarchical edge features, which are upsampled, the number of channels adjusted, and normalized to obtain edge-aware features .

[0028] As a possible implementation of this embodiment, the step S5 of fusing the multi-scale hierarchical features to obtain the fused change features includes: Step S51: hierarchical features and hierarchical features Upsample to the same size as the original image: (10) Step S52, connect the upsampled bi-temporal features of the same layer: (11) Step S53: Fusing the features of each layer to obtain the change feature : (12) in, There are 4 layers of hierarchical change characteristics .

[0029] As a possible implementation of this embodiment, the step S6, performing information interaction between the edge perception feature and the fused change feature to obtain the edge coordinated change feature, includes: Step S61: edge-sensing features With changing characteristics Conduct information interaction and obtain interactive features : (13) Step S62, performing channel-dimensional feature enhancement on the interactive features: (14) (15).

[0030] As a possible implementation of this embodiment, step S7 uses a hybrid loss function composed of edge perception loss and change detection loss to perform network training on the change characteristics of edge collaboration and output a change detection result, including: Step S71, calculating edge perception loss: (16) (17) (18) in, For Huber loss, is the structural similarity loss, and Represent the weights of Huber loss and structural similarity loss respectively, It is a hyperparameter that determines the emphasis of Huber loss on MAE and MSE. and Represent edge perception prediction value and label value respectively, and Represent the means of the predicted values ​​and label values, respectively. and Represent the variance of the predicted value and the label value respectively, represents the covariance of the predicted value and the label value, and represents a small constant used to avoid the problem of zero denominator; Step S72, calculate the change detection loss: (19) (20) (twenty one) in, is the focal loss, is the Dice loss, and Represent the weights of Focal loss and Dice loss respectively, and Represent the change detection prediction value and label value respectively, and is the weight parameter specified in the Focal loss; Step S73, calculate a hybrid loss function consisting of edge perception loss and change detection loss: (twenty two) in, and Represent the edge-aware loss and change detection loss weights respectively; Step S74, using a mixed loss function to perform network training on the change features of edge collaboration and output the change detection results.

[0031] like Figure 2 As shown, an edge collaborative change detection device based on edge perception map guidance provided by an embodiment of the present invention includes: The data set acquisition module is used to select a change detection data set with before and after phase images and label data from a high-resolution remote sensing satellite image data set, and pre-process the before and after phase images in the change detection data set; The edge perception map generation module is used to obtain the change edge and the influence of the edge on the internal pixels of the change area from the label data in the change detection data set based on the Gaussian inverse distance weighting mechanism, and generate the edge perception map; A hierarchical feature extraction module, used to extract multi-scale hierarchical features from the preprocessed change detection dataset; The edge feature processing module is used to perform edge feature enhancement and fusion processing on multi-scale hierarchical features to obtain edge perception features; Hierarchical feature fusion module, used to fuse multi-scale hierarchical features to obtain fused change features; The edge collaboration module is used to exchange information between the edge perception features and the fused change features to obtain the edge collaborative change features; The network training module is used to use a hybrid loss function consisting of edge perception loss and change detection loss to perform network training on the change features of edge collaboration and output change detection results.

[0032] Figure 3 is a schematic diagram of a specific implementation of the present invention for detecting a change region in a public data set, Figure 3 In the above equation, EFFM is the edge feature fusion module, CFFM is the change feature fusion module, DEHM is the directional edge enhancement module, and Ⓓ is the absolute value subtraction. is activation, ⊗ is dot product, full connected is the fully connected layer, relu is the relu activation layer, max pool is the maximum pooling layer, avg pool is the average pooling layer, SSIM loss+Huber loss is the structural similarity loss and Huber loss, and Dice loss+Focal loss is the Dice loss and Focal loss.

[0033] like Figure 3 As shown, the specific process of edge collaborative change detection guided by edge perception graph in the present invention is as follows.

[0034] Step 1: Select a change detection dataset with before and after phase images and a label dataset, and pre-process the before and after phase images.

[0035] The label data set is a public data set CDD and / or LEVIR-CD; the preprocessing of the change detection data set is to enhance the change detection data set by using a data enhancement method of randomly adding fog effects, randomly adding noise, randomly blurring, randomly graying, and contrast-limited adaptive histogram equalization.

[0036] Step 2: According to the change detection label of the data set, an edge perception map is generated based on the Gaussian inverse distance weighting mechanism.

[0037] Step 21, use the canny operator to extract the change edge from the change detection label: (1) The Canny operator is a classic edge detection algorithm that uses a dual threshold detection method to effectively reduce missed detection and false detection rates and is more resistant to noise. Step 22, calculate the distance from the internal pixel of the change area to the nearest edge point, and weight the pixel point according to the Gaussian function: (2) In the formula, is the lowest perceived value of pixels in the change area, is the Euclidean distance between the point and the nearest edge point of the changing object, is the attenuation coefficient, which represents how fast the edge perception value changes with distance; Step 23, in order to better emphasize the edge, all edge points are weighted with a weight of 1: (3).

[0038] Step 3: extract multi-scale hierarchical features from the preprocessed change detection data set, and input the extracted multi-scale hierarchical features into the edge-aware decoder and the change feature decoder respectively.

[0039] A residual network is used to extract multi-scale hierarchical features from the dual-phase images of the CDD or LEVIR-CD dataset. Specifically, the size of The three-dimensional matrix of the front and back phase images are input into the residual network Resnet18, and after passing through the four basic blocks, the output resolution is The hierarchical features ,in , , H is the height of the before and after phase images, W is the width of the before and after phase images, is the number of channels of the before and after phase images.

[0040] Step 4: In the edge-aware decoder, edge feature enhancement and fusion are performed on the multi-scale hierarchical features to obtain an effective edge feature representation.

[0041] Step 41: extract the hierarchical features Subtract the absolute values ​​to obtain the dual-phase difference features: (4); Step 42: Obtaining difference features After that, convolution, normalization and activation operations are performed on it to improve the model's expressiveness, stability and learning efficiency: (5); Step 43, then, the features obtained in the previous step Perform directional convolution. Three parallel directional convolution modules with kernel sizes of 1×3, 3×1, and 3×3 can enhance edge features vertically, horizontally, and globally. The process of passing through the directional convolution module can be expressed as follows: (6) (7); Step 44, Perform batch normalization and activation to obtain hierarchical features after directional edge highlighting : (8); Step 45, obtain layered edge features After that, we send it to the edge feature fusion module to obtain the fused edge feature representation. The fusion process can be expressed as: (9) in, Indicates a 2x upsampling operation, Indicates fusion to The edge features of the layer, Used to extract the spatial information of features. After convolution, the number of channels is adjusted to half of the original number. Contains information about 4 layers of hierarchical edge features, upsamples them, adjusts the number of channels and normalizes them to obtain edge-aware features ,on the one hand, For edge-aware graphs to compute losses, on the other hand, The edge synergy change feature is obtained by fusion with the change feature.

[0042] Step 5: In the change feature decoder, the multi-scale hierarchical features are fused to obtain the fused change features.

[0043] Step 51, upsampling the bi-phase hierarchical features by 2, 4, 8, and 16 times respectively. This process is expressed as: (10); Step 52, connect the upsampled bi-temporal features of the same layer: (11); Step 53: Fuse the features of each layer to obtain the change feature : (12) in, There are 4 layers of hierarchical change characteristics .

[0044] Step 6: Input the edge features and the change features into the edge collaboration module to obtain the change features of the edge collaboration.

[0045] Step 61: Edge-aware features With changing characteristics To interact with information, a method similar to spatial attention is used to integrate edge perception information into the change features to obtain interactive features. : (13); Step 62, perform channel-dimensional feature enhancement on the interactive features: (14) (15).

[0046] Step 7: Use the hybrid loss function consisting of edge perception loss and change detection loss to train the network and output the change detection result.

[0047] Step 71, edge-aware loss is used to supervise the training results of edge branches. Edge-aware loss uses a hybrid loss function of Huber loss and SSIM. Huber loss combines the advantages of mean square error and mean absolute error. It is expressed as MSE when the error is small and as MAE when the error is large. This makes the loss function smoother and more robust to outliers. SSIM loss considers image brightness, contrast and structural information, and can capture and retain spatial difference information well. and are the weights of the two losses, which are 0.4 and 0.6 respectively in this embodiment; (16) and Represents the weight of Huber loss and structural similarity loss, the sum of which is 1; the formula for Huber loss and structural similarity loss is: (17) (18) in, and Represent edge perception prediction value and label value respectively, and Represent the means of the predicted values ​​and label values, respectively. and Represent the variance of the predicted value and the label value respectively, represents the covariance of the predicted value and the label value, and represents a small constant used to avoid the problem of zero denominator; Step 72, calculate the change detection loss composed of Focal loss and Dice loss: (19) in, is the focal loss, is the Dice loss, and Represent the weights of Focal loss and Dice loss respectively, and the sum is 1; Focal Loss is a loss function used to solve the problem of class imbalance. It reduces the weight of easy-to-classify samples (i.e., low-probability samples) to focus on difficult-to-classify samples (i.e., high-probability samples), thereby solving the problem of class imbalance. Dice loss measures the similarity of two samples through the Dice coefficient. It is very sensitive to the change in similarity between the predicted and true values, and is particularly suitable for processing segmentation tasks with class imbalance or blurred boundaries. The two losses are expressed as: (20) (twenty one) in, and Represent the change detection prediction value and label value respectively, and is the weight parameter specified in the Focal loss; Step 73, calculate the hybrid loss function consisting of edge perception loss and change detection loss: (twenty two) and They respectively represent the proportion of edge perception loss and change detection loss, and in this embodiment, each takes 0.5.

[0048] The experimental results on the CDD public dataset show that: from the perspective of the F1 index under the pixel-level accuracy evaluation system, compared with the existing classic and latest change detection methods, the change detection accuracy of the proposed method is higher, and it shows better performance in the change detection task. Table 1 shows the experimental results of the proposed method and the existing change detection methods on CDD, and all values ​​are presented in percentage (%).

[0049] Table 1: Comparative experimental results on CDD

[0050] Table 1 MapsNet * The method detection results are the results of reproducing the network under the Pytorch framework.

[0051] The change detection results obtained by the present invention are visualized, such as Figure 4 shown. Figure 4 In the figure, Figure (a) is the pre-phase image, Figure (b) is the post-phase image, Figure (c) is the reference image, Figure (d) is the detection result of the FC_Siam_diff method, Figure (e) is the detection result of the ChangeFormer method, Figure (f) is the detection result of the MapsNet* method, Figure (g) is the detection result of the BIT-CD method, Figure (h) is the detection result of the AMT_Net method, Figure (i) is the detection result of the USSFC method, Figure (j) is the detection result of the STADE-CDNet method, and Figure (k) is the detection result of the method of the present invention. It can be seen from the above results that the present invention achieves the purpose of accurately outputting the fine boundary representation of the change object and obtains satisfactory change detection results.

[0052] The present invention performs edge perception and change detection tasks in a unified model, and improves the performance of change detection with the help of edge perception; an edge perception map is formed through an inverse distance weighted mechanism based on a Gaussian function, simulating the influence of the edge on the pixels inside the change area, while reducing the influence of edge detection errors on the change detection results. In addition, the present invention strengthens the information interaction between edge perception and change detection tasks through the application of edge collaboration, and enhances the synergistic effect of the model on the two tasks.

[0053] An embodiment of the present invention provides an electronic device, including a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory through the bus, and the processor executes the machine-readable instructions to perform any step of the above-mentioned edge collaborative change detection method guided by an edge perception graph.

[0054] Specifically, the above-mentioned memory and processor can be general-purpose memory and processor, which are not specifically limited here. When the processor runs the computer program stored in the memory, it can execute the above-mentioned edge collaborative change detection method guided by the edge perception graph.

[0055] Those skilled in the art will appreciate that the structure of the computer device does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently.

[0056] In some embodiments, the computer device may also include a touch screen that can be used to display a graphical user interface (e.g., a startup interface of an application) and receive user operations on the graphical user interface (e.g., startup operations on an application). The specific touch screen may include a display panel and a touch panel. The display panel may be configured in the form of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc. The touch panel may collect the user's contact or non-contact operations on or near it, and generate pre-set operation instructions, for example, the user uses any suitable object such as a finger, a stylus, or an accessory on or near the touch panel. In addition, the touch panel may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and posture, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into information that the processor can process, and then sends it to the processor, and can receive and execute commands from the processor. In addition, the touch panel can be implemented by various types such as resistive, capacitive, infrared and surface acoustic wave, and any technology developed in the future can also be used to implement the touch panel. Further, the touch panel can cover the display panel, and the user can operate on or near the touch panel covered on the display panel according to the graphical user interface displayed on the display panel. After the touch panel detects the operation on or near it, it is transmitted to the processor to determine the user input, and then the processor provides corresponding visual output on the display panel in response to the user input. In addition, the touch panel and the display panel can be implemented as two independent components or integrated.

[0057] Corresponding to the method for starting the above-mentioned application, an embodiment of the present invention further provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned edge collaborative change detection methods guided by edge perception graphs are executed.

[0058] The application startup device provided in the embodiment of the present application can be specific hardware on the device or software or firmware installed on the device. The device provided in the embodiment of the present application, its implementation principle and the technical effect produced are the same as those in the aforementioned method embodiment. For the sake of brief description, the parts not mentioned in the device embodiment can refer to the corresponding contents in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0059] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0060] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0061] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0062] In addition, each functional module in the embodiments provided in the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0063] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0064] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0065] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0066] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for edge collaborative change detection based on edge perception graph guidance, characterized in that: The steps include: Step S1, selecting a change detection dataset having before and after phase images and label data from a high-resolution remote sensing satellite image dataset, and preprocessing the before and after phase images in the change detection dataset; Step S2, based on the Gaussian inverse distance weighting mechanism, obtain the change edge and the influence information of the edge on the internal pixels of the change area from the label data in the change detection data set, and generate an edge perception map; Step S3, extracting multi-scale hierarchical features from the preprocessed change detection dataset; Step S4, performing edge feature enhancement and fusion processing on the multi-scale hierarchical features to obtain edge perception features; Step S5, fusing the multi-scale hierarchical features to obtain fused change features; Step S6, performing information interaction between the edge perception feature and the fused change feature to obtain the edge coordinated change feature; Step S7, using a hybrid loss function consisting of edge perception loss and change detection loss, network training is performed on the change features of edge collaboration, and the change detection results are output.

2. The edge collaborative change detection method based on edge perception graph guidance according to claim 1 is characterized in that: The step S2, based on the Gaussian inverse distance weighting mechanism, obtains the change edge and the influence information of the edge on the internal pixels of the change area from the label data in the change detection data set, and generates an edge perception map, including: Step S21, using the canny operator to extract the change edge from the label data in the change detection data set: (1) Step S22, calculate the distance from the internal pixel of the change area to the nearest edge point, and weight the pixel point according to the Gaussian function: (2) In the formula, is the pixel value inside the weighted change area, Indicates the lowest weight of pixels in the change area, is the Euclidean distance between the point and the nearest edge point of the changing object. is the attenuation coefficient; Step S23, weighting the changed edge points: (3)。 3. The edge collaborative change detection method based on edge perception graph guidance according to claim 1, characterized in that: The step S3, extracting multi-scale hierarchical features from the preprocessed change detection data set, comprises: The size is The anterior and posterior phase images are input into the residual network Resnet18, and the residual network is used to extract multi-scale hierarchical features from the anterior and posterior dual phase images. After passing through 4 basic blocks, the output resolution is The hierarchical features and hierarchical features ,in , .

4. The edge collaborative change detection method based on edge perception graph guidance according to claim 3 is characterized in that: The step S4, performing edge feature enhancement and fusion processing on the multi-scale hierarchical features to obtain edge perception features, includes: Step S41: extract the hierarchical features and hierarchical features Subtract the absolute value to obtain the difference feature: (4) Step S42: Differential features Perform 1×1 convolution Conv, standardize BatchNorm and Relu activation operations to extract features : (5) Step S43, feature Perform three parallel directional convolutions to obtain the features : (6) (7) The convolution kernel sizes of the three parallel directional convolutions are 1×3, 3×1, and 3×3, respectively, wherein the 1×3 and 3×1 directional convolutions are used to extract edge features in the horizontal and vertical directions, respectively, and the 3×3 convolution is used to obtain global spatial information; Step S44, feature Perform batch normalization and activation processing to obtain hierarchical features after directional edge highlighting : (8) Step S45: layering edge features Perform upward edge feature fusion layer by layer to obtain the fused edge feature representation: (9) in, Indicates a 2x upsampling operation, Indicates fusion to The edge features of the layer, Used to extract the spatial information of features. After convolution, the number of channels is adjusted to half of the original number. Contains information about 4 layers of hierarchical edge features, which are upsampled, the number of channels adjusted, and normalized to obtain edge-aware features .

5. The edge collaborative change detection method based on edge perception graph guidance according to claim 4 is characterized in that: The step S5, fusing the multi-scale hierarchical features to obtain fused change features, includes: Step S51: hierarchical features and hierarchical features Upsample to the same size as the original image: (10) Step S52, connect the upsampled bi-temporal features of the same layer: (11) Step S53: Fusing the features of each layer to obtain the change feature : (12) in, There are 4 layers of hierarchical change characteristics .

6. The edge collaborative change detection method based on edge perception graph guidance according to claim 5, characterized in that: The step S6, performing information interaction between the edge perception feature and the fused change feature to obtain the edge coordinated change feature, includes: Step S61: edge-sensing features and changing characteristics Conduct information interaction and obtain interactive features : (13) Step S62, performing channel-dimensional feature enhancement on the interactive features: (14) (15)。 7. The edge collaborative change detection method based on edge perception graph guidance according to claim 6, characterized in that: The step S7 uses a hybrid loss function composed of edge perception loss and change detection loss to perform network training on the change characteristics of edge collaboration and output the change detection result, including: Step S71, calculating edge perception loss: (16) (17) (18) in, For Huber loss, is the structural similarity loss, and Represent the weights of Huber loss and structural similarity loss respectively, It is a hyperparameter that determines the emphasis of Huber loss on MAE and MSE. and Represent edge perception prediction value and label value respectively, and Represent the means of the predicted values ​​and label values, respectively. and Represent the variance of the predicted value and the label value respectively, represents the covariance of the predicted value and the label value, and represents a small constant used to avoid the problem of zero denominator; Step S72, calculate the change detection loss: (19) (20) (21) in, is the focal loss, is the Dice loss, and Represent the weights of Focal loss and Dice loss respectively, and Represent the change detection prediction value and label value respectively, and is the weight parameter specified in the Focal loss; Step S73, calculate a hybrid loss function consisting of edge perception loss and change detection loss: (22) in, and Represent the edge-aware loss and change detection loss weights respectively; Step S74, using a mixed loss function to perform network training on the change features of edge collaboration and output the change detection results.

8. An edge collaborative change detection device based on edge perception map guidance, characterized in that: include: The data set acquisition module is used to select a change detection data set with before and after phase images and label data from a high-resolution remote sensing satellite image data set, and pre-process the before and after phase images in the change detection data set; The edge perception map generation module is used to obtain the change edge and the influence of the edge on the internal pixels of the change area from the label data in the change detection data set based on the Gaussian inverse distance weighting mechanism, and generate the edge perception map; A hierarchical feature extraction module, used to extract multi-scale hierarchical features from the preprocessed change detection dataset; The edge feature processing module is used to perform edge feature enhancement and fusion processing on multi-scale hierarchical features to obtain edge perception features; Hierarchical feature fusion module, used to fuse multi-scale hierarchical features to obtain fused change features; The edge collaboration module is used to exchange information between the edge perception features and the fused change features to obtain the edge collaborative change features; The network training module is used to use a hybrid loss function consisting of edge perception loss and change detection loss to perform network training on the change features of edge collaboration and output change detection results.

9. An electronic device, characterized in that: It includes a processor, a memory and a bus, the memory stores machine-readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, and the processor executes the machine-readable instructions to perform the steps of the edge collaborative change detection method based on edge perception graph guidance as described in any one of claims 1-7.

10. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, executes the steps of the edge collaborative change detection method based on edge perception graph guidance as described in any one of claims 1 to 7.

Images