An edge feature knowledge distillation guided change detection method and device, computer equipment and storage medium

Abstract

The application discloses an edge feature knowledge distillation guided change detection method and device, computer equipment and a storage medium. Based on the feature knowledge distillation mode, layered edge knowledge distillation is adopted to construct an edge knowledge transmission channel from a teacher network to a student network. Through this improvement, the detection accuracy of the student network is obviously improved, and compared with the fine and complex teacher network, the parameter quantity, calculation quantity and reasoning time are obviously reduced, the network successfully balances between the fine detection performance and the lightweight network structure, and solves the coordination problem of edge learning and change detection tasks. The method balances between the fine edge perception, change detection performance and calculation resource consumption of the model, and provides reference and guidance for the practical application of the edge collaborative change detection network.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing technology, and relates to a change detection method, device, computer equipment, and storage medium guided by edge feature knowledge distillation. Background Technology

[0002] In recent years, significant progress has been made in the research of deep learning-based change detection methods based on high-resolution remote sensing imagery, which has become one of the effective means to achieve rapid and accurate monitoring of large-scale land cover changes. The rich morphological features of land cover, such as edges, shapes, and shadows, contained in high-resolution remote sensing imagery provide a decisive basis for the accurate identification of change targets. Among these features, edge information is particularly crucial—it can not only clearly define the spatial distribution range of the changed object, but also accurately characterize its geometric shape features. Therefore, maintaining the integrity of edge features is of great significance for improving the accuracy of change detection.

[0003] In the field of remote sensing change detection, many researchers have improved detection accuracy by integrating edge prior information or introducing edge learning tasks. For example, the EADDer model innovatively designs an edge-aware decoder architecture, optimizing model performance by calculating the loss function between the generated edge map and edge priors; the ERFM method uses the Canny operator to extract edge prior information and trains a dedicated edge feature extraction branch, thereby strengthening the model's ability to represent edge features; EGPNet uses a dual-branch information fusion approach to enhance the information exchange between edge features and multi-scale hierarchical features of change. Building on this, ESMII-Net introduces an edge-aware map based on Gaussian inverse distance weighting to optimize the edge learning task, further enhancing the model's interactive perception ability of change features and change edge features.

[0004] The methods described above have effectively improved the change detection accuracy of the model. However, edge collaborative change detection networks that directly add edge learning branches often have a large number of parameters and computational costs, which severely restricts their practical application in mobile terminals, edge devices, or resource-constrained scenarios. Therefore, achieving an effective balance between maintaining the model's refined edge perception, change detection performance, and computational resource consumption is key to solving the practical application problem of edge collaborative change detection networks. Summary of the Invention

[0005] To address the issues of large parameter and computational load in existing edge collaborative change detection networks, this invention proposes a change detection method, apparatus, computer equipment, and storage medium guided by edge feature knowledge distillation. By employing feature knowledge distillation, an edge knowledge distillation framework is established from the teacher network to the student network. This approach retains edge perception knowledge while avoiding a significant increase in parameters, achieving an effective balance between refined edge perception, change detection performance, and computational resource consumption.

[0006] One of the technical solutions adopted in this invention is: A change detection method guided by edge feature knowledge distillation is provided. The change detection method includes the following steps: Step 1: Select a change detection dataset containing before-and-after temporal images and label datasets, and preprocess the before-and-after temporal images; Step 2: Construct a dual-branch, multi-task change detection teacher network with refined edge perception and change detection capabilities. The teacher network consists of a twin encoder, an edge perception decoder, a change feature decoder, and an edge collaboration module. Step 3: Based on the basic architecture of the teacher network, construct the change detection student network. The student network consists of a twin encoder, a decoder, and an edge extraction module. Step 4: Construct an edge knowledge distillation framework from the teacher network to the student network. Using the feature knowledge distillation method, design an edge knowledge distillation module and establish a hierarchical edge knowledge transfer channel from the teacher network edge collaboration module to the student network edge extraction module. Step 5: Train the teacher network. Under the joint constraints of edge perception labels and change detection labels, train the model using a hybrid edge perception and change detection loss function to obtain high-precision edge perception and change detection results. Step 6: Freeze the teacher network parameters, use distillation loss to supervise the training of the student network hierarchical edge features, and use the hybrid change detection loss function to supervise the training of the student network change features. Step 7: Evaluate the number of network parameters, computational load, and inference time for the student network.

[0007] Furthermore, in step 1, the label dataset is a public dataset containing before and after temporal images and change labels; the before and after temporal images are preprocessed by using a data augmentation method of randomly adding fog effects, randomly adding noise, randomly blurring, randomly grayscale, and contrast-limited adaptive histogram equalization to enhance the label dataset.

[0008] Further, step 2 includes: Step 21: Input the preceding and following temporal images of size H×W×3 into the ResNet18 residual neural network and extract layered features; where H and W represent the image height and width, respectively; after passing through four feature extraction units, output resolutions of [missing information]. Layered features , ,in, , They represent the first The temporal characteristics of the preceding and following layers, , , The number of layers representing the feature layers; Step 22: Use a multi-dimensional information interaction and fusion module to perform information interaction in the time-space-channel dimension, and enhance the change characteristics after fusion by fusing dual-temporal hierarchical features; First, fusion is achieved through connection operations. : , (1); in, This represents the merged features; Concatenate indicates the join operation; and dim indicates the dimension. Then, a feature grouping strategy is used to process multiple sub-features in parallel. Represents a convolution function for sequential grouping. Indicates the number of groups. Indicates the first Sub-features after grouping: (2); Then, each sub-feature is input into three parallel branches, two of which use 1×1 kernels and the other branch uses a 3×3 kernel; Given two 1×1 branches, perform average pooling operations in the horizontal and vertical directions respectively. Then, concatenate the features obtained from the two branches in the height dimension, followed by 1×1 convolution and non-linear activation operations. Finally, divide the data into width and height features, and multiply the two features with the original sub-features. The above process can be represented as: (3); (4); (5); in, , Features after average pooling along the horizontal and vertical directions. , These represent average pooling operations along the horizontal and vertical directions, respectively. Indicates a connection operation. This represents a 1×1 convolution operation. Represents a non-linear activation function. This represents the two-directional sub-features after aggregation. and These represent operations that divide along the width and height dimensions, respectively. This indicates the final characteristics of information exchange between channels; Capturing spatial dependencies between pixels using a 3×3 convolution operation is represented as follows: (6); in, Representing spatial dimensional features, This represents a 3×3 convolution operation; And exchange channel-level and spatial dimension information in the following manner, wherein Represents matrix multiplication. Indicates average pooling. Represents the normalized activation function. Representing spatial dimensional features, Indicates channel dimension features, Represents a non-linear activation function. Indicates the first Post-interaction features across multiple feature dimensions; (7); Finally, aggregate the grouped sub-features: (8); in, For the final hierarchical features, This indicates a join operation, where g is the number of feature groups and dim represents the dimension. Hierarchical features , The image is fed into the feature fusion module, which upsamples each layer of features by 2, 4, 8, and 16 times respectively to generate features of the same size as the original image. Then, a connection operation is used to obtain the fused feature. Step 23: Input the dual-temporal layered features into the edge-aware decoder to obtain edge-aware features; First, the dual-temporal hierarchical features extracted by the ResNet18 residual neural network are... Subtract the absolute values ​​to obtain the difference features. : (9); In the above formula, This represents a function for subtracting absolute values; Obtain differential features After that, it first needs to be subjected to 1×1 convolution, standardization, and... Activation operation, in the following formula Features indicating activation: (10); in, This represents the activation function. This indicates standardized operation. This represents a 1×1 convolution operation; Subsequently, Oriented convolution is performed, specifically three parallel directional convolution modules with kernel sizes of 1×3, 3×1, and 3×3, respectively. The 1×3 and 3×1 directional convolutions are used to extract edge features in the horizontal and vertical directions, respectively, while the 3×3 convolution is used to obtain global spatial information. The process of performing directional convolution can be represented as: (11); (12); in, This represents a 1×3 convolution operation; This represents a 3×1 convolution operation; This represents a 3×3 convolution operation; , , These represent the sub-features after 1×3, 3×1, and 3×3 convolution operations, respectively. This represents the sub-feature after performing the addition operation; Then, to Perform standardized operations and Activation operation yields hierarchical features. : (13); in, This represents the activation function. Indicates standardized operation; Then, The data is fed into the edge feature fusion module to obtain the fused edge features. The edge feature fusion module adopts a feature pyramid-like structure to fuse features layer by layer. The fusion process can be expressed by the following formula: (14); in, Indicates fusion to the first Layer edge features, This represents a 3×3 convolution operation. Indicates an upsampling operation; Then, the features are upsampled, the number of channels is adjusted, and normalization is performed to obtain edge collaborative decoding features. : (15); in, Represents a non-linear activation function. Indicates an upsampling operation; Step 24: Exchange information between edge-aware features and change features to obtain enhanced change features; First, interactive edge-aware features Characteristics of change Obtain post-interaction features : (16); Then, to Perform channel feature enhancement: (17); (18); in, These represent max pooling and average pooling operations, respectively. Indicates a fully connected operation; This represents the output characteristics of the decoder; Represents a non-linear activation function; Indicates the activation function; , These represent the input features after average pooling and max pooling, respectively.

[0009] Further, step 3 includes: Step 31: Input the preceding and following temporal images of size H×W×3 into the ResNet18 residual neural network and extract hierarchical features; where H and W represent the image height and width, respectively; after passing through four feature extraction units, output the features with a resolution of [missing information]. Layered features , ,in, , They represent the first The temporal characteristics of the layers before and after, among which , In the above formula, The number of layers representing the feature layers; Step 32, fusion through connection operation : , (19); in, This represents the merged features; Concatenate indicates the join operation. Indicates dimension; The fused bi-temporal features are then upsampled layer by layer and added to the features of adjacent layers: + (20); in, Indicates an upsampling operation. This represents the shallow features after fusion. This refers to the changing characteristics of the final output.

[0010] Further, step 4 includes: Step 41, the input feature is the absolute value difference of the two-phase hierarchical features, expressed by the formula: (twenty one); in, This represents a function for subtracting absolute values; and Represents dual-phase characteristics, Represents differences and characteristics; Step 42, calculate the gradient of the feature map, including the gradient in the horizontal and vertical directions; wherein, the horizontal gradient is expressed as: (twenty two); (twenty three); The vertical gradient is represented as: (twenty four); (25); in, and These represent the gradient convolution kernels in the horizontal and vertical directions, respectively. , These represent the horizontal gradient and the vertical gradient, respectively. Step 43: Calculate the gradient magnitude using the horizontal and vertical gradients. : = （26）； Then, for the gradient magnitude Perform Gaussian blur processing, specifically as follows: (27); (28); in, Represents the Gaussian kernel function. The standard deviation of the Gaussian kernel is represented. This represents the final edge features.

[0011] Further, step 5 includes: Step 51, construct the edge-aware loss, including loss With structural similarity loss , represented as: (29); in, This represents edge perception loss. , Parameters for adjusting the weights of the two; The loss is represented as: (30); in, For hyperparameters, it means The emphasis of loss on mean absolute error and mean square error. and These represent the true value and predicted value of the function, respectively. The structural similarity loss is expressed as: (31); in, , The average of the true and predicted pixel values; , These are two constants to avoid the denominator being 0; , These represent the standard deviations of the true and predicted pixel values, respectively. This represents the covariance between the true value and the predicted value; Step 52, construct the change detection loss, including loss and loss , represented as: (32); in, , For loss weights; This indicates the loss in change detection.

[0012] loss The formula is: (33); In the formula, and This represents two adjustable weight parameters. This represents the probability that the model is a positive class. loss The formula is: (34); In the formula, and These represent the predicted and actual areas of change, respectively. Step 53: Construct the total loss function for the teacher network. : (35); in, For the change detection value, For label values, For edge-sensing prediction values, For edge-aware label values, and These represent the proportions of change detection and edge perception losses to the total loss, respectively.

[0013] Further, step 6 includes: Step 61, construct the hierarchical edge distillation loss function The loss function consists of mean squared error loss. and gradient loss The composition, expressed by the formula, is as follows: (36); in, and It is used to balance the mean square error loss. and gradient loss The weighting coefficient of the contribution is taken here. , ; in, and It is used to balance the mean square error loss. and gradient loss The weighting coefficient of the contribution is taken here. , ;: (37); in, This represents the feature value of the teacher model at position i. This represents the feature value of the student model at position i. Indicates the total number of features; Gradient loss Represented as: (38); in, and These represent the gradients of the feature map in the horizontal and vertical directions, respectively. Indicates the total number of features; Step 62: Constrain student network training by using a hybrid loss function.

[0014] Further, step 7 includes Step 71: Calculate the number of floating-point operations for the student network using the following formula. : (39); in, Represents the number of model layers. Representing the Number of input channels of the layer Representing the Number of output channels of the layer Representing the kernel size of the layer and Representing the first The layer outputs the height and width of the feature map; Step 72, calculate the student network parameters using the following formula. : (40); in, Represents the number of model layers. Representing the Number of input channels of the layer Representing the Number of output channels of the layer Representing the Layer convolution kernel size; Step 73, calculate the student's online reasoning time using the following formula. : (41); in, It is the total number of reasoning attempts. It is the first The time required for each reasoning step.

[0015] The second technical solution adopted in this invention is: An edge feature knowledge distillation-guided change detection apparatus is provided for performing the edge feature knowledge distillation-guided change detection method described in any one of the preceding claims, comprising: The dataset acquisition module is used to select a change detection dataset containing before-and-after temporal images and label datasets, and to preprocess the before-and-after temporal images; The feature extraction module is used to extract multi-scale hierarchical features from the preprocessed change detection dataset; The edge extraction module is used to extract change edge-aware features from the preprocessed change detection dataset; The hierarchical edge knowledge distillation module is used to obtain trained hierarchical edge perception features from the teacher's network edge perception module and supervise the edge features of the student's network. The change feature decoding module integrates distilled edge-aware knowledge and hierarchical change features to obtain change features guided by edge knowledge distillation. The network training module trains the network based on hierarchical edge distillation loss and hybrid loss functions, and outputs the results of change region detection.

[0016] The third technical solution adopted in this invention is: A computer device is provided, including a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of any of the edge feature knowledge distillation-guided change detection methods described above.

[0017] The fourth technical solution adopted in this invention is: A storage medium is provided that stores a computer program, which, when executed by a processor, performs the steps of any of the edge feature knowledge distillation-guided change detection methods described above.

[0018] Compared with the prior art, the beneficial effects of the present invention are: This invention, based on feature knowledge distillation, employs hierarchical edge knowledge distillation to construct an edge knowledge transfer channel from the teacher network to the student network. This improvement significantly enhances the detection accuracy of the student network. Simultaneously, compared to the complex teacher network, it significantly reduces the number of parameters, computational cost, and inference time, achieving an effective balance between refined detection performance and a lightweight network structure. This invention achieves an effective balance between maintaining refined edge perception, change detection performance, and computational resource consumption, providing a reference and guidance for the practical application of edge collaborative change detection networks. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating a change detection method guided by edge feature knowledge distillation according to an exemplary embodiment.

[0020] Figure 2 This is a schematic diagram of the structure of an edge feature knowledge distillation-guided change detection device according to an exemplary embodiment.

[0021] Figure 3 This is a schematic diagram illustrating a specific implementation of the present invention for detecting changing regions within a publicly available dataset.

[0022] Figure 4 This is a partial visualization of the change detection results of the method of the present invention on the CDD and LEVIR-CD datasets.

[0023] Figure 5 This is a comparison diagram of the edge-aware features of the teacher network and the hierarchical edge features of the student network in the method of this invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention; that is, the described embodiments are merely some embodiments of the invention, and not all embodiments. The components of the embodiments of the invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0025] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0026] The specific embodiments of the present invention are described below with reference to the accompanying drawings.

[0027] Example 1

[0028] This embodiment provides a change detection method guided by edge feature knowledge distillation, which includes the following steps: Step 1: Select a change detection dataset with before and after temporal images and label datasets, and preprocess the before and after temporal images.

[0029] Step 2: Construct a dual-branch, multi-task change detection teacher network with refined edge perception and change detection capabilities. The teacher network consists of a twin encoder, an edge perception decoder, a change feature decoder, and an edge collaboration module.

[0030] Step 3: Based on the basic architecture of the teacher network, construct the change detection student network, which consists of a twin encoder, a decoder, and an edge extraction module.

[0031] Step 4: Construct an edge knowledge distillation framework from the teacher network to the student network. Using the feature knowledge distillation method, design an edge knowledge distillation module and establish a hierarchical edge knowledge transfer channel from the teacher network edge collaboration module to the student network edge extraction module.

[0032] Step 5: Train the teacher network. Under the joint constraints of edge perception labels and change detection labels, train the model using a hybrid edge perception and change detection loss function to obtain high-precision edge perception and change detection results.

[0033] Step 6: Freeze the teacher network parameters, use distillation loss to supervise the training of the student network hierarchical edge features, and use the hybrid change detection loss function to supervise the training of the student network change features.

[0034] Step 7: Evaluate the number of network parameters, computational load, and inference time for the student network.

[0035] As one possible implementation of this embodiment, in step 1, the label dataset is the public datasets CDD and LEVIR-CD, which contain before-and-after temporal images and change labels; the before-and-after temporal images are preprocessed by using a data augmentation method of randomly adding fog effects, randomly adding noise, randomly blurring, randomly grayscale, and contrast-limited adaptive histogram equalization to enhance the label dataset.

[0036] As one possible implementation of this embodiment, step 2 includes: Step 21: Input the preceding and following temporal images of size H×W×3 into the ResNet18 residual neural network and extract layered features; where H and W represent the image height and width, respectively; after passing through four feature extraction units, output resolutions of [missing information]. Layered features , ,in, , They represent the first The temporal characteristics of the preceding and following layers, , , The number of layers representing the feature layers.

[0037] Step 22: Use a multi-dimensional information interaction and fusion module to perform information interaction in the time-space-channel dimensions, and enhance the fused change features by fusing dual-temporal hierarchical features.

[0038] First, fusion is achieved through connection operations. : , (1); in, The 'dim' indicates the merged features; 'Concatenate' indicates the join operation; and 'dim' indicates the dimension.

[0039] Then, a feature grouping strategy is used to process multiple sub-features in parallel. Represents a convolution function for sequential grouping. Indicates the number of groups. Indicates the first Sub-features after grouping: (2); Then, each sub-feature is input into three parallel branches, two of which use 1×1 kernels and the other branch uses a 3×3 kernel; Given two 1×1 branches, perform average pooling operations in the horizontal and vertical directions respectively. Then, concatenate the features obtained from the two branches in the height dimension, followed by 1×1 convolution and non-linear activation operations. Finally, divide the data into width and height features, and multiply the two features with the original sub-features. The above process can be represented as: (3); (4); (5); in, , Features after average pooling along the horizontal and vertical directions. , These represent average pooling operations along the horizontal and vertical directions, respectively. Indicates a connection operation. This represents a 1×1 convolution operation. Represents a non-linear activation function. This represents the two-directional sub-features after aggregation. and These represent operations that divide along the width and height dimensions, respectively. This indicates the final characteristics of information exchange between channels; Capturing spatial dependencies between pixels using a 3×3 convolution operation is represented as follows: (6); in, Representing spatial dimensional features, This represents a 3×3 convolution operation; And exchange channel-level and spatial dimension information in the following manner, wherein Represents matrix multiplication. Indicates average pooling. Represents the normalized activation function. Representing spatial dimensional features, Indicates channel dimension features, Represents a non-linear activation function. Indicates the first Post-interaction features across multiple feature dimensions; (7); Finally, aggregate the grouped sub-features: (8); in, For the final hierarchical features, This indicates a join operation, where g is the number of feature groups and dim represents the dimension. Hierarchical features , The image is fed into the feature fusion module, which upsamples each layer of features by 2, 4, 8, and 16 times respectively to generate features of the same size as the original image. Then, a connection operation is used to obtain the fused feature. Step 23: Input the dual-temporal layered features into the edge-aware decoder to obtain edge-aware features; First, the dual-temporal hierarchical features extracted by the ResNet18 residual neural network are... Subtract the absolute values ​​to obtain the difference features. : (9); In the above formula, This represents a function for subtracting absolute values; Obtain differential features After that, it first needs to be subjected to 1×1 convolution, standardization, and... Activation operation, in the following formula Features indicating activation: (10); in, This represents the activation function. This indicates standardized operation. This represents a 1×1 convolution operation; Subsequently, Oriented convolution is performed, specifically three parallel directional convolution modules with kernel sizes of 1×3, 3×1, and 3×3, respectively. The 1×3 and 3×1 directional convolutions are used to extract edge features in the horizontal and vertical directions, respectively, while the 3×3 convolution is used to obtain global spatial information. The process of performing directional convolution can be represented as: (11); (12); in, This represents a 1×3 convolution operation; This represents a 3×1 convolution operation; This represents a 3×3 convolution operation; , , These represent the sub-features after 1×3, 3×1, and 3×3 convolution operations, respectively. This represents the sub-feature after performing the addition operation; Then, to Perform standardized operations and Activation operation yields hierarchical features. : (13); in, This represents the activation function. Indicates standardized operation; Then, The data is fed into the edge feature fusion module to obtain the fused edge features. The edge feature fusion module adopts a feature pyramid-like structure to fuse features layer by layer. The fusion process can be expressed by the following formula: (14); in, Indicates fusion to the first Layer edge features, This represents a 3×3 convolution operation. Indicates an upsampling operation; Then, the features are upsampled, the number of channels is adjusted, and normalization is performed to obtain edge collaborative decoding features. : (15); in, Represents a non-linear activation function. Indicates an upsampling operation; Step 24: Exchange information between edge-aware features and change features to obtain enhanced change features; First, interactive edge-aware features Characteristics of change Obtain post-interaction features : (16); Then, to Perform channel feature enhancement: (17); (18); in, These represent max pooling and average pooling operations, respectively. Indicates a fully connected operation; This represents the output characteristics of the decoder; Represents a non-linear activation function; Indicates the activation function; , These represent the input features after average pooling and max pooling, respectively.

[0040] As one possible implementation of this embodiment, step 3 includes: Step 31: Input the preceding and following temporal images of size H×W×3 into the ResNet18 residual neural network and extract hierarchical features; where H and W represent the image height and width, respectively; after passing through four feature extraction units, output the features with a resolution of [missing information]. Layered features , ,in, , They represent the first The temporal characteristics of the layers before and after, among which , In the above formula, The number of layers representing the feature layers.

[0041] Step 32, fusion through connection operation : , (19); in, This represents the merged features; Concatenate indicates the join operation. Indicates dimension.

[0042] The fused bi-temporal features are then upsampled layer by layer and added to the features of adjacent layers: + (20); in, Indicates an upsampling operation. This represents the shallow features after fusion. This refers to the changing characteristics of the final output.

[0043] As one possible implementation of this embodiment, step 4 includes: Step 41, the input feature is the absolute value difference of the two-phase hierarchical features, expressed by the formula: (twenty one); in, This represents a function for subtracting absolute values; and Represents dual-phase characteristics, Represents the differences in characteristics.

[0044] Step 42, calculate the gradient of the feature map, including the gradient in the horizontal and vertical directions; wherein, the horizontal gradient is expressed as: (twenty two); (twenty three); The vertical gradient is represented as: (twenty four); (25); in, and These represent the gradient convolution kernels in the horizontal and vertical directions, respectively. , These represent the horizontal gradient and the vertical gradient, respectively.

[0045] Step 43: Calculate the gradient magnitude using the horizontal and vertical gradients. : = (26); To smooth the gradient magnitude and reduce noise, the gradient magnitude... Perform Gaussian blur processing, specifically as follows: (27); (28); in, Represents the Gaussian kernel function. The standard deviation of the Gaussian kernel is represented. This represents the final edge features.

[0046] As one possible implementation of this embodiment, step 5 includes: Step 51, construct the edge-aware loss, including loss With structural similarity loss , represented as: (29); in, This represents edge perception loss. , The parameters are used to adjust the weights of the two.

[0047] The loss is represented as: (30); in, For hyperparameters, it means The emphasis of loss on mean absolute error and mean square error. and These represent the true value and predicted value of the function, respectively.

[0048] The structural similarity loss is expressed as: (31); in, , The average of the true and predicted pixel values; , These are two constants to avoid the denominator being 0; , These represent the standard deviations of the true and predicted pixel values, respectively. This represents the covariance between the true value and the predicted value.

[0049] Step 52, construct the change detection loss, including loss and loss , represented as: (32); in, , For loss weights; This indicates the loss in change detection.

[0050] loss The formula is: (33); In the formula, and This represents two adjustable weight parameters. This represents the probability that the model is a positive class. loss The formula is: (34); In the formula, and These represent the predicted and actual areas of change, respectively.

[0051] Step 53: Construct the total loss function for the teacher network. : (35); in, For the change detection value, For label values, For edge-sensing prediction values, For edge-aware label values, and These represent the proportions of change detection and edge perception losses to the total loss, respectively.

[0052] As one possible implementation of this embodiment, step 6 includes: Step 61, construct the hierarchical edge distillation loss function The loss function consists of mean squared error loss. and gradient loss The composition, expressed by the formula, is as follows: (36); in, and It is used to balance the mean square error loss. and gradient loss The weighting coefficient of the contribution is taken here. , .

[0053] Mean square error loss Represented as: (37); in, This represents the feature value of the teacher model at the i-th position; This represents the feature value of the student model at the i-th position; This indicates the total number of features, such as the number of pixels in a feature map or the length of a feature vector.

[0054] Gradient loss Represented as: (38); in, and These represent the gradients of the feature map in the horizontal and vertical directions, respectively. This indicates the total number of features, such as the number of pixels in a feature map or the length of a feature vector.

[0055] Step 62: Constrain student network training using a hybrid loss function, where the loss function is composed of... loss and loss The composition and specific steps are the same as those in step 52.

[0056] As one possible implementation of this embodiment, step 7 includes: Step 71: Calculate the number of floating-point operations for the student network using the following formula. : (39); in, Represents the number of model layers. Representing the Number of input channels of the layer Representing the Number of output channels of the layer Representing the kernel size of the layer and Representing the first The layer outputs the height and width of the feature map.

[0057] Step 72, calculate the student network parameters using the following formula. : (40); in, Represents the number of model layers. Representing the Number of input channels of the layer Representing the Number of output channels of the layer Representing the The size of the convolution kernel in the layer.

[0058] Step 73, calculate the student's online reasoning time using the following formula. : (41); in, It is the total number of reasoning attempts. It is the first The time required for each reasoning step.

[0059] Experimental results on the CDD and LEVIR-CD public datasets show that, based on the F1 score in the pixel-level accuracy evaluation system, the change detection accuracy of the teacher network and student network in the method of this invention is basically within the same range (difference of less than 0.5%). However, compared to the student network, the number of floating-point operations, the number of parameters, and the inference time all show a significant decrease. Table 1 shows the experimental results of the teacher network and student network in the method of this invention on CDD and LEVIR-CD, with all values ​​presented as percentages (%).

[0060] Table 1: Experimental results on the CDD and LEVIR-CD datasets

[0061] The change detection results obtained by this invention are visualized, such as... Figure 4As shown above, the method described in this invention can completely detect changing objects. Comparing the hierarchical edge features of the student network with the hierarchical edge perception features of the teacher network, the results show that the student network exhibits a highly consistent feature response pattern with the teacher network in the representation of various edge structures (such as object contours and texture details). This high similarity at the feature level not only verifies the effectiveness of the hierarchical edge knowledge distillation strategy but also confirms from the perspective of visual interpretability that the edge knowledge distillation module proposed in this invention can successfully transfer the edge perception capabilities of the teacher network to the student network.

[0062] Example 2

[0063] This embodiment provides a change detection device guided by edge feature knowledge distillation, including: The dataset acquisition module is used to select a change detection dataset containing before-and-after temporal images and label datasets, and to preprocess the before-and-after temporal images; The feature extraction module is used to extract multi-scale hierarchical features from the preprocessed change detection dataset; The edge extraction module is used to extract change edge-aware features from the preprocessed change detection dataset; The hierarchical edge knowledge distillation module is used to obtain trained hierarchical edge perception features from the teacher's network edge perception module and supervise the edge features of the student's network. The change feature decoding module integrates distilled edge-aware knowledge and hierarchical change features to obtain change features guided by edge knowledge distillation. The network training module trains the network based on hierarchical edge distillation loss and hybrid loss functions, and outputs the results of change region detection.

[0064] Example 3

[0065] This embodiment provides a computer device, characterized in that it includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of the arbitrary edge feature knowledge distillation-guided change detection method described above.

[0066] Specifically, the aforementioned memory and processor can be general-purpose memory and processor, without any specific limitations. When the processor runs the computer program stored in the memory, it can execute the aforementioned edge feature knowledge distillation-guided change detection method.

[0067] Those skilled in the art will understand that the structure of the computer device does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine some components, or split some components, or have different component arrangements.

[0068] Example 4

[0069] This embodiment provides a storage medium, characterized in that the storage medium stores a computer program, which, when run by a processor, executes the steps of the change detection method guided by the distillation of arbitrary edge feature knowledge as described above.

[0070] In some embodiments, the computer device may further include a touchscreen for displaying a graphical user interface (e.g., an application launch screen) and receiving user actions on the graphical user interface (e.g., launching an application). Specifically, the touchscreen may include a display panel and a touch panel. The display panel may be configured as an LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), or similar type. The touch panel can collect user touch or non-touch operations on or near it and generate pre-set operation instructions, such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel. Additionally, the touch panel may include a touch detection device and a touch controller. The touch detection device detects the user's touch position and posture, and detects the signals generated by the touch operation, transmitting the signals to the touch controller. The touch controller receives touch information from the touch detection device, converts it into information that the processor can process, sends it to the processor, and can also receive and execute commands from the processor. Furthermore, touch panels can be implemented using various types of sensors, including resistive, capacitive, infrared, and surface acoustic wave sensors, as well as any future technologies. Moreover, the touch panel can cover the display panel. Users can operate on or near the touch panel, which is covered by the graphical user interface displayed on the display panel. After detecting the operation on or near the touch panel, the touch panel transmits it to the processor to determine the user input. The processor then responds to the user input by providing corresponding visual output on the display panel. Additionally, the touch panel and display panel can be implemented as two separate components or integrated together.

[0071] The application launch device provided in this application embodiment can be specific hardware on the device or software or firmware installed on the device. The device provided in this application embodiment has the same implementation principle and technical effects as the foregoing method embodiments. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the foregoing method embodiments. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can all be referred to the corresponding processes in the above method embodiments, and will not be repeated here.

[0072] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0073] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interface, and the indirect coupling or communication connection of the apparatus or modules may be electrical, mechanical, or other forms.

[0074] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0075] In addition, the functional modules in the embodiments provided in this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0076] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0077] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0078] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0079] The above description is merely an example and illustration of the structure of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the structure of the invention or exceed the scope defined in the claims, all of which should fall within the protection scope of the present invention.

Claims

1. A change detection method guided by edge feature knowledge distillation, characterized in that, The change detection method includes the following steps: Step 1: Select a change detection dataset containing before-and-after temporal images and label datasets, and preprocess the before-and-after temporal images; Step 2: Construct a dual-branch, multi-task change detection teacher network with refined edge perception and change detection capabilities. The teacher network consists of a twin encoder, an edge perception decoder, a change feature decoder, and an edge collaboration module. Step 3: Based on the basic architecture of the teacher network, construct a change detection student network, which consists of a twin encoder, a decoder, and an edge extraction module; Step 4: Construct an edge knowledge distillation framework from the teacher network to the student network. Using the feature knowledge distillation method, design an edge knowledge distillation module and establish a hierarchical edge knowledge transfer channel from the teacher network edge collaboration module to the student network edge extraction module. Step 5: Train the teacher network. Under the joint constraints of edge perception labels and change detection labels, train the model using a hybrid edge perception and change detection loss function to obtain high-precision edge perception and change detection results. Step 6: Freeze the teacher network parameters, use distillation loss to supervise the training of the student network hierarchical edge features, and use the hybrid change detection loss function to supervise the training of the student network change features. Step 7: Evaluate the number of student network parameters, computational load, and inference time; Step 3 includes: Step 31: Input the preceding and following temporal images of size H×W×3 into the ResNet18 residual neural network and extract hierarchical features; where H and W represent the image height and width, respectively; after passing through four feature extraction units, output the features with a resolution of [missing information]. Layered features , ,in, , They represent the first The temporal characteristics of the layers before and after, among which , In the above formula, The number of layers representing the feature layers; Step 32, fusion through connection operation : ， (19)； in, This represents the merged features; Concatenate indicates the join operation. Indicates dimension; The fused bi-temporal features are then upsampled layer by layer and added to the features of adjacent layers: + (20)； in, Indicates an upsampling operation. This represents the shallow features after fusion. This refers to the changing characteristics of the final output.

2. The edge feature knowledge distillation-guided change detection method according to claim 1, characterized in that, In step 1, the label dataset is a public dataset containing before-and-after temporal images and change labels; the preprocessing of the before-and-after temporal images is to enhance the label dataset by using a data augmentation method that randomly adds fog effects, randomly adds noise, randomly blurs, randomly grayscales, and contrast-limited adaptive histogram equalization.

3. The edge feature knowledge distillation-guided change detection method according to claim 1, characterized in that, Step 2 includes: Step 21: Input the preceding and following temporal images of size H×W×3 into the ResNet18 residual neural network and extract layered features; where H and W represent the image height and width, respectively; after passing through four feature extraction units, output resolutions of [missing information]. Layered features , ,in, , They represent the first The temporal characteristics of the preceding and following layers, , , The number of layers representing the feature layers; Step 22: Use a multi-dimensional information interaction and fusion module to perform information interaction in the time-space-channel dimension, and enhance the change characteristics after fusion by fusing dual-temporal hierarchical features; First, fusion is achieved through connection operations. : ， (1)； in, This represents the merged features; Concatenate indicates the join operation; and dim indicates the dimension. Then, a feature grouping strategy is used to process multiple sub-features in parallel. Represents a convolution function for sequential grouping. Indicates the number of groups. Indicates the first Sub-features after grouping: (2)； Then, each sub-feature is input into three parallel branches, two of which use 1×1 kernels and the other branch uses a 3×3 kernel; Given two 1×1 branches, perform average pooling operations in the horizontal and vertical directions respectively. Then, concatenate the features obtained from the two branches in the height dimension, followed by 1×1 convolution and non-linear activation operations. Finally, divide the data into width and height features, and multiply the two features with the original sub-features. The above process can be represented as: (3)； (4)； (5)； in, , Features after average pooling along the horizontal and vertical directions. , These represent average pooling operations along the horizontal and vertical directions, respectively. Indicates a connection operation. This represents a 1×1 convolution operation. Represents a non-linear activation function. This represents the two-directional sub-features after aggregation. and These represent operations that divide along the width and height dimensions, respectively. This indicates the final characteristics of information exchange between channels; Capturing spatial dependencies between pixels using a 3×3 convolution operation is represented as follows: (6)； in, Representing spatial dimensional features, This represents a 3×3 convolution operation; And exchange channel-level and spatial dimension information in the following manner, wherein Represents matrix multiplication. Indicates average pooling. Represents the normalized activation function. Representing spatial dimensional features, Indicates channel dimension features, Represents a non-linear activation function. Indicates the first Post-interaction features across multiple feature dimensions; (7)； Finally, aggregate the grouped sub-features: (8)； in, For the final hierarchical features, This indicates a join operation, where g is the number of feature groups and dim represents the dimension. Hierarchical features , The image is fed into the feature fusion module, which upsamples each layer of features by 2, 4, 8, and 16 times respectively to generate features of the same size as the original image. Then, a connection operation is used to obtain the fused feature. Step 23: Input the dual-temporal layered features into the edge-aware decoder to obtain edge-aware features; First, the dual-temporal hierarchical features extracted by the ResNet18 residual neural network are... Subtract the absolute values ​​to obtain the difference features. : (9)； In the above formula, This represents a function for subtracting absolute values; Obtain differential features After that, it first needs to be subjected to 1×1 convolution, standardization, and... Activation operation, in the following formula Features indicating activation: (10)； in, This represents the activation function. This indicates standardized operation. This represents a 1×1 convolution operation; Subsequently, Oriented convolution is performed, specifically three parallel directional convolution modules with kernel sizes of 1×3, 3×1, and 3×3, respectively. The 1×3 and 3×1 directional convolutions are used to extract edge features in the horizontal and vertical directions, respectively, while the 3×3 convolution is used to obtain global spatial information. The process of performing directional convolution can be represented as: (11)； (12)； in, This represents a 1×3 convolution operation; This represents a 3×1 convolution operation; This represents a 3×3 convolution operation; , , These represent the sub-features after 1×3, 3×1, and 3×3 convolution operations, respectively. This represents the sub-feature after performing the addition operation; Then, to Perform standardized operations and Activation operation yields hierarchical features. : (13)； in, This represents the activation function. Indicates standardized operation; Then, The data is fed into the edge feature fusion module to obtain the fused edge features. The edge feature fusion module adopts a feature pyramid-like structure to fuse features layer by layer. The fusion process can be expressed by the following formula: (14)； in, Indicates fusion to the first Layer edge features, This represents a 3×3 convolution operation. Indicates an upsampling operation; Then, the features are upsampled, the number of channels is adjusted, and normalization is performed to obtain edge collaborative decoding features. : (15)； in, Represents a non-linear activation function. Indicates an upsampling operation; Step 24: Exchange information between edge-aware features and change features to obtain enhanced change features; First, interactive edge-aware features Characteristics of change Obtain post-interaction features : (16); Then, to Perform channel feature enhancement: (17)； (18)； in, These represent max pooling and average pooling operations, respectively. Indicates a fully connected operation; This represents the output characteristics of the decoder; Represents a non-linear activation function; Indicates the activation function; , These represent the input features after average pooling and max pooling, respectively.

4. The edge feature knowledge distillation-guided change detection method according to claim 1, characterized in that, Step 4 includes: Step 41, the input feature is the absolute value difference of the two-phase hierarchical features, expressed by the formula: (21)； in, This represents a function for subtracting absolute values; and Represents dual-phase characteristics, Represents differences and characteristics; Step 42, calculate the gradient of the feature map, including the gradient in the horizontal and vertical directions; wherein, the horizontal gradient is expressed as: (22)； (23)； The vertical gradient is represented as: (24)； (25)； in, and These represent the gradient convolution kernels in the horizontal and vertical directions, respectively. , These represent the horizontal gradient and the vertical gradient, respectively. Step 43: Calculate the gradient magnitude using the horizontal and vertical gradients. : = (26)； Then, for the gradient magnitude Perform Gaussian blur processing, specifically as follows: (27)； (28)； in, Represents the Gaussian kernel function. The standard deviation of the Gaussian kernel is represented. This represents the final edge features.

5. The edge feature knowledge distillation-guided change detection method according to claim 1, characterized in that, Step 5 includes: Step 51, construct the edge-aware loss, including loss With structural similarity loss , is represented as: (29)； in, This represents edge perception loss. , Parameters for adjusting the weights of the two; The loss is represented as: (30)； in, For hyperparameters, it means The emphasis of loss on mean absolute error and mean square error. and These represent the true value and predicted value of the function, respectively. The structural similarity loss is expressed as: (31)； in, , The average of the true and predicted pixel values; , These are two constants to avoid the denominator being 0; , These represent the standard deviations of the true and predicted pixel values, respectively. This represents the covariance between the true value and the predicted value; Step 52, construct the change detection loss, including loss and loss , is represented as: (32)； in, , For loss weights; Indicates the loss in change detection; loss The formula is: (33)； In the formula, and This represents two adjustable weight parameters. This represents the probability that the model is a positive class. loss The formula is: (34)； In the formula, and These represent the predicted and actual areas of change, respectively. Step 53: Construct the total loss function for the teacher network. : (35)； in, For the change detection value, For label values, For edge-sensing prediction values, For edge-aware label values, and These represent the proportions of change detection and edge perception losses to the total loss, respectively.

6. The edge feature knowledge distillation-guided change detection method according to claim 1, characterized in that, Step 6 includes: Step 61, construct the hierarchical edge distillation loss function The loss function consists of mean squared error loss. and gradient loss The composition, expressed by the formula, is as follows: (36)； in, and It is used to balance the mean square error loss. and gradient loss The weighting coefficient of the contribution is taken here. , ; Mean square error loss Represented as: (37)； in, This represents the feature value of the teacher model at position i. This represents the feature value of the student model at position i. Indicates the total number of features; Gradient loss Represented as: (38)； in, and These represent the gradients of the feature map in the horizontal and vertical directions, respectively. Indicates the total number of features; Step 62: Constrain student network training by using a hybrid loss function.

7. The edge feature knowledge distillation-guided change detection method according to claim 1, characterized in that, Step 7 includes Step 71: Calculate the number of floating-point operations for the student network using the following formula. : (39)； in, Represents the number of model layers. Representing the Number of input channels of the layer Representing the Number of output channels of the layer Representing the kernel size of the layer and Representing the first The layer outputs the height and width of the feature map; Step 72, calculate the student network parameters using the following formula. : (40)； in, Represents the number of model layers. Representing the Number of input channels of the layer Representing the Number of output channels of the layer Representing the Layer convolution kernel size; Step 73, calculate the student's online reasoning time using the following formula. : (41)； in, It is the total number of reasoning attempts. It is the first The time required for each reasoning step.

8. A change detection apparatus guided by edge feature knowledge distillation, used to perform the change detection method guided by edge feature knowledge distillation as described in any one of claims 1-7, characterized in that, include: The dataset acquisition module is used to select a change detection dataset containing before-and-after temporal images and label datasets, and to preprocess the before-and-after temporal images; The feature extraction module is used to extract multi-scale hierarchical features from the preprocessed change detection dataset; The edge extraction module is used to extract change edge-aware features from the preprocessed change detection dataset; The hierarchical edge knowledge distillation module is used to obtain trained hierarchical edge perception features from the teacher's network edge perception module and supervise the edge features of the student's network. The change feature decoding module integrates distilled edge-aware knowledge and hierarchical change features to obtain change features guided by edge knowledge distillation. The network training module trains the network based on hierarchical edge distillation loss and hybrid loss functions, and outputs the results of change region detection.

9. A computer device, characterized in that, The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of the edge feature knowledge distillation-guided change detection method as described in any one of claims 1-7.

10. A storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, performs the steps of the edge feature knowledge distillation-guided change detection method as described in any one of claims 1-7.

Images