Multi-dimensional information interaction and small target enhancement change detection method and device

Abstract

The invention discloses a change detection method and device for multi-dimensional information interaction and small target enhancement, and belongs to the technical field of remote sensing image processing, and the method comprises the steps: selecting a change detection data set with front and back time phase images and label data from a remote sensing image data set, and carrying out the preprocessing of the front and back time phase images in the change detection data set; screening small targets of which the connected domain area is smaller than 32 * 32 pixels from the labels in the change detection data set, and calculating small target enhancement factors; extracting double-time-phase layering features from the preprocessed front and back time-phase images; performing time-space-channel three-dimensional information interaction on the extracted dual-time-phase hierarchical features, and outputting hierarchical change features fusing dual-time-phase information; fusing the layered change features to obtain fused change features; and performing network training on the fused change features by using the change detection loss of small target enhancement, and outputting a change detection result. According to the invention, the small target detection rate and the change detection accuracy are improved.

Description

Technical Field

[0001] The invention relates to a method and a device for detecting changes in multi-dimensional information interaction and small target enhancement, and belongs to the technical field of remote sensing image processing. Background Art

[0002] In recent years, as an effective means of remote sensing image feature mining, deep learning change detection methods have the significant advantages of automated feature extraction and high detection efficiency. They can complete the change detection of a large area in a short time, and have very broad research and application prospects in the field of remote sensing change detection. However, the current change detection network often takes overall accuracy as the optimization goal. Therefore, when the scales of the changed objects vary greatly, it will be difficult for the model optimization process to take into account objects of different scales. In addition, multiple pooling on feature maps will also lead to serious loss of feature-level information in the local space, which is particularly unfavorable for detecting small change targets with fewer features. Therefore, information interaction and information enhancement between different dimensions are crucial to the final change detection results, especially the change detection results of small targets.

[0003] Based on this understanding, researchers have taken different measures to enhance the change features, such as: introducing densely connected UNet++ structures into change detection networks to retain high-resolution, fine-grained feature representations; using dense skip connections and cross-attention mechanisms to effectively propagate contextual information, accurately capture and fuse global spatial features; designing multi-scale, multi-level dense reconstruction modules to enhance the network's robustness to pseudo-changes. In recent years, attention-based feature enhancement methods have been widely used in change detection tasks. The attention mechanism can guide the network to focus on the change area, and it has fewer parameters than dense connections that indiscriminately focus on all areas. Attention mechanisms include temporal attention, spatial attention, channel attention, self-attention, etc. The latest research shows that a change detection network that combines two or more attentions can measure the differences between dual-phase images from multiple perspectives such as time, space, and channels, thereby more effectively extracting deep change features of dual-phase images and avoiding redundant calculations.

[0004] However, most existing methods only use the attention modules in a superimposed manner without conducting in-depth exploration of the relationship between modules, resulting in weak synergy between modules, and the lack of effective interaction of spatiotemporal information, which makes it impossible to achieve effective feature enhancement. Therefore, the present invention proposes a change detection method and device for multi-dimensional information interaction and small target enhancement. Summary of the invention

[0005] In order to solve the problem of weak coordination between the above-mentioned attention modules, the present invention proposes a change detection method and device for multi-dimensional information interaction and small target enhancement, which can realize information interaction of dual-phase features in the time-space-channel dimension, and at the same time enhance the network's ability to extract small targets.

[0006] The technical solution adopted by the present invention to solve the technical problem is:

[0007] In a first aspect, an embodiment of the present invention provides a change detection method for multi-dimensional information interaction and small target enhancement, comprising the following steps:

[0008] Step S1, selecting a change detection dataset having before and after phase images and label data from a remote sensing image dataset, and preprocessing the before and after phase images in the change detection dataset;

[0009] Step S2, filtering small targets with a connected domain area less than 32×32 pixels from the labels in the change detection dataset, and calculating the small target enhancement factor;

[0010] Step S3, extracting dual-phase hierarchical features from the preprocessed front and back phase images;

[0011] Step S4, performing time-space-channel three-dimensional information interaction on the extracted dual-phase hierarchical features, and outputting hierarchical change features that fuse the dual-phase information;

[0012] Step S5, fusing the layered change features to obtain fused change features;

[0013] Step S6, using the small target enhanced change detection loss, performs network training on the fused change features and outputs the change detection results.

[0014] As a possible implementation of this embodiment, the preprocessing of the before and after phase images in the change detection data set includes:

[0015] One or more data enhancement methods including random addition of fog effect, random addition of noise, random blur, random grayscale and contrast-limited adaptive histogram equalization are used to enhance the before and after phase image data.

[0016] As a possible implementation of this embodiment, the step S2, screening small targets with a connected domain area less than 32×32 pixels from the labels in the change detection data set, and calculating the small target enhancement factor, includes:

[0017] Step S21, using the connected domain algorithm to obtain all targets in the tag:

[0018] i target ,area i=Findconnected(image) (1)

[0019] Among them, i target ,area i They represent the serial number of the target in the image and the area of ​​the target respectively;

[0020] Step S22, filtering small targets from all acquired targets:

[0021]

[0022] Among them, j,area j Represents the serial number of the small target in the image and the area of ​​the target, k,area k Represents the serial number and area of ​​other targets;

[0023] Step S23, calculate the small target enhancement factor:

[0024]

[0025] Among them, area_threshold represents the small target threshold, that is, 32×32 pixels, ε is a minimum value to ensure that the denominator is not 0, γ is the degree of the power function, which is a constant less than 1, representing the degree of enhancement of small targets;

[0026] Step S24, setting the weights of other objects in the image to 1.

[0027] As a possible implementation of this embodiment, the step S3 of extracting the dual-phase hierarchical features from the preprocessed front and back phase images includes:

[0028] The H×W×3-sized anterior and posterior phase images are input into the residual network Resnet18. The residual network is used to extract multi-scale hierarchical features from the dual-phase images. After passing through the four basic blocks, the output resolutions are The hierarchical features and hierarchical features Among them, i represents the i-th layer, C i =2 i+5 ,i=1,2,3,4.

[0029] As a possible implementation of this embodiment, step S4, performing time-space-channel three-dimensional information interaction on the extracted dual-phase hierarchical features, and outputting hierarchical change features integrating the dual-phase information, includes:

[0030] Step S41, the extracted hierarchical features are connected by a connection operation and hierarchical features To perform the fusion:

[0031]

[0032] Among them, i represents the i-th layer, F i It represents the fused dual-phase hierarchical features, and Concatenate represents feature concatenation;

[0033] Step S42, the fused dual-phase hierarchical features are grouped sequentially in the channel dimension, each sub-feature is input into two parallel branches, and average pooling is performed in the horizontal and vertical directions respectively. Then, the features obtained by the two branches are spliced ​​together in the height dimension, and then 1×1 convolution and sigmoid operations are performed to divide them into wide-dimensional features and high-dimensional features, and the wide-dimensional features and high-dimensional features are multiplied with the original sub-features respectively to obtain the inter-channel information interaction features:

[0034]

[0035] Among them, xAvgpool and yAvgpool represent average pooling in the horizontal and vertical directions respectively, Concatenate represents feature concatenation, Conv1×1 represents 1×1 convolution, and sigmoid represents a nonlinear activation function. It is the sub-feature in two directions after aggregation. SplitW and SplitH represent the division operations along the width and height dimensions respectively. It is the final information interaction feature between channels;

[0036] Step S43, using parallel 3×3 branches to capture the spatial dependency between pixels to retain the feature space dimension information and obtain the spatial dimension feature

[0037]

[0038] Step S44: Inter-channel information interaction features and spatial dimension features Both softmax and Avgpool operations are performed, and matrix cross multiplication and addition are performed, and then multiplied with the original sub-features and sigmoid operations are performed to obtain the final sub-features.

[0039]

[0040] Step S45: sub-features Aggregate to obtain the final hierarchical features

[0041]

[0042] in, is the final hierarchical feature, and g is the number of feature groups.

[0043] As a possible implementation of this embodiment, the step S5 of fusing the layered change features to obtain the fused change features includes:

[0044] Step S51: hierarchical features and hierarchical features Upsample to the same size as the original image

[0045]

[0046] Step S52, connect the upsampled bi-temporal features of the same layer:

[0047]

[0048] Step S53: fuse the features of each layer to obtain the change feature F change :

[0049]

[0050] in, There are four layers of hierarchical change features F change .

[0051] As a possible implementation of this embodiment, step S6, using the small target enhanced change detection loss, performs network training on the fused change features, and outputs the change detection result, includes:

[0052] Step S61, calculate the change detection loss:

[0053]

[0054] Among them, y and y′ represent the change detection prediction value and label value respectively, α and λ are the weight parameters specified in Focal loss, and L Focal is the focal loss, L dice For Dice loss;

[0055] Step S62, calculating the change detection loss L for small target enhancement change_sob :

[0056] L change_sob =τ×ψ1L 2ocal +ψ c L dice (18)

[0057] Among them, τ represents the value of the small target enhancement factor, ψ1 and ψ2 represent the proportion of Focal loss and Dice loss respectively;

[0058] Step S63, using the change detection loss L change_sob The network is trained on the fused change features and the change detection results are output.

[0059] In a second aspect, an embodiment of the present invention provides a change detection device for multi-dimensional information interaction and small target enhancement, comprising:

[0060] A data acquisition module is used to select a change detection data set having before and after phase images and label data from a remote sensing image data set, and to pre-process the before and after phase images in the change detection data set;

[0061] Small target screening module, used to screen small targets with a connected domain area less than 32×32 pixels from the labels in the change detection dataset and calculate the small target enhancement factor;

[0062] A hierarchical feature extraction module is used to extract dual-phase hierarchical features from the preprocessed anterior and posterior phase images;

[0063] An information interaction module is used to perform time-space-channel three-dimensional information interaction on the extracted dual-phase hierarchical features, and output hierarchical change features that integrate the dual-phase information;

[0064] A hierarchical feature fusion module is used to fuse hierarchical change features to obtain fused change features;

[0065] The network training module is used to utilize the change detection loss enhanced by small targets to perform network training on the fused change features and output the change detection results.

[0066] In the third aspect, an embodiment of the present invention provides an electronic device, including a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory through the bus, and the processor executes the machine-readable instructions to perform the steps of any of the above-mentioned multi-dimensional information interaction and small target enhancement change detection methods.

[0067] In a fourth aspect, an embodiment of the present invention provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned methods for multi-dimensional information interaction and small target enhancement change detection are executed.

[0068] The beneficial effects of the technical solution of the embodiment of the present invention are as follows:

[0069] A change detection method for multi-dimensional information interaction and small target enhancement according to the technical solution of an embodiment of the present invention comprises the following steps: step S1, selecting a change detection data set having before and after phase images and label data from a remote sensing image data set, and preprocessing the before and after phase images in the change detection data set; step S2, screening small targets with a connected domain area less than 32×32 pixels from the labels in the change detection data set, and calculating the small target enhancement factor; step S3, extracting dual-phase hierarchical features from the preprocessed before and after phase images; step S4, performing time-space-channel three-dimensional information interaction on the extracted dual-phase hierarchical features, and outputting hierarchical change features that fuse the dual-phase information; step S5, fusing the hierarchical change features to obtain fused change features; step S6, performing network training on the fused change features using the change detection loss for small target enhancement, and outputting the change detection results. The present invention detects small targets by adopting a small target enhancement factor, thereby improving the model's detection rate for small targets and improving the small target detection rate; through the effective fusion of two features in the time-space-channel dimension, the problem of difficult information interaction between dual-phase features is solved, the accuracy of change detection is improved, and satisfactory change detection results are achieved.

[0070] The present invention not only realizes the information interaction of dual-phase features in the time-space-channel dimension, but also enhances the network's ability to extract small targets. The present invention uses a small target enhancement factor for small target detection performance, based on an exponential function, to determine the weighted intensity according to the degree of difference between the small target area and the threshold. The enhancement factor sets an adjustable exponential parameter. Through parameter adjustment, the model can maintain the performance of large-area target detection without decreasing while increasing the focus on small targets.

[0071] A change detection device for multi-dimensional information interaction and small target enhancement according to the technical solution of an embodiment of the present invention has the same beneficial effects as a change detection method for multi-dimensional information interaction and small target enhancement according to the technical solution of an embodiment of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 is a flow chart of a change detection method for multi-dimensional information interaction and small target enhancement according to an exemplary embodiment;

[0073] Figure 2 is a schematic structural diagram of a change detection device for multi-dimensional information interaction and small target enhancement according to an exemplary embodiment;

[0074] Figure 3 It is a specific implementation schematic diagram of the present invention for detecting a change region in a public data set;

[0075] Figure 4is a schematic diagram showing a method of performing time-space-channel three-dimensional information interaction on dual-phase layered features according to an exemplary embodiment;

[0076] Figure 5 It is a partial visualization of the change detection results of the method of the present invention on the LEVIR-CD dataset. DETAILED DESCRIPTION

[0077] In order to more clearly illustrate the technical features of the solution of the present invention, the present invention is described in detail below through specific implementation methods and in conjunction with the accompanying drawings.

[0078] Aiming at the problems of low information interaction between modules and difficulty in extracting small targets in existing change detection models, the present invention proposes a change detection method with multi-dimensional information interaction and small target enhancement.

[0079] like Figure 1 As shown, a change detection method for multi-dimensional information interaction and small target enhancement provided by an embodiment of the present invention includes the following steps:

[0080] Step S1, selecting a change detection dataset having before and after phase images and label data from a remote sensing image dataset, and preprocessing the before and after phase images in the change detection dataset;

[0081] Step S2, filtering small targets with a connected domain area less than 32×32 pixels from the labels in the change detection dataset, and calculating the small target enhancement factor;

[0082] Step S3, extracting dual-phase hierarchical features from the preprocessed front and back phase images;

[0083] Step S4, performing time-space-channel three-dimensional information interaction on the extracted dual-phase hierarchical features, and outputting hierarchical change features that fuse the dual-phase information;

[0084] Step S5, fusing the layered change features to obtain fused change features;

[0085] Step S6, using the small target enhanced change detection loss, performs network training on the fused change features and outputs the change detection results.

[0086] As a possible implementation of this embodiment, the preprocessing of the before and after phase images in the change detection data set includes:

[0087] One or more data enhancement methods including random addition of fog effect, random addition of noise, random blur, random grayscale and contrast-limited adaptive histogram equalization are used to enhance the before and after phase image data.

[0088] As a possible implementation of this embodiment, the step S2, screening small targets with a connected domain area less than 32×32 pixels from the labels in the change detection data set, and calculating the small target enhancement factor, includes:

[0089] Step S21, using the connected domain algorithm to obtain all targets in the tag:

[0090] i target ,area i =Findconnected(image) (1)

[0091] Among them, i target ,area i They represent the serial number of the target in the image and the area of ​​the target respectively;

[0092] Step S22, filtering small targets from all acquired targets:

[0093]

[0094] Among them, j,area j Represents the serial number of the small target in the image and the area of ​​the target, k,area k Represents the serial number and area of ​​other targets;

[0095] Step S23, calculate the small target enhancement factor:

[0096]

[0097] Among them, area_threshold represents the small target threshold, that is, 32×32 pixels, ε is a minimum value to ensure that the denominator is not 0, γ is the degree of the power function, which is a constant less than 1, representing the degree of enhancement of small targets;

[0098] Step S24, setting the weights of other objects in the image to 1.

[0099] As a possible implementation of this embodiment, the step S3 of extracting the dual-phase hierarchical features from the preprocessed front and back phase images includes:

[0100] The H×W×3-sized anterior and posterior phase images are input into the residual network Resnet18. The residual network is used to extract multi-scale hierarchical features from the dual-phase images. i represents the i-th layer feature. After passing through the four basic blocks, the output resolution is The hierarchical features and hierarchical features Among them C i =2 i+5 ,i=1,2,3,4.

[0101] As a possible implementation of this embodiment, step S4, performing time-space-channel three-dimensional information interaction on the extracted dual-phase hierarchical features, and outputting hierarchical change features integrating the dual-phase information, includes:

[0102] Step S41, the extracted hierarchical features are connected by a connection operation and hierarchical features To perform the fusion:

[0103]

[0104] Among them, i represents the i-th layer, F i It represents the fused dual-phase hierarchical features, and Concatenate represents feature concatenation;

[0105] Step S42, the fused dual-phase hierarchical features are grouped sequentially in the channel dimension, each sub-feature is input into two parallel branches, and average pooling is performed in the horizontal and vertical directions respectively. Then, the features obtained by the two branches are spliced ​​together in the height dimension, and then 1×1 convolution and sigmoid operations are performed to divide them into wide-dimensional features and high-dimensional features, and the wide-dimensional features and high-dimensional features are multiplied with the original sub-features respectively to obtain the inter-channel information interaction features:

[0106]

[0107] Among them, xAvgpool and yAvgpool represent average pooling in the horizontal and vertical directions respectively, Concatenate represents feature concatenation, Conv1×1 represents 1×1 convolution, and sigmoid represents a nonlinear activation function. It is the sub-feature in two directions after aggregation. SplitW and SplitH represent the division operations along the width and height dimensions respectively. It is the final information interaction feature between channels;

[0108] Step S43, using parallel 3×3 branches to capture the spatial dependency between pixels to retain the feature space dimension information and obtain the spatial dimension feature

[0109]

[0110] Step S44: Inter-channel information interaction features and spatial dimension features Both softmax and Avgpool operations are performed, and matrix cross multiplication and addition are performed, and then multiplied with the original sub-features and sigmoid operations are performed to obtain the final sub-features.

[0111]

[0112] Step S45: sub-features Aggregate to obtain the final hierarchical features

[0113]

[0114] in, is the final hierarchical feature, and g is the number of feature groups.

[0115] As a possible implementation of this embodiment, the step S5 of fusing the layered change features to obtain the fused change features includes:

[0116] Step S51: hierarchical features and hierarchical features Upsample to the same size as the original image

[0117]

[0118] Step S52, connect the upsampled bi-temporal features of the same layer:

[0119]

[0120] Step S53: fuse the features of each layer to obtain the change feature F change :

[0121]

[0122] in, There are four layers of hierarchical change features F change .

[0123] As a possible implementation of this embodiment, step S6, using the small target enhanced change detection loss, performs network training on the fused change features, and outputs the change detection result, includes:

[0124] Step S61, calculate the change detection loss:

[0125]

[0126] Among them, y and y′ represent the change detection prediction value and label value respectively, α and λ are the weight parameters specified in Focal loss, and L Focal is the focal loss, L dice For Dice loss;

[0127] Step S62, calculating the change detection loss L for small target enhancement change_sob :

[0128] L change_sob =τ×ψ1L Focal +ψ2L dice (18)

[0129] Among them, τ represents the value of the small target enhancement factor, ψ1 and ψ2 represent the proportion of Focal loss and Dice loss respectively;

[0130] Step S63, using the change detection loss L change_sob The network is trained on the fused change features and the change detection results are output.

[0131] like Figure 2 As shown, an embodiment of the present invention provides a change detection device for multi-dimensional information interaction and small target enhancement, comprising:

[0132] A data acquisition module is used to select a change detection data set having before and after phase images and label data from a remote sensing image data set, and to pre-process the before and after phase images in the change detection data set;

[0133] Small target screening module, used to screen small targets with a connected domain area less than 32×32 pixels from the labels in the change detection dataset and calculate the small target enhancement factor;

[0134] A hierarchical feature extraction module is used to extract dual-phase hierarchical features from the preprocessed anterior and posterior phase images;

[0135] An information interaction module is used to perform time-space-channel three-dimensional information interaction on the extracted dual-phase hierarchical features, and output hierarchical change features that integrate the dual-phase information;

[0136] A hierarchical feature fusion module is used to fuse hierarchical change features to obtain fused change features;

[0137] The network training module is used to utilize the change detection loss enhanced by small targets to perform network training on the fused change features and output the change detection results.

[0138] Figure 3 is a schematic diagram of a specific implementation of the present invention for detecting a change region in a public data set, Figure 3 In the figure, batchsize is the batch size, Group Norm is group normalization, Softmax is the normalized exponential function, avg pool is the average pooling layer, Matmul is the matrix multiplication, SplitH+Sigmoid and SplitW+Sigmoid represent nonlinear activation after splitting operations along the width and height dimensions respectively, X Avg Pool and Y Avg Pool represent average pooling along the horizontal and vertical directions respectively, For connection, is the dot product and ⊕ is the element-wise addition.

[0139] like Figure 3 As shown, the specific process of change detection based on multi-dimensional information interaction and small target enhancement of the present invention is as follows.

[0140] Step 1: Select a change detection dataset with before and after phase images and a label dataset, and pre-process the before and after phase images.

[0141] The label data set is a public data set CDD and / or LEVIR-CD; the preprocessing of the change detection data set is to enhance the change detection data set by using a data enhancement method of randomly adding fog effects, randomly adding noise, randomly blurring, randomly graying, and contrast-limited adaptive histogram equalization.

[0142] Step S2, processing the changed labels, screening small targets with connected domain areas less than 32×32 pixels in the labels, and calculating the small target enhancement factor.

[0143] Step S21, obtain all the targets in the tag, this step is implemented using the connected domain algorithm:

[0144] i target ,area i =Findconnected(image) (1)

[0145] In the formula, i target ,area i They represent the serial number of the target in the image and the area of ​​the target respectively;

[0146] Step S22, filter the small targets:

[0147]

[0148] Among them, j,area j Represents the serial number of the small target in the image and the area of ​​the target, k,area k Represents the serial number and area of ​​other targets;

[0149] Step S23, calculate the small target enhancement factor according to the following formula:

[0150]

[0151] In the formula, area_threshold is the threshold for determining small targets, that is, 32×32 pixels, area j is the area of ​​the small target, and ε is a very small constant to ensure that the denominator is not 0 (we take 1×10 -5), γ is the power of the function, which is a constant less than 1, representing the degree of enhancement of small targets. In order to ensure that the weight of large targets is not weakened too much, we usually take a smaller γ. For example, when γ = 0.1, the value range of Factor is (1, 2); then, the calculated enhancement factor Factor is used j Weight the loss values ​​of all pixels within the small target j;

[0152] Step S24, setting the weights of the remaining parts of the image to 1.

[0153] Step 3: Extract multi-scale hierarchical features from the preprocessed change detection dataset.

[0154] The residual network is used to extract multi-scale hierarchical features from the dual-phase images of the CDD or LEVIR-CD dataset. Specifically, the anterior and posterior phase images of the three-dimensional matrix of size H×W×3 are input into the residual network Resnet18, and after passing through four basic blocks, the resolutions are respectively output. The hierarchical features Among them, i represents the i-th layer, C i =2 i+5 , i = (1, 2, 3, 4), H is the height of the before and after phase images, W is the width of the before and after phase images, C i is the number of channels of the before and after phase images.

[0155] Step S4, the extracted dual-phase hierarchical features are input into the multi-dimensional information interactive fusion module, such as Figure 4 As shown, information interaction in the time-space-channel dimensions is performed, and hierarchical change features that fuse dual-phase information are output.

[0156] Step S41: Fusion of bi-temporal hierarchical features through concatenation

[0157]

[0158] Among them, i represents the i-th layer, and the function Concatenate represents feature concatenation;

[0159] Step S42, the features obtained in step S41 are grouped sequentially in the channel dimension, and then each sub-feature is input into two parallel branches, and average pooling is performed in the horizontal and vertical directions respectively, so as to perform feature interaction in the channel dimension in different directions and capture the inter-pixel dependency in the channel dimension. After that, the features obtained by the two branches are concatenated in the height dimension, and then 1×1 convolution and sigmoid operations are performed, and finally they are divided into wide and high dimensional features, and the two features are multiplied with the original sub-features to obtain the inter-channel information interaction features:

[0160]

[0161]

[0162] xAvgpool and yAvgpool represent average pooling in the horizontal and vertical directions, Concatenate represents feature concatenation along the height dimension, Conv1×1 represents 1×1 convolution, and sigmoid represents a nonlinear activation function; It is the sub-feature in two directions after aggregation. SplitW and SplitH represent the division operations along the width and height dimensions respectively. It is the final information interaction feature between channels;

[0163] Step S43, using parallel 3×3 branches to capture the spatial dependency between pixels to preserve the feature space dimension information:

[0164]

[0165] In step S44, we interact the channel dimension and the spatial dimension information. Specifically, we perform softmax and Avgpool operations on the features obtained in the first two steps, and then perform matrix cross-multiplication on the two parts of the features and add them together. After multiplying them with the original sub-features, we perform a sigmoid operation to obtain the final sub-features. The specific expressions are as follows:

[0166]

[0167] Step S45, aggregate and group sub-features:

[0168]

[0169] is the final hierarchical feature, g is the number of feature groups;

[0170] Through the above operations, in a form similar to the attention mechanism, feature grouping and cross-space learning strategies are adopted to perform time-channel-space dimension information interaction of hierarchical features within the module, thereby achieving enhanced expression of changing features.

[0171] Step S5: In the change feature decoder, the change features obtained by the encoder are fused.

[0172] Step S51, upsampling the bi-phase hierarchical features by 2, 4, 8, and 16 times respectively. This process is expressed as:

[0173]

[0174] Step S52, connect the upsampled bi-temporal features of the same layer:

[0175]

[0176] Step S53: fuse the features of each layer to obtain the change feature F change :

[0177]

[0178] Step S6, using the small target enhanced change detection loss to perform network training and output the change detection result.

[0179] Step S61, calculate the change detection loss, which is composed of Focal loss and Dice loss, expressed as:

[0180]

[0181] Among them, y and y′ represent the change detection prediction value and label value respectively, α and λ are the weight parameters specified in Focal loss, and L Focal is the focal loss, L dice For Dice loss;

[0182] Step S62, calculate the change detection loss of small target enhancement:

[0183] L change_sob =τ×ψ1L Focal +ψ2L dice (18) τ represents the value of the small target enhancement factor, ψ1 and ψ2 represent the proportion of Focal loss and Dice loss respectively.

[0184] The experimental results on the LEVIR-CD public dataset show that: from the perspective of the F1 index under the pixel-level accuracy evaluation system, compared with the existing classic and latest change detection methods, the change detection accuracy of the proposed method is higher, and it shows better performance in the change detection task. Table 1 shows the experimental results of the proposed method and the existing change detection methods on LEVIR-CD, and all values ​​are presented in percentage (%).

[0185] Table 1: Comparative experimental results on LEVIR-CD

[0186]

[0187] Table 1 MapsNet * The method detection results are the results of reproducing the network under the Pytorch framework.

[0188] The change detection results obtained by the present invention are visualized, such as Figure 5 shown. Figure 5 In the figure, Figure (a) is the image of the previous phase, Figure (b) is the image of the later phase, Figure (c) is the reference image, Figure (d) is the detection result of the FC_Siam_diff method, Figure (e) is the detection result of the ChangeFormer method, Figure (f) is the detection result of the MapsNet* method, Figure (g) is the detection result of the BIT-CD method, Figure (h) is the detection result of the AMT_Net method, Figure (i) is the detection result of the USSFC method, Figure (j) is the detection result of the STADE-CDNet method, and Figure (k) is the detection result of the method of the present invention. It can be seen from the above results that the present invention achieves the purpose of accurately outputting change detection results and improving the detection rate of small targets, and obtains satisfactory change detection results.

[0189] An embodiment of the present invention provides an electronic device, including a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory through the bus, and the processor executes the machine-readable instructions to perform steps of any of the above-mentioned multi-dimensional information interaction and small target enhancement change detection methods.

[0190] Specifically, the above-mentioned memory and processor can be general-purpose memory and processor, which are not specifically limited here. When the processor runs the computer program stored in the memory, it can execute the above-mentioned multi-dimensional information interaction and small target enhanced change detection method.

[0191] Those skilled in the art will appreciate that the structure of the computer device does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently.

[0192] In some embodiments, the computer device may also include a touch screen that can be used to display a graphical user interface (e.g., a startup interface of an application) and receive user operations on the graphical user interface (e.g., startup operations on an application). The specific touch screen may include a display panel and a touch panel. The display panel may be configured in the form of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc. The touch panel may collect the user's contact or non-contact operations on or near it, and generate pre-set operation instructions, for example, the user uses any suitable object such as a finger, stylus, or accessories on or near the touch panel. In addition, the touch panel may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation and posture, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into information that the processor can process, and then sends it to the processor, and can receive and execute commands from the processor. In addition, the touch panel can be implemented by various types such as resistive, capacitive, infrared and surface acoustic wave, and any technology developed in the future can also be used to implement the touch panel. Further, the touch panel can cover the display panel, and the user can operate on or near the touch panel covered on the display panel according to the graphical user interface displayed on the display panel. After the touch panel detects the operation on or near it, it is transmitted to the processor to determine the user input, and then the processor provides corresponding visual output on the display panel in response to the user input. In addition, the touch panel and the display panel can be implemented as two independent components or integrated.

[0193] Corresponding to the method for starting the above-mentioned application, an embodiment of the present invention also provides a storage medium, on which a computer program is stored. When the computer program is run by a processor, the steps of any of the above-mentioned multi-dimensional information interaction and small target enhancement change detection methods are executed.

[0194] The application startup device provided in the embodiment of the present application can be specific hardware on the device or software or firmware installed on the device. The device provided in the embodiment of the present application, its implementation principle and the technical effect produced are the same as those in the aforementioned method embodiment. For the sake of brief description, the parts not mentioned in the device embodiment can refer to the corresponding contents in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0195] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0196] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0197] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0198] In addition, each functional module in the embodiments provided in the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0199] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0200] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0201] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0202] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A change detection method with multi-dimensional information interaction and small target enhancement, characterized in that: The steps include: Step S1, selecting a change detection dataset having before and after phase images and label data from a remote sensing image dataset, and preprocessing the before and after phase images in the change detection dataset; Step S2, filtering small targets with a connected domain area less than 32×32 pixels from the labels in the change detection dataset, and calculating the small target enhancement factor; Step S3, extracting dual-phase hierarchical features from the preprocessed front and back phase images; Step S4, performing time-space-channel three-dimensional information interaction on the extracted dual-phase hierarchical features, and outputting hierarchical change features that fuse the dual-phase information; Step S5, fusing the layered change features to obtain fused change features; Step S6, using the small target enhanced change detection loss, performs network training on the fused change features and outputs the change detection results.

2. The change detection method for multi-dimensional information interaction and small target enhancement according to claim 1 is characterized in that: The step S2, screening small targets with a connected domain area less than 32×32 pixels from the labels in the change detection data set, and calculating the small target enhancement factor, includes: Step S21, using the connected domain algorithm to obtain all targets in the tag: i target ,area i =Findconnected(image) (1) Among them, i target ,area i They represent the serial number of the target in the image and the area of ​​the target respectively; Step S22, filtering small targets from all acquired targets: Among them, j,area j Represents the serial number of the small target in the image and the area of ​​the target, k,area k Represents the serial number and area of ​​other targets; Step S23, calculate the small target enhancement factor: Among them, area_threshold represents the small target threshold, that is, 32×32 pixels, ε is a minimum value to ensure that the denominator is not 0, and γ is the degree of the power function; Step S24, setting the weights of other objects in the image to 1.

3. The change detection method for multi-dimensional information interaction and small target enhancement according to claim 2 is characterized in that: The step S3, extracting the dual-phase hierarchical features from the pre-processed front and back phase images, comprises: The H×W×3-sized anterior and posterior phase images are input into the residual network Resnet18. The residual network is used to extract multi-scale hierarchical features from the dual-phase images. After passing through the four basic blocks, the output resolutions are The hierarchical features and hierarchical features Among them, i represents the i-th layer, C i =2 i+5 ,i=1,2,3,4.

4. The change detection method for multi-dimensional information interaction and small target enhancement according to claim 3 is characterized in that: The step S4 is to perform time-space-channel three-dimensional information interaction on the extracted dual-phase hierarchical features, and output hierarchical change features that fuse the dual-phase information, including: Step S41, the extracted hierarchical features are connected by a connection operation and hierarchical features To perform the fusion: Among them, i represents the i-th layer, F i It represents the fused dual-phase hierarchical features, and Concatenate represents feature concatenation; Step S42, the fused dual-phase hierarchical features are grouped sequentially in the channel dimension, each sub-feature is input into two parallel branches, and average pooling is performed in the horizontal and vertical directions respectively. Then, the features obtained by the two branches are spliced ​​together in the height dimension, and then 1×1 convolution and sigmoid operations are performed to divide them into wide-dimensional features and high-dimensional features, and the wide-dimensional features and high-dimensional features are multiplied with the original sub-features respectively to obtain the inter-channel information interaction features: Among them, xAvgpool and yAvgpool represent average pooling in the horizontal and vertical directions respectively, Concatenate represents feature concatenation, Conv1×1 represents 1×1 convolution, and sigmoid represents a nonlinear activation function. It is the sub-feature in two directions after aggregation. SplitW and SplitH represent the division operations along the width and height dimensions respectively. It is the final information interaction feature between channels; Step S43, using parallel 3×3 branches to capture the spatial dependency between pixels to retain the feature space dimension information and obtain the spatial dimension feature Step S44: Inter-channel information interaction features and spatial dimension features Both softmax and Avgpool operations are performed, and matrix cross multiplication and addition are performed, and then multiplied with the original sub-features and sigmoid operations are performed to obtain the final sub-features Step S45: sub-features Aggregate to obtain the final hierarchical features in, is the final hierarchical feature, and g is the number of feature groups.

5. The change detection method for multi-dimensional information interaction and small target enhancement according to claim 4 is characterized in that: The step S5, fusing the layered change features to obtain fused change features, includes: Step S51: hierarchical features and hierarchical features Upsample to the same size as the original image Step S52, connect the upsampled bi-temporal features of the same layer: Step S53: fuse the features of each layer to obtain the change feature F change : in, There are four layers of hierarchical change features F change .

6. The change detection method for multi-dimensional information interaction and small target enhancement according to claim 5, characterized in that: The step S6, using the small target enhanced change detection loss, performs network training on the fused change features, and outputs the change detection results, including: Step S61, calculate the change detection loss: Among them, y and y′ represent the change detection prediction value and label value respectively, α and λ are the weight parameters specified in Focal loss, and L Focal is the focal loss, L dice For Dice loss; Step S62, calculating the change detection loss L for small target enhancement change_sob : L change_sob =τ×ψ1L Focal +ψ2L dice (18) Among them, τ represents the value of the small target enhancement factor, ψ1 and ψ2 represent the proportion of Focal loss and Dice loss respectively; Step S63, using the change detection loss L change_sob The network is trained on the fused change features and the change detection results are output.

7. The change detection method for multi-dimensional information interaction and small target enhancement according to any one of claims 1 to 6, characterized in that: The preprocessing of the before and after phase images in the change detection data set includes: One or more data enhancement methods including random addition of fog effect, random addition of noise, random blur, random grayscale and contrast-limited adaptive histogram equalization are used to enhance the before and after phase image data.

8. A change detection device for multi-dimensional information interaction and small target enhancement, characterized in that: include: A data acquisition module is used to select a change detection data set having before and after phase images and label data from a remote sensing image data set, and to pre-process the before and after phase images in the change detection data set; Small target screening module, used to screen small targets with a connected domain area less than 32×32 pixels from the labels in the change detection dataset and calculate the small target enhancement factor; A hierarchical feature extraction module is used to extract dual-phase hierarchical features from the preprocessed anterior and posterior phase images; An information interaction module is used to perform time-space-channel three-dimensional information interaction on the extracted dual-phase hierarchical features, and output hierarchical change features that integrate the dual-phase information; A hierarchical feature fusion module is used to fuse hierarchical change features to obtain fused change features; The network training module is used to utilize the change detection loss enhanced by small targets to perform network training on the fused change features and output the change detection results.

9. An electronic device, characterized in that: It includes a processor, a memory and a bus, the memory stores machine-readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, and the processor executes the machine-readable instructions to perform the steps of the change detection method for multi-dimensional information interaction and small target enhancement as described in any one of claims 1-7.

10. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, executes the steps of the change detection method for multi-dimensional information interaction and small target enhancement as described in any one of claims 1 to 7.

Images