A change detection method based on small area feature information enhancement

Abstract

The present invention discloses a change detection method based on small area feature information enhancement, and relates to the field of information perception and recognition technology. The present invention first uses the LEVIR‑CD data set as a training set, performs feature fusion on input image one and input image two in the training data, uses a two-dimensional Gaussian to obtain a corresponding small area feature map for the input label map, hierarchically decodes the fused features, and fuses them with the small area feature map, then converts the fusion result into a binary prediction map according to a threshold, and calculates the total loss value in combination with the input label map, performs back propagation training according to the total loss value, and finally processes image one and image two with a time span in the same scene according to the trained model to obtain a corresponding change detection binary map. The present invention can fuse small area feature information, enhance small area feature information, and improve change detection performance.

Description

Technical Field

[0001] The present invention relates to the technical field of information perception and recognition, and in particular to a change detection method based on small area feature information enhancement. Background Art

[0002] Change detection usually compares a pair of remote sensing images, which is of great value in studying changes in the environment and natural resources of the earth, changes in land use, natural disaster assessment, urban evolution, etc. With the continuous development of various aerospace and satellite remote sensing technologies, remote sensing data of different resolutions are constantly accumulated, providing strong support for deep learning technology.

[0003] In recent years, with the rapid development of the Internet and artificial intelligence software and hardware, the impact of deep learning technology on change detection performance has been increasing. Many research institutions have launched their own change detection platforms. Due to the ability of deep networks to effectively extract high-order features, the problems of large errors and low detection efficiency in traditional change detection technology have been effectively improved, and at the same time, it has provided the possibility for industrial automation and process standardization.

[0004] At present, change detection is one of the important research directions in the field of computer vision. There are three main existing methods: supervised, semi-supervised, and unsupervised. In terms of performance, supervised methods currently have the best performance. However, they also have the problem that the deep network feature layer is small in scale and cannot represent the feature information of small areas. This will cause the entire change detection model to tend to focus on large areas with obvious feature information, and small areas will be misdetected or missed. Summary of the invention

[0005] In view of this, the present invention proposes a change detection method based on small area feature information enhancement. The method can fuse small area feature information, enhance small area feature information, and improve change detection performance.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] A change detection method based on small area feature information enhancement includes the following steps:

[0008] Step 1, using the LEVIR-CD dataset as a training set, each training data in the training set includes an input image 1 and an input image 2 with a time span in the same scene, and a corresponding input label map, the input image 1 and the input image 2 are 3-channel RGB images, the input label map is a single-channel binary image, the black area in the input label map represents the unchanged area between the input image 1 and the input image 2, and the white area represents the changed area between the input image 1 and the input image 2;

[0009] Step 2, inputting the input image 1 and the input image 2 in the training data into a high-resolution network MobileNet respectively, for extracting image features, wherein the high-resolution network MobileNet has a four-layer structure, wherein the first layer corresponds to the highest resolution, and the resolutions of the second, third, and fourth layers decrease in sequence, and each layer of the two high-resolution networks MobileNet outputs image features to the same FPN feature pyramid for fusing feature information of different scales, wherein the FPN feature pyramid has a four-layer structure, wherein the first layer corresponds to the highest resolution, and the resolutions of the second, third, and fourth layers decrease in sequence; the outputs of the first to fourth layers of the high-resolution network MobileNet respectively correspond to the inputs of the first to fourth layers of the FPN feature pyramid, and the outputs of the first to fourth layers of the FPN feature pyramid are recorded as d1, d2, d3, and d4, respectively;

[0010] Among them, the first to fourth layers of the high-resolution network MobileNet and the FPN feature pyramid downsample the input image by 1 / 4, 1 / 8, 1 / 16, and 1 / 32 respectively;

[0011] Step 3, set the area threshold. For each white area in the input label map, obtain the connected domain parameters according to the connectedComponentsWithSt-ats() function of OpenCV, select the white area whose area is smaller than the area threshold, and calculate the coordinates of the corresponding center point. The key parameter connectivity is set to 8.

[0012] According to the two-dimensional Gaussian function, the feature information corresponding to each selected white area is calculated in turn, and the small area feature map corresponding to the input label map is obtained according to all the feature information:

[0013]

[0014] Among them, (x0, y0) is the coordinate of the center point of a selected white area, (x, y) is the coordinate variable in the white area corresponding to (x0, y0), Represents the feature information corresponding to the white area with (x0, y0) as the center coordinate, λ is the two-dimensional Gaussian coefficient, and σ is the area width control coefficient;

[0015] Step 4, constructing a fusion feature layer, wherein the fusion feature layer includes the first fusion feature layer to the fourth fusion feature layer, and the four fusion feature layers all use a supervisory attention module;

[0016] After d4 is fused with the small area feature map, it is used as the input value of the first fusion feature layer. The input value of the first fusion feature layer is first convolved, and then the BatchNorm2d operation is performed to normalize the features, and then sigmoid activation is performed. The sigmoid activation value is randomly extracted through the Dropout operation to prevent overfitting. The Dropout result is negated and fused with the input value of the first fusion feature layer. The fusion result is fused with the small area feature map again as MASK1 of the first fusion feature layer;

[0017] MASK1 of the first fusion feature layer is bilinearly interpolated and then fused with d3. The fusion result is fused with the small area feature map again, and the result is used as the input value of the second fusion feature layer. The input value of the second fusion feature layer is first convolved, and then the BatchNorm2d operation is performed to normalize the features, and then sigmoid activation is performed. The sigmoid activation value is randomly extracted through the Dropout operation to prevent overfitting. The Dropout result is negated and fused with the input value of the second fusion feature layer. The fusion result is fused with the small area feature map again as MASK2 of the second fusion feature layer;

[0018] MASK2 of the second fusion feature layer is bilinearly interpolated and then fused with d2. The fusion result is fused with the small area feature map again, and the result is used as the input value of the third fusion feature layer. The input value of the third fusion feature layer is first convolved, and then the BatchNorm2d operation is performed to normalize the features, and then sigmoid activation is performed. The sigmoid activation value is randomly extracted through the Dropout operation to prevent overfitting. The Dropout result is negated and fused with the input value of the third fusion feature layer. The fusion result is fused with the small area feature map again as MASK3 of the third fusion feature layer;

[0019] MASK3 of the third fusion feature layer is bilinearly interpolated and then fused with d1. The fusion result is fused with the small area feature map again, and the result is used as the input value of the fourth fusion feature layer. The input value of the fourth fusion feature layer is first convolved, and then the BatchNorm2d operation is performed to normalize the features, and then sigmoid activation is performed. The sigmoid activation value is randomly extracted through the Dropout operation to prevent overfitting. The Dropout result is negated and fused with the input value of the fourth fusion feature layer. The fusion result is fused with the small area feature map again as MASK4 of the fourth fusion feature layer;

[0020] Step 5: Upsample the MASK1, MASK2, MASK3, and MASK4 of the first to fourth fusion feature layers by 32 times, 16 times, 8 times, and 4 times, respectively, and convert them into prediction images of the size corresponding to the input label image. Set the binary classification threshold thres, and convert the prediction image into a binary prediction image according to the binary classification threshold thres:

[0021]

[0022] Among them, p n (i, j) represents the predicted value of the pixel with coordinates (i, j) in the prediction map corresponding to the nth fusion feature layer, n = 1, 2, 3, 4, F n (i, j) represents the gray value of the pixel with coordinates (i, j) in the binary prediction image corresponding to the nth fusion feature layer;

[0023] Set the initial learning rate, calculate BCELoss and DiceLoss for the binary prediction images corresponding to the first to fourth fusion feature layers and the input label images, and sum the BCELoss and DiceLoss calculation results corresponding to the nth fusion feature layer. The sum is recorded as loss n , and then calculate the total loss value Loss corresponding to the first fusion feature layer to the fourth fusion feature layer:

[0024] Loss=loss1+loss2+loss3+loss4;

[0025] Then, the high-resolution network MobileNet, FPN feature pyramid and fusion feature layer are back-propagated and trained by the gradient descent method until the total loss value Loss converges or the maximum number of iterations is reached. The training is terminated and the current high-resolution network MobileNet, FPN feature pyramid and fusion feature layer models are saved.

[0026] Step 6: Obtain image 1 and image 2 with a time span in the same scene, input them into the trained high-resolution network MobileNet and FPN feature pyramid, and obtain the outputs corresponding to the first to fourth layers of the FPN feature pyramid, which are recorded as d 1 ,d 2 ,d 3 With d 4 ;

[0027] D 4As the input value of the trained first fusion feature layer, the input value of the first fusion feature layer is first convolved, then the BatchNorm2d operation is performed to normalize the features, and then sigmoid activation is performed. The sigmoid activation value is inverted and fused with the input value of the first fusion feature layer to obtain the MASK of the first fusion feature layer. 1 ;

[0028] The MASK of the first fusion feature layer 1 After bilinear interpolation and d 3 The fusion result is used as the input value of the trained second fusion feature layer. The input value of the second fusion feature layer is first convolved, and then the BatchNorm2d operation is performed to normalize the features. Then, sigmoid activation is performed. The sigmoid activation value is inverted and fused with the input value of the second fusion feature layer to obtain the MASK of the second fusion feature layer. 2 ;

[0029] The MASK of the second fusion feature layer 2 After bilinear interpolation and d 2 The fusion result is used as the input value of the trained third fusion feature layer. The input value of the third fusion feature layer is first convolved, and then the BatchNorm2d operation is performed to normalize the features. Then, sigmoid activation is performed. The sigmoid activation value is inverted and fused with the input value of the third fusion feature layer to obtain the mask of the third fusion feature layer. 3 ;

[0030] The MASK of the third fusion feature layer 3 After bilinear interpolation and d 1 The fusion result is used as the input value of the trained fourth fusion feature layer. The input value of the fourth fusion feature layer is first convolved, and then the BatchNorm2d operation is performed to normalize the features, and then sigmoid activation is performed. The sigmoid activation value is inverted and fused with the input value of the fourth fusion feature layer to obtain the MASK of the fourth fusion feature layer. 4 ;

[0031] Set the MASK of the first to fourth fusion feature layers 1 、MASK 2 、MASK 3 、MASK 4 After upsampling by 32 times, 16 times, 8 times, and 4 times, they are converted into corresponding prediction maps. According to the prediction maps corresponding to the first fusion feature layer to the fourth fusion feature layer, the change detection binary maps corresponding to image one and image two are obtained:

[0032]

[0033] Among them, p n (i,j) represents the predicted value of the pixel with coordinates (i,j) in the prediction map corresponding to the nth fusion feature layer; F(i,j) represents the grayscale value of the pixel with coordinates (i,j) in the change detection binary map; the black area represents the unchanged area between image one and image two, and the white area represents the changed area between image one and image two, thus completing the change detection based on the enhancement of small area feature information.

[0034] Due to the adoption of the above technical solution, the beneficial effects of the present invention compared with the prior art are:

[0035] The network design of the present invention is significantly different from previous network designs. It records the coordinates of the center point of a small area and uses two-dimensional Gaussian to generate feature information of the small area, which can well characterize the feature information of the small area. The two-dimensional Gaussian can highlight the local small area and suppress other areas, thereby improving the problem of being unable to characterize small areas in change detection due to the deepening of the network layer. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 The figure is an overall flow chart of a change detection method based on small area feature information enhancement in an embodiment of the present invention.

[0037] Figure 2 Schematic diagram of the feature encoding stage in an embodiment of the present invention.

[0038] Figure 3 Schematic diagram of the feature decoding stage in an embodiment of the present invention.

[0039] Figure 4 Schematic diagram of small area feature enhancement in an embodiment of the present invention.

[0040] Figure 5 , Figure 6 Schematic diagram of the effect in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0042] A change detection method based on small area feature information enhancement, such as Figure 1 Said, comprising the following steps;

[0043] Step 1, using the LEVIR-CD dataset as a training set, each training data in the training set includes an input image 1 and an input image 2 with a time span in the same scene, and a corresponding input label map, the input image 1 and the input image 2 are 3-channel RGB images, the input label map is a single-channel binary image, the black area in the input label map represents the unchanged area between the input image 1 and the input image 2, and the white area represents the changed area between the input image 1 and the input image 2;

[0044] Step 2, such as Figure 2 As shown, the input image 1 and the input image 2 in the training data are respectively input into a high-resolution network MobileNet for extracting image features. The high-resolution network MobileNet is a four-layer structure, the first layer corresponds to the highest resolution, and the resolutions of the second, third, and fourth layers decrease in sequence. Each layer of the two high-resolution networks MobileNet outputs image features to the same FPN feature pyramid for fusing feature information of different scales. The FPN feature pyramid is a four-layer structure, the first layer corresponds to the highest resolution, and the resolutions of the second, third, and fourth layers decrease in sequence. The outputs of the first to fourth layers of the high-resolution network MobileNet correspond to the inputs of the first to fourth layers of the FPN feature pyramid, respectively. The outputs of the first to fourth layers of the FPN feature pyramid are recorded as d1, d2, d3, and d4, respectively.

[0045] Among them, the first to fourth layers of the high-resolution network MobileNet and the FPN feature pyramid downsample the input image by 1 / 4, 1 / 8, 1 / 16, and 1 / 32 respectively;

[0046] Step 3, such as Figure 4 As shown in the figure, set the area threshold, for each white area in the input label map, obtain the connected domain parameters according to the connectedComponentsWithSt-ats() function of opencv, select the white area whose area is smaller than the area threshold, and calculate the coordinates of the corresponding center point, where the key parameter connectivity is set to 8;

[0047] According to the two-dimensional Gaussian function, the feature information corresponding to each selected white area is calculated in turn, and the small area feature map corresponding to the input label map is obtained according to all the feature information:

[0048]

[0049] Among them, (x0, y0) is the coordinate of the center point of a selected white area, (x, y) is the coordinate variable in the white area corresponding to (x0, y0), Represents the feature information corresponding to the white area with (x0, y0) as the center coordinate, λ is the two-dimensional Gaussian coefficient, and σ is the area width control coefficient;

[0050] Specifically, in this embodiment, the area threshold is set to 400-500 pixels, λ is set to 1000, and σ is set to 4;

[0051] Step 4, such as Figure 3 As shown, a fusion feature layer is constructed, the fusion feature layer includes a first fusion feature layer to a fourth fusion feature layer, and the four fusion feature layers all use a supervisory attention module;

[0052] After d4 is fused with the small area feature map, it is used as the input value of the first fusion feature layer. The input value of the first fusion feature layer is first convolved, and then the BatchNorm2d operation is performed to normalize the features, and then sigmoid activation is performed. The sigmoid activation value is randomly extracted through the Dropout operation to prevent overfitting. The Dropout result is negated and fused with the input value of the first fusion feature layer. The fusion result is fused with the small area feature map again as MASK1 of the first fusion feature layer;

[0053] MASK1 of the first fusion feature layer is bilinearly interpolated and then fused with d3. The fusion result is fused with the small area feature map again, and the result is used as the input value of the second fusion feature layer. The input value of the second fusion feature layer is first convolved, and then the BatchNorm2d operation is performed to normalize the features, and then sigmoid activation is performed. The sigmoid activation value is randomly extracted through the Dropout operation to prevent overfitting. The Dropout result is negated and fused with the input value of the second fusion feature layer. The fusion result is fused with the small area feature map again as MASK2 of the second fusion feature layer;

[0054] MASK2 of the second fusion feature layer is bilinearly interpolated and then fused with d2. The fusion result is fused with the small area feature map again, and the result is used as the input value of the third fusion feature layer. The input value of the third fusion feature layer is first convolved, and then the BatchNorm2d operation is performed to normalize the features, and then sigmoid activation is performed. The sigmoid activation value is randomly extracted through the Dropout operation to prevent overfitting. The Dropout result is negated and fused with the input value of the third fusion feature layer. The fusion result is fused with the small area feature map again as MASK3 of the third fusion feature layer;

[0055] MASK3 of the third fusion feature layer is bilinearly interpolated and then fused with d1. The fusion result is fused with the small area feature map again, and the result is used as the input value of the fourth fusion feature layer. The input value of the fourth fusion feature layer is first convolved, and then the BatchNorm2d operation is performed to normalize the features, and then sigmoid activation is performed. The sigmoid activation value is randomly extracted through the Dropout operation to prevent overfitting. The Dropout result is negated and fused with the input value of the fourth fusion feature layer. The fusion result is fused with the small area feature map again as MASK4 of the fourth fusion feature layer;

[0056] Specifically, in this embodiment, Dropout is set to 0.3;

[0057] Step 5, such as Figure 3 As shown, MASK1, MASK2, MASK3, and MASK4 of the first to fourth fusion feature layers are upsampled by 32 times, 16 times, 8 times, and 4 times respectively, and then converted into a prediction map of a size corresponding to the input label map. The binary classification threshold thres is set, and the prediction map is converted into a binary prediction map according to the binary classification threshold thres:

[0058]

[0059] Among them, p n (i, j) represents the predicted value of the pixel with coordinates (i, j) in the prediction map corresponding to the nth fusion feature layer, n = 1, 2, 3, 4, F n (i, j) represents the gray value of the pixel with coordinates (i, j) in the binary prediction image corresponding to the nth fusion feature layer;

[0060] Specifically, in this embodiment, the binary classification threshold thres is set to 0.3;

[0061] Set the initial learning rate, calculate BCELoss and DiceLoss for the binary prediction images corresponding to the first to fourth fusion feature layers and the input label images, and sum the BCELoss and DiceLoss calculation results corresponding to the nth fusion feature layer. The sum is recorded as loss n , and then calculate the total loss value Loss corresponding to the first fusion feature layer to the fourth fusion feature layer:

[0062] loss=loss1+loss2+loss3+loss4;

[0063] Then, the high-resolution network MobileNet, FPN feature pyramid and fusion feature layer are back-propagated and trained by the gradient descent method until the total loss value Loss converges or the maximum number of iterations is reached. The training is terminated and the current high-resolution network MobileNet, FPN feature pyramid and fusion feature layer models are saved.

[0064] Step 6: Obtain image 1 and image 2 with a time span in the same scene, input them into the trained high-resolution network MobileNet and FPN feature pyramid, and obtain the outputs corresponding to the first to fourth layers of the FPN feature pyramid, which are recorded as d 1 ,d 2 ,d 3 With d 4 ;

[0065] D 4 As the input value of the trained first fusion feature layer, the input value of the first fusion feature layer is first convolved, then the BatchNorm2d operation is performed to normalize the features, and then sigmoid activation is performed. The sigmoid activation value is inverted and fused with the input value of the first fusion feature layer to obtain the MASK of the first fusion feature layer. 1 ;

[0066] The MASK of the first fusion feature layer 1 After bilinear interpolation and d 3 The fusion result is used as the input value of the trained second fusion feature layer. The input value of the second fusion feature layer is first convolved, and then the BatchNorm2d operation is performed to normalize the features. Then, sigmoid activation is performed. The sigmoid activation value is inverted and fused with the input value of the second fusion feature layer to obtain the MASK of the second fusion feature layer. 2 ;

[0067] The MASK of the second fusion feature layer 2 After bilinear interpolation and d 2 The fusion result is used as the input value of the trained third fusion feature layer. The input value of the third fusion feature layer is first convolved, and then the BatchNorm2d operation is performed to normalize the features. Then, sigmoid activation is performed. The sigmoid activation value is inverted and fused with the input value of the third fusion feature layer to obtain the mask of the third fusion feature layer. 3 ;

[0068] The MASK of the third fusion feature layer 3 After bilinear interpolation and d 1The fusion result is used as the input value of the trained fourth fusion feature layer. The input value of the fourth fusion feature layer is first convolved, and then the BatchNorm2d operation is performed to normalize the features, and then sigmoid activation is performed. The sigmoid activation value is inverted and fused with the input value of the fourth fusion feature layer to obtain the MASK of the fourth fusion feature layer. 4 ;

[0069] Set the MASK of the first to fourth fusion feature layers 1 、MASK 2 、MASK 3 、MASK 4 After upsampling by 32 times, 16 times, 8 times, and 4 times, they are converted into corresponding prediction maps. According to the prediction maps corresponding to the first fusion feature layer to the fourth fusion feature layer, the change detection binary maps corresponding to image one and image two are obtained:

[0070]

[0071] Among them, p n (i,j) represents the predicted value of the pixel with coordinates (i,j) in the prediction map corresponding to the nth fusion feature layer; F(i,j) represents the grayscale value of the pixel with coordinates (i,j) in the change detection binary map; the black area represents the unchanged area between image one and image two, and the white area represents the changed area between image one and image two, thus completing the change detection based on the enhancement of small area feature information.

[0072] Specifically, Figure 5 , Figure 6 As shown in the figure, the change detection result based on small area feature information enhancement in this embodiment; in this embodiment, the algorithm training environment is built in the training stage, and the hardware requirements are: equipped with a GPU of NVIDIA GeForce RTX2080Ti, a server with more than 32G memory, CUDA 11.1, CUDNN 8.0.4, Python 3.8 and related dependent libraries, OpenCV 4.5.3, Pytorch 1.9.1, etc.; the important parameter settings are: batch size: 32, optimization algorithm: Adam, initial learning rate: 9e-4, iterative training epoch: 150 times; data sets LEVIR-CD, CDD.

[0073] In summary, the network design of the present invention is significantly different from previous network designs. It records the coordinates of the center point of a small area and uses two-dimensional Gaussian to generate feature information of the small area, which can well characterize the feature information of the small area. The two-dimensional Gaussian can highlight the local small area and suppress other areas, thereby improving the problem of being unable to characterize small areas in change detection due to the deepening of the network layer.

[0074] Those skilled in the art will appreciate that the embodiments described are intended to help readers understand the principles of the present invention, and should be understood that the scope of protection of the present invention is not limited to the embodiments described. For those skilled in the art, the present invention may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A change detection method based on small area feature information enhancement, characterized in that: The steps include: Step 1, using the LEVIR-CD dataset as a training set, each training data in the training set includes an input image 1 and an input image 2 with a time span in the same scene, and a corresponding input label map, the input image 1 and the input image 2 are 3-channel RGB images, the input label map is a single-channel binary image, the black area in the input label map represents the unchanged area between the input image 1 and the input image 2, and the white area represents the changed area between the input image 1 and the input image 2; Step 2, inputting the input image 1 and the input image 2 in the training data into a high-resolution network MobileNet respectively, for extracting image features, wherein the high-resolution network MobileNet has a four-layer structure, wherein the first layer corresponds to the highest resolution, and the resolutions of the second, third, and fourth layers decrease in sequence, and each layer of the two high-resolution networks MobileNet outputs image features to the same FPN feature pyramid for fusing feature information of different scales, wherein the FPN feature pyramid has a four-layer structure, wherein the first layer corresponds to the highest resolution, and the resolutions of the second, third, and fourth layers decrease in sequence; the outputs of the first to fourth layers of the high-resolution network MobileNet respectively correspond to the inputs of the first to fourth layers of the FPN feature pyramid, and the outputs of the first to fourth layers of the FPN feature pyramid are recorded as d1, d2, d3, and d4, respectively; Among them, the first to fourth layers of the high-resolution network MobileNet and the FPN feature pyramid downsample the input image by 1 / 4, 1 / 8, 1 / 16, and 1 / 32 respectively; Step 3, set the area threshold. For each white area in the input label map, obtain the connected domain parameters according to the connectedComponentsWithSt-ats() function of OpenCV, select the white area whose area is smaller than the area threshold, and calculate the coordinates of the corresponding center point. The key parameter connectivity is set to 8. According to the two-dimensional Gaussian function, the feature information corresponding to each selected white area is calculated in turn, and the small area feature map corresponding to the input label map is obtained according to all the feature information: Among them, (x0, y0) is the coordinate of the center point of a selected white area, (x, y) is the coordinate variable in the white area corresponding to (x0, y0), Represents the feature information corresponding to the white area with (x0, y0) as the center coordinate, λ is the two-dimensional Gaussian coefficient, and σ is the area width control coefficient; Step 4, constructing a fusion feature layer, wherein the fusion feature layer includes the first fusion feature layer to the fourth fusion feature layer, and the four fusion feature layers all use a supervisory attention module; After d4 is fused with the small area feature map, it is used as the input value of the first fusion feature layer. The input value of the first fusion feature layer is first convolved, and then the BatchNorm2d operation is performed to normalize the features, and then sigmoid activation is performed. The sigmoid activation value is randomly extracted through the Dropout operation to prevent overfitting. The Dropout result is negated and fused with the input value of the first fusion feature layer. The fusion result is fused with the small area feature map again as MASK1 of the first fusion feature layer; MASK1 of the first fusion feature layer is bilinearly interpolated and then fused with d3. The fusion result is fused with the small area feature map again, and the result is used as the input value of the second fusion feature layer. The input value of the second fusion feature layer is first convolved, and then the BatchNorm2d operation is performed to normalize the features, and then sigmoid activation is performed. The sigmoid activation value is randomly extracted through the Dropout operation to prevent overfitting. The Dropout result is negated and fused with the input value of the second fusion feature layer. The fusion result is fused with the small area feature map again as MASK2 of the second fusion feature layer; MASK2 of the second fusion feature layer is bilinearly interpolated and then fused with d2. The fusion result is fused again with the small area feature map, and the result is used as the input value of the third fusion feature layer. The input value of the third fusion feature layer is first convolved, and then the BatchNorm2d operation is performed. After the feature is normalized, sigmoid activation is performed. The sigmoid activation value is randomly extracted through the Dropout operation to prevent overfitting. The Dropout result is negated and fused with the input value of the third fusion feature layer. The fusion result is fused again with the small area feature map as MASK3 of the third fusion feature layer; MASK3 of the third fusion feature layer is bilinearly interpolated and then fused with d1. The fusion result is fused with the small area feature map again, and the result is used as the input value of the fourth fusion feature layer. The input value of the fourth fusion feature layer is first convolved, and then the BatchNorm2d operation is performed to normalize the features, and then sigmoid activation is performed. The sigmoid activation value is randomly extracted through the Dropout operation to prevent overfitting. The Dropout result is negated and fused with the input value of the fourth fusion feature layer. The fusion result is fused with the small area feature map again as MASK4 of the fourth fusion feature layer; Step 5: Upsample the MASK1, MASK2, MASK3, and MASK4 of the first to fourth fusion feature layers by 32 times, 16 times, 8 times, and 4 times, respectively, and convert them into prediction images of the size corresponding to the input label image. Set the binary classification threshold thres, and convert the prediction image into a binary prediction image according to the binary classification threshold thres: Among them, p n (i, j) represents the predicted value of the pixel with coordinates (i, j) in the prediction map corresponding to the nth fusion feature layer, n = 1, 2, 3, 4, F n (i, j) represents the gray value of the pixel with coordinates (i, j) in the binary prediction image corresponding to the nth fusion feature layer; Set the initial learning rate, calculate BCELoss and DiceLoss for the binary prediction images corresponding to the first to fourth fusion feature layers and the input label images, and sum the BCELoss and DiceLoss calculation results corresponding to the nth fusion feature layer. The sum is recorded as loss n , and then calculate the total loss value Loss corresponding to the first fusion feature layer to the fourth fusion feature layer: Loss=loss1+loss2+loss3+loss4; Then, the high-resolution network MobileNet, FPN feature pyramid and fusion feature layer are back-propagated and trained by the gradient descent method until the total loss value Loss converges or the maximum number of iterations is reached. The training is terminated and the current high-resolution network MobileNet, FPN feature pyramid and fusion feature layer models are saved. Step 6: Obtain image 1 and image 2 with a time span in the same scene, input them into the trained high-resolution network MobileNet and FPN feature pyramid, and obtain the outputs corresponding to the first to fourth layers of the FPN feature pyramid, which are recorded as d 1 d 2 d 3 With d 4 ; D 4 As the input value of the trained first fusion feature layer, the input value of the first fusion feature layer is first convolved, then the BatchNorm2d operation is performed to normalize the features, and then sigmoid activation is performed. The sigmoid activation value is inverted and fused with the input value of the first fusion feature layer to obtain the MASK of the first fusion feature layer. 1 ; The MASK of the first fusion feature layer 1 After bilinear interpolation and d 3 The fusion result is used as the input value of the trained second fusion feature layer. The input value of the second fusion feature layer is first convolved, and then the BatchNorm2d operation is performed to normalize the features. Then, sigmoid activation is performed. The sigmoid activation value is inverted and fused with the input value of the second fusion feature layer to obtain the MASK of the second fusion feature layer. 2 ; The MASK of the second fusion feature layer 2 After bilinear interpolation and d 2 The fusion result is used as the input value of the trained third fusion feature layer. The input value of the third fusion feature layer is first convolved, and then the BatchNorm2d operation is performed to normalize the features. Then, sigmoid activation is performed. The sigmoid activation value is inverted and fused with the input value of the third fusion feature layer to obtain the mask of the third fusion feature layer. 3 ; The MASK of the third fusion feature layer 3 After bilinear interpolation and d 1 The fusion result is used as the input value of the trained fourth fusion feature layer. The input value of the fourth fusion feature layer is first convolved, and then the BatchNorm2d operation is performed to normalize the features, and then sigmoid activation is performed. The sigmoid activation value is inverted and fused with the input value of the fourth fusion feature layer to obtain the MASK of the fourth fusion feature layer. 4 ; Set the MASK of the first to fourth fusion feature layers 1 、MASK 2 、MASK 3 、MASK 4 After upsampling by 32 times, 16 times, 8 times, and 4 times, they are converted into corresponding prediction maps. According to the prediction maps corresponding to the first fusion feature layer to the fourth fusion feature layer, the change detection binary maps corresponding to image one and image two are obtained: Among them, p n (i,j) represents the predicted value of the pixel with coordinates (i,j) in the prediction map corresponding to the nth fusion feature layer; F(i,j) represents the grayscale value of the pixel with coordinates (i,j) in the change detection binary map; the black area represents the unchanged area between image one and image two, and the white area represents the changed area between image one and image two, thus completing the change detection based on the enhancement of small area feature information.

Images