A change detection method that takes into account both weak changes and sudden changes

Abstract

The present invention discloses a change detection method that takes into account both weak changes and sudden changes. First, an existing optical flow data set and a target detection data set are used to construct a new data set with two characteristics of slow movement and sudden change, and the labeled data set is divided into a training set, a verification set and a test set. Then, a change detection network is constructed, including an optical flow detection branch and a sudden change detection branch. The optical flow detection branch uses a pyramid structure to extract multi-scale displacements; the sudden change detection branch uses a ResNet structure as the backbone network, and converts the output result into a black and white binary image through multi-scale pooling, upsampling and convolution. The constructed neural network is trained using an SGD optimizer. Finally, the image to be tested is input and tested using the trained model. The present invention can fuse multi-scale displacement change features, can simultaneously detect the features of sudden changes and weak changes, and obtain better change detection effects.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to a change detection method. Background Art

[0002] Change detection is a very important field in computer vision. It refers to identifying pixel-level change information in a scene by analyzing and comparing image pairs taken at the same location at different times. Its goal is to detect whether a dual-phase image has undergone weak or sudden changes.

[0003] In the field of image processing, optical flow detection technology is generally used for subtle changes, and change detection technology is generally used for changes that appear or disappear due to rapid motion. However, subtle changes and sudden changes often exist at the same time in real life. For example, in a landslide scene, there are weak changes caused by slow movement and the alternation of new and old objects appearing and disappearing due to rapid motion. The above single detection technology cannot take into account two different types of changes.

[0004] For changes that appear and disappear, the paper "Building Change Detection for Remote Sensing Images Using a Dual Task Constrained Deep Siamese Convolutional Network Model" proposes using semantic segmentation to assist in change detection, which helps to learn more discriminative object-level features and obtain change detection results. The network is based on a twin network structure and includes two encoding branches and one decoding branch. The two encoding branches are twin networks for building semantic segmentation. By learning the semantic information of the building, they provide object-level feature guidance for the model. Specifically, the encoding branch uses SE-ResNet as the backbone network and introduces a spatial pyramid pooling module to expand the receptive field and embed features of different scales. The decoding branch uses D-LinkNet as the backbone network, receives feature inputs of different scales from the two twin encoding branches, and uses spatial and channel attention mechanisms to predict change detection. This type of change detection can focus on sudden changes of different scales, but the effect is poor for weak change detection.

[0005] As an important branch of change detection, deep optical flow can detect even smaller changes. The paper "Optical Flow Estimation using a Spatial Pyramid Network" proposes a deep optical flow learning method based on a pyramid model. The paper downsamples the dual-phase image layer by layer, constructs an inverted pyramid structure from fine resolution to coarse resolution, and estimates the motion in the image from the bottom of the pyramid from coarse to fine (from large-scale displacement to small-scale displacement). Then, the optical flow estimator output by each layer is input into the previous layer until the layer with the original resolution is reached to obtain the final optical flow estimator. The network can estimate weak changes well, but for situations in real life where there are both weak changes and sudden changes, such as slopes, dams, tailings and other scenes (there are not only weak changes of slow movement but also rapid changes of collapse), the detection effect of the network is not good. Summary of the invention

[0006] In order to overcome the shortcomings of the prior art, the present invention provides a change detection method that takes into account both weak changes and sudden changes. First, the existing optical flow data set and target detection data set are used to construct a new data set with two characteristics of slow movement and sudden change, and the labeled data set is divided into a training set, a verification set and a test set. Then, a change detection network is constructed, including an optical flow detection branch and a sudden change detection branch. The optical flow detection branch uses a pyramid structure to extract multi-scale displacements; the sudden change detection branch uses a ResNet structure as the backbone network, and converts the output result into a black and white binary image through multi-scale pooling, upsampling and convolution. The constructed neural network is trained using an SGD optimizer. Finally, the image to be tested is input and tested using the trained model. The present invention can fuse multi-scale displacement change features, can simultaneously detect the features of sudden changes and weak changes, and obtain better change detection effects.

[0007] The technical solution adopted by the present invention to solve the technical problem includes the following steps:

[0008] Step 1: Build a dataset;

[0009] Each pair of bi-temporal images includes an optical flow detection label and a change detection label;

[0010] Select potential change objects from the semantic segmentation dataset, paste them into the dual-phase image through image preprocessing technology, and save the corresponding binary change detection labels to form training sets and test sets;

[0011] Step 2: Build a change detection network;

[0012] The change detection network includes an optical flow detection branch and a sudden change detection branch; the optical flow detection branch adopts a pyramid structure to extract multi-scale displacement changes and obtain an optical flow output; the sudden change detection branch uses a network structure with a backbone network of ResNet, and then transforms it into a black and white binary image through multi-scale pooling, upsampling and convolution as a sudden change output;

[0013] Step 3: Train the change detection network, input the training set data into the change detection network in batches, and obtain the optical flow output and sudden change output; the sudden change detection branch uses the change detection label to calculate the binary cross entropy loss; the optical flow detection branch uses the optical flow detection label to calculate the L2 norm loss, and uses the true value label of the sudden change detection as a mask to only calculate the loss of the non-sudden change area; the two losses are combined and optimized using the SGD optimizer until the value of the objective function stops decreasing and the training ends;

[0014] Step 4: Input the test set into the change detection network trained in step 3 to obtain the predicted result graph, and use the average EPE index and the comprehensive FEPE index of the F1 index for comprehensive evaluation.

[0015] Preferably, the sudden change refers to that the moving object in the two previous and next frames moves beyond the image boundary, and the weak change refers to that the moving object in the two previous and next frames does not move beyond the image boundary.

[0016] Preferably, the semantic segmentation dataset is the PASCAL VOC 2007 dataset; the potential change objects are people, animals, and vehicles.

[0017] Preferably, the image preprocessing technology includes copying, pasting, data enhancement, image brightness enhancement, and contrast enhancement.

[0018] Preferably, the dual-phase image adopts the FlyingChairs dataset.

[0019] Preferably, the F1 index is the average intersection-over-union ratio, and the average EPE index is the mean of the Euclidean distance between the estimated flow and the actual flow.

[0020] Preferably, the step 2 is specifically as follows:

[0021] Step 2-1: Construct optical flow detection branch:

[0022] Input dual-phase image T0 0 and T0 1 , assuming its size is (H, W), the optical flow detection has a total of 5 U-shaped cascade branch layers, the specific number of layers is recorded as k∈[0,4]; each branch includes a warp operation w(·), an upsampling operation u(·), a 2x downsampling operation and a convolution block G k, namely G0, G1, G2, G3, G4;

[0023] The input of the kth branch layer is the bi-temporal image T k-1 0 With T k-1 1 ; Downsample the dual-phase image, denoted as T k 0 and T k 1 , the corresponding size is (H / 2 k ,W / 2 k );The optical flow estimation V for the k+1th branch layer k+1 Upsampling is performed to obtain u(V k+1 ), through the convolutional block G k Estimate the current optical flow residual v k , add the two together to get the optical flow estimation output V of the current kth layer k , as shown in formula (1):

[0024] V k =u(V k+1 )+v k (1)

[0025] Convolutional block G in the optical flow detection branch k The input is the image T at time T0 k 0 , the upsampling result u(V k+1 ) and motion compensated images in It means that u(V k+1 ) and the image T at time T1 k 1 Perform warp operation; after stacking the channels of the above three inputs, they pass through five convolutional layers, with convolution kernels of 5*5 and step sizes of 1. The number of output channels is 32, 64, 32, 16, and 2 respectively; convolution block G k The output is recorded as v k , as the optical flow residual of the current branch layer, see formula (2):

[0026]

[0027] For the bottom layer of the optical flow detection branch, i.e., the 4th layer, since there is no optical flow estimation V obtained from the previous layer, k+1 Therefore, the 0 matrix is ​​selected as the initialization input. After passing through the convolution block G4, the output V4 of the bottom layer of the pyramid is obtained; for the top layer of the pyramid network, that is, the 0th layer, no downsampling operation is required. The output of this layer is recorded as V0, which is the final output of the optical flow network branch, that is, output1;

[0028] Step 2-2: Construct a sudden change detection branch:

[0029] The sudden change detection branch has three inputs: dual-phase image T0 0 and T0 1 And the result output1 obtained by the optical flow detection branch;

[0030] First, output1 and image T0 at time T1 are compared. 1 Perform warp operation to obtain motion compensated image The image T0 at time T0 0 With motion compensated image Perform an absolute difference operation; then, input the absolute difference result into the convolution block of the backbone network ResNet50, and the output is recorded as F0; then, input the feature map F0 into 4 parallel maximum pooling operations (1*1, 2*2, 3*3, 6*6) to generate 4 feature maps of different sizes F1, F2, F3, F4; then, F1, F2, F3, F4 are upsampled differently to obtain 4 feature maps of the same size (H, W), and stacked with the original feature map F0, and then through 3*3 convolution with a step size of 1, regularization and ReLU activation function to obtain a feature map F5 with 512 channels. Finally, after a 3*3 convolution with a step size of 1 and a Sigmoid activation function, a single-channel binary change feature map F6 is obtained as the final output of the sudden change detection branch, that is, output2.

[0031] Preferably, the step 4 is specifically:

[0032] Combining the F1 index and the average EPE index, a new evaluation index FEPE was designed;

[0033] 1) F1 Index

[0034] There are four combinations of prediction results and change detection labels obtained through change detection: TP, FP, FN, and TN. Specifically, TP refers to a positive sample that is predicted to be correct, TN refers to a negative sample that is predicted to be incorrect, FP refers to a negative sample that is predicted to be correct, and FN refers to a positive sample that is predicted to be incorrect. The F1 index is the harmonic mean of Precision (see formula (6)) and Recall (see formula (7), and its value range is [0,1], see formula (8).

[0035] Precision=TP / (TP+FP) (6)

[0036] Recall=TP / (TP+FN) (7)

[0037]

[0038] 2) Average EPE index

[0039] The EPE indicator calculates the Euclidean distance between the optical flow estimate and the actual flow of the label. The optical flow estimate is recorded as The actual flow rate is recorded as See formula 9:

[0040]

[0041] A new average EPE index, denoted as mEPE, is proposed. The mEPE index is a measure of the offset label area Q and the offset prediction area Q. * All pixels in the union of are averaged, see formula (10):

[0042]

[0043] 3) FEPE index

[0044] The value range of FEPE index is [0, +∞]. In order to ensure that the denominator is not 0, a disturbance ξ is added to the denominator. For details, see formula (11):

[0045]

[0046] When the F1 index is larger / smaller, the change detection effect is better / worse; when the average EPE index is smaller / larger, the optical flow estimation effect is better / worse; when the FEPE index is larger / smaller, the prediction effect is better / worse; and a comprehensive evaluation is achieved.

[0047] The beneficial effects of the present invention are as follows:

[0048] The present invention can detect sudden changes and slight changes at the same time, and obtain better change detection effect; a new loss function is designed, which integrates the binary cross entropy loss of the change detection branch and the L2 norm loss of the optical flow detection branch; at the same time, a new evaluation index FEPE is designed, which can standardize the average EPE index of optical flow estimation and the F1 index of transformation detection for comprehensive evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a structural diagram of the change detection network of the present invention.

[0050] Figure 2 It is a structural diagram of the convolution module in the optical flow detection branch of the present invention.

[0051] Figure 3 It is a network structure diagram of the sudden change detection branch of the present invention. DETAILED DESCRIPTION

[0052] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0053] The present invention designs a change detection method that takes into account both weak changes and sudden changes. Since both weak changes and sudden changes exist in real life, if two consecutive frames are selected, it is not enough to effectively detect the changes of slow movement (1-2 pixels), so image frames with a certain interval are required. At this time, fast-moving objects will appear suddenly or disappear in the two frames before and after. The distinction between weak changes and sudden changes is related to the interval length between the two image frames. When the moving object exceeds the image boundary, it is a sudden change, otherwise it is a weak change.

[0054] First, the existing optical flow dataset FlyingChairs and the target detection dataset PASCAL VOC 2007 are used to construct a new dataset with two characteristics of slow movement and sudden change, and the labeled dataset is divided into a training set, a validation set and a test set. Then, a change detection network is constructed, and the network includes an optical flow detection branch and a sudden change detection branch. The pyramid structure is used in the optical flow detection branch to extract multi-scale displacement changes, and the distance between the corresponding pixels of the dual-phase image is reduced by the warp operation (bilinear difference is performed between the image at time T1 and the estimated optical flow vector, and the image at time T1 is transformed to the perspective of the image at time T0). The sudden change detection branch uses the structure of ResNet as the backbone network, and the output result is converted into a black and white binary image through multi-scale pooling, upsampling and convolution. Then, the data in the training set is put into the network in batches, and the constructed neural network is trained using the SGD optimizer. Finally, the image to be tested is input and tested using the trained model. The present invention can fuse the multi-scale displacement change features, can simultaneously detect the features of sudden changes and weak changes, and obtain a better change detection effect.

[0055] The present invention solves the technical problem by adopting a technical solution: a detection method that takes both weak changes and sudden changes into consideration, which is characterized by comprising the following steps:

[0056] Step 1, data set construction, in which each pair of dual-phase images has an optical flow detection label and a change detection label. In real life, it is very difficult to make fine labels for images with both weak changes and sudden changes, so the present invention adopts the idea of ​​making artificial data sets. Specifically, potential change objects (such as people, animals, vehicles, etc.) can be selected from semantic segmentation data sets (such as PASCAL VOC 2007 data sets), and pasted into dual-phase images (such as FlyingChairs data sets) by copying and pasting data enhancement and image brightness enhancement, contrast enhancement and other technologies, and save the corresponding binary change detection labels. The idea of ​​constructing the data set is as described above, and the specific operations are not limited to these two data sets.

[0057] Step 2: Build a change detection network, including an optical flow detection branch and a change detection branch. The optical flow detection branch uses a pyramid structure to extract multi-scale displacement changes and obtain optical flow output. The sudden change detection branch uses a network structure with ResNet as the backbone network, and then transforms it into a black and white binary image through multi-scale pooling, upsampling and convolution as the sudden change output.

[0058] Step 3: Train the constructed network, put the training data into the network in batches, and obtain the optical flow output and sudden change output. The sudden change detection branch uses the change detection label to calculate the binary cross entropy loss. The optical flow detection branch uses the optical flow detection label to calculate the L2 norm loss, and uses the true value label of the sudden change detection as a mask to only calculate the loss of the non-sudden change area. Combine the two losses and use the SGD optimizer for optimization until the value of the objective function does not decrease and the training ends.

[0059] Step 4: Input the test image set into the change detection network constructed in step 3 to obtain the predicted result map, and use the average EPE index (the mean of the Euclidean distance between the estimated flow and the actual flow) and the F1 index for comprehensive evaluation. Specific embodiment:

[0061] 1. Prepare and build the dataset

[0062] The data set of the present invention takes the existing optical flow data set FlyingChairs and the semantic segmentation data set PASCALVOC 2007 as examples, but the specific operation is not limited to these two data sets. Each group of data has 4 images, including the transformed FlyingChairs dual-phase image, optical flow detection label and change detection label. Specifically, by copying and pasting data enhancement and image brightness enhancement, contrast enhancement and other technologies, the potential change objects (people, animals, vehicles) selected in the PASCAL VOC 2007 data set are scaled and pasted into the T0 / T1 image of the dual-phase image set (FlyingChairs data set), and the corresponding binary change detection label is saved, and the corresponding T1 / T0 image and optical flow detection label are retained. The training set contains 10,000 pairs of images, the test set contains 2,000 pairs of images, and the verification set contains multiple video sets involving both weak changes and sudden changes. By extracting frames, the detection effect of the network for the two changes is verified.

[0063] 2. Build a change detection network

[0064] The network includes an optical flow detection branch and a sudden change detection branch. The optical flow detection branch uses a pyramid structure to extract multi-scale displacement changes, and the sudden change detection branch uses a network structure based on spatial pyramid pooling to convert the output results into black and white binary images through convolution. The specific steps are as follows:

[0065] 1) Construct optical flow detection branch: refer to Figure 1 , input dual-phase image T0 0 and T0 1 , assuming its size is (H, W), the optical flow detection has a total of 5 U-shaped cascade branch layers, the specific number of layers is recorded as k∈[0,4]. Each branch contains a warp operation (i.e. w(·)), an upsampling operation (i.e. u(·)), a 2x downsampling operation and a convolution block G k (i.e. G0, G1, G2, G3, G4). The input of the kth branch layer is the dual-phase image T k-1 0 With T k-1 1 , First, downsample the dual-phase image, denoted by T k 0 and T k 1 , the corresponding size is (H / 2 k ,W / 2 k ); Then, in order to obtain the optical flow estimation output V of the current branch layer k , the optical flow estimator of the previous layer (the k+1th branch layer) (i.e. V k+1 ) is upsampled to obtain u(V k+1) to ensure that the image size is the same as that of the current branch layer, and passes through the convolution block G k Estimate the current optical flow residual v k , add the two together to get the output V of the current kth layer k (i.e., the optical flow estimator of the current layer), as shown in Formula 1.

[0066] V k =u(V k+1 )+v k (1)

[0067] The specific structure of the convolution block in the optical flow detection branch is referenced Figure 2 , convolutional block G k The input is the image T at time T0 k 0 , the upsampling result u(V k+1 ) and motion compensated images in It means that u(V k+1 ) and the image T at time T1 k 1 After stacking the channels of the above three inputs, they pass through five convolutional layers, with convolution kernels of 5*5 and strides of 1. The number of output channels is 32, 64, 32, 16, and 2 respectively. Convolutional block G k The output is recorded as v k , as the optical flow residual of the current branch layer, see formula 2.

[0068]

[0069] For the bottom layer (layer 4) of the pyramid network, since there is no optical flow estimate V obtained in the previous layer k+1 (i.e. V5), so the 0 matrix is ​​selected as the initialization input, and after passing through the convolution block G4, the output V4 of the bottom layer of the pyramid is obtained. For the top layer (layer 0) of the pyramid network, no downsampling operation is required, and the output of this layer is recorded as V0, which is the final output of the optical flow network branch, i.e. output1.

[0070] 2) Construct a sudden change detection branch: refer to Figure 3 , the sudden change detection branch has three inputs: dual-phase image T0 0 and T0 1 (share the input of the optical flow detection branch) and the result output1 of the optical flow detection branch. First, compare output1 with the image T0 at time T1 1 Perform warp operation to obtain motion compensated image The image T0 at time T0 0With motion compensated image Perform an absolute difference operation; then, input the absolute difference result into the convolution block of the backbone network ResNet50, and the output is recorded as F0; then, the feature map F0 is input into 4 parallel maximum pooling operations (1*1, 2*2, 3*3, 6*6) to generate 4 feature maps of different sizes F1, F2, F3, F4; then, F1, F2, F3, F4 are upsampled differently to obtain 4 feature maps of the same size (H, W), and the channels are stacked with the original feature map F0, and then 3*3 convolution (step size is 1), regularization and ReLU activation function are performed to obtain the feature map F5 with 512 channels. Finally, after 3*3 convolution (step size is 1) and Sigmoid activation function, a single-channel binary change feature map F6 is obtained as the final output of the sudden change detection branch, that is, output2.

[0071] 3. Network training

[0072] The constructed network is trained, and the training data is put into the network in batches to obtain the optical flow output and change output. In the optical flow detection branch, in order to make the optical flow detection detect only weak changes, the area of ​​the change detection label label2 is removed from the optical flow detection label label1, and then the L2 norm loss is calculated, see formula 3. The sudden change detection branch uses the change detection label label2 to calculate the binary cross entropy loss (BCEloss), see formula 4. Different weights w are assigned according to the different orders of magnitude of the two losses (for the example data set, w = 100), and the total loss of training is obtained, see formula 5. The training process is optimized using the SGD optimizer until the value of the objective function does not decrease and the training ends.

[0073] loss l2 =||output1-label1*(1-label2)||2 (3)

[0074] loss BCE =-(label2 log(output2+(1-label2)log(1-output2))) (4)

[0075] loss=loss l2 +w*loss BCE (5)

[0076] 4. Test images

[0077] The test image set is input into the change detection network constructed in step 3 to obtain the predicted result graph, and a new evaluation index FEPE is designed by combining the F1 index (average intersection-over-union ratio) and the average EPE index (the average of the Euclidean distance between the estimated flow and the actual flow). The three indicators are introduced below.

[0078] 1) F1 Index

[0079] There are four combinations of prediction results and change detection labels obtained through change detection: TP, FP, FN, and TN. Specifically, TP refers to the prediction of a correct positive sample, TN refers to the prediction of an incorrect negative sample, FP refers to the prediction of a correct negative sample, and FN refers to the prediction of an incorrect positive sample. The F1 index is the harmonic mean of Precision (see Formula 6) and Recall (see Formula 7), and its value range is [0,1], see Formula 8.

[0080] Precision=TP / (TP+FP) (6)

[0081] Recall=TP / (TP+FN) (7)

[0082]

[0083] 2) Average EPE index

[0084] The EPE indicator calculates the Euclidean distance between the optical flow estimate and the actual flow of the label. The optical flow estimate is recorded as The actual flow rate is recorded as See Formula 9.

[0085]

[0086] In order to better reflect the prediction effect of the network in the weak motion area, a new average EPE index (denoted as mEPE) is proposed. Different from the existing average EPE index (taking the average of all pixels in the entire pixel area), the mEPE index is an average of the label area with offset (denoted as Q) and the prediction area with offset (denoted as Q * ) is used to take the average of all pixels in the union of , see formula 10.

[0087]

[0088] 3) FEPE index

[0089] In order to combine the F1 index and the average EPE index, the FEPE index is designed, and its value range is [0, +∞]. In order to ensure that the denominator is not 0, a very small disturbance (denoted as ξ) is added to the denominator. See formula 11 for details.

[0090]

[0091] When the F1 index is larger / smaller (i.e., the change detection effect is better / worse), the average EPE index is smaller / larger (i.e., the optical flow estimation effect is better / worse), and the FEPE index is larger / smaller (i.e., the prediction effect is better / worse), a comprehensive evaluation can be achieved.

Claims

1. A change detection method that takes into account both weak changes and sudden changes, characterized in that: The steps include: Step 1: Build a dataset; Each pair of bi-temporal images includes an optical flow detection label and a change detection label; Select potential change objects from the semantic segmentation dataset, paste them into the dual-phase image through image preprocessing technology, and save the corresponding binary change detection labels to form training sets and test sets; Step 2: Build a change detection network; The change detection network includes an optical flow detection branch and a sudden change detection branch; the optical flow detection branch adopts a pyramid structure to extract multi-scale displacement changes and obtain an optical flow output; the sudden change detection branch uses a network structure with a backbone network of ResNet, and then transforms it into a black and white binary image through multi-scale pooling, upsampling and convolution as a sudden change output; Step 2-1: Construct optical flow detection branch: Input dual-phase image T0 0 and T0 1 , assuming its size is (H, W), the optical flow detection has a total of 5 U-shaped cascade branch layers, the specific number of layers is recorded as k∈[0,4]; each branch includes a warp operation w(·), an upsampling operation u(·), a 2x downsampling operation and a convolution block G k , namely G0, G1, G2, G3, G4; The input of the kth branch layer is the bi-temporal image T k-1 0 With T k-1 1 ; Downsample the dual-phase image, denoted by T k 0 and T k 1 , the corresponding size is (H / 2 k ,W / 2 k );The optical flow estimation V for the k+1th branch layer k+1 Up-sample to get u(V k+1 ), through the convolutional block G k Estimate the current optical flow residual v k , add the two together to get the optical flow estimation output V of the current kth layer k , as shown in formula (1): V k =u(V k+1 )+v k (1) Convolutional block G in the optical flow detection branch k The input is the image T at time T0 k 0 , the upsampling result u(V k+1 ) and the motion compensated image w(T k 1 ,u(V k+1 )), where w(T k 1 ,u(V k+1 )) refers to u(V k+1 ) and the image T at time T1 k 1 Perform warp operation; after stacking the channels of the above three inputs, they pass through five convolutional layers, with convolution kernels of 5*5 and step sizes of 1. The number of output channels is 32, 64, 32, 16, and 2 respectively; convolution block G k The output is recorded as v k , as the optical flow residual of the current branch layer, see formula (2): For the bottom layer of the optical flow detection branch, i.e., the 4th layer, since there is no optical flow estimation V obtained from the previous layer, k+1 Therefore, the 0 matrix is ​​selected as the initialization input. After passing through the convolution block G4, the output V4 of the bottom layer of the pyramid is obtained; for the top layer of the pyramid network, that is, the 0th layer, no downsampling operation is required. The output of this layer is recorded as V0, which is the final output of the optical flow network branch, that is, output1; Step 2-2: Construct a sudden change detection branch: The sudden change detection branch has three inputs: dual-phase image T0 0 and T0 1 And the result output1 obtained by the optical flow detection branch; First, output1 and image T0 at time T1 are compared 1 Perform warp operation to obtain the motion compensated image w(T0 1 ,output1), the image T0 at time T0 0 Compared with the motion compensated image w(T0 1 ,output1) for absolute difference operation; then, the absolute difference result is input into the convolution block of the backbone network ResNet50, and the output is recorded as F0; then, the feature map F0 is input into 4 parallel maximum pooling operations (1*1, 2*2, 3*3, 6*6) respectively to generate 4 feature maps of different sizes F1, F2, F3, F4; then, F1, F2, F3, F4 are upsampled differently to obtain 4 feature maps of the same size (H, W), which are stacked with the original feature map F0, and then through 3*3 convolution with a step size of 1, regularization and ReLU activation function to obtain a feature map F5 with 512 channels; finally, after 3*3 convolution with a step size of 1 and Sigmoid activation function, a single-channel binary change feature map F6 is obtained as the final output of the sudden change detection branch, namely output2; Step 3: Train the change detection network, input the training data into the change detection network in batches, and obtain the optical flow output and sudden change output; The sudden change detection branch uses the change detection label to calculate the binary cross entropy loss; the optical flow detection branch uses the optical flow detection label to calculate the L2 norm loss, and uses the true value label of the sudden change detection as a mask to only calculate the loss of the non-sudden change area; the two losses are combined and optimized using the SGD optimizer until the value of the objective function stops decreasing and the training ends; Step 4: Input the test set into the change detection network trained in step 3 to obtain the predicted result graph, and use the average EPE index and the comprehensive FEPE index of the F1 index for comprehensive evaluation; Combining the F1 index and the average EPE index, a new evaluation index FEPE was designed; 1) F1 Index There are four combinations of prediction results and change detection labels obtained through change detection: TP, FP, FN, and TN. Specifically, TP refers to a positive sample that is predicted to be correct, TN refers to a negative sample that is predicted to be incorrect, FP refers to a negative sample that is predicted to be correct, and FN refers to a positive sample that is predicted to be incorrect. The F1 index is the harmonic mean of Precision (see formula (6)) and Recall (see formula (7), and its value range is [0,1], see formula (8). Precision=TP / (TP+FP) (6) Recall=TP / (TP+FN) (7) 2) Average EPE index The EPE indicator calculates the Euclidean distance between the optical flow estimate and the actual flow of the label. The optical flow estimate is recorded as The actual flow rate is recorded as See formula 9: A new average EPE index, denoted as mEPE, is proposed. The mEPE index is a measure of the offset label area Q and the offset prediction area Q. * All pixels in the union of are averaged, see formula (10): 3) FEPE index The value range of FEPE index is [0, +∞]. In order to ensure that the denominator is not 0, a disturbance ξ is added to the denominator. For details, see formula (11): When the F1 index is larger / smaller, the change detection effect is better / worse; when the average EPE index is smaller / larger, the optical flow estimation effect is better / worse; when the FEPE index is larger / smaller, the prediction effect is better / worse; and a comprehensive evaluation is achieved.

2. A change detection method taking into account both weak changes and sudden changes according to claim 1, characterized in that: The sudden change refers to that the moving object in the two frames before and after moves beyond the image boundary, and the weak change refers to that the moving object in the two frames before and after moves within the image boundary.

3. A change detection method taking into account both weak changes and sudden changes according to claim 1, characterized in that: The semantic segmentation dataset is the PASCAL VOC 2007 dataset; the potential objects of variation are humans, animals, and vehicles.

4. The change detection method according to claim 1, wherein: The image preprocessing techniques include copying, pasting, data enhancement, image brightness enhancement, and contrast enhancement.

5. The change detection method according to claim 1, wherein: The dual-phase image adopts the FlyingChairs dataset.

6. A change detection method taking into account both weak changes and sudden changes according to claim 1, characterized in that: The F1 index is the average intersection-over-union ratio, and the average EPE index is the mean of the Euclidean distance between the estimated flow and the actual flow.

Images