A Deep Neural Network Image Change Detection Method Incorporating Confidence Correspondence Estimation

Abstract

The present invention discloses a method for detecting image changes in a deep neural network that integrates confidence correspondence estimation, belonging to the technical field of image processing. This method is implemented using two cascaded deep sub-neural networks, namely a deep confidence correspondence estimation network and a change detection network. The method includes inputting two images to be detected into the deep confidence correspondence estimation network, outputting the correspondence between the two images and the confidence of this correspondence, while extracting image features and correspondence uncertainty features; inputting the confidence, image features, and correspondence uncertainty features into the change detection network, and further processing them through a multi-level convolutional neural network to output the change detection result. The present invention organically integrates image correspondence estimation, correspondence confidence estimation, and change detection, unifying image registration and change detection into one network framework, and simultaneously improving the effect of image change detection.

Description

Technical Field

[0001] The present invention relates to image change detection technology, and particularly to a deep neural network image change detection method integrating confidence correspondence estimation. Background Art

[0002] Image change detection refers to analyzing and comparing two images to find the changed parts. Image change detection technology has a wide range of application scenarios, such as in the fields of video surveillance, remote sensing image analysis, medical image analysis, etc.

[0003] Traditional image change detection methods mainly include threshold-based methods, difference map-based methods, and feature extraction-based methods:

[0004] The threshold-based method is one of the simplest and most direct image change detection methods. This method first grayscales the two images and then performs pixel-level comparison. If the difference between two pixel values exceeds the set threshold, it is considered that the pixel has changed. This method is sensitive to illumination changes and noise.

[0005] The difference map-based method represents the difference information between two images as a difference map. This method first grayscales the image sequence and then performs pixel-level difference calculation to obtain the difference map. In the difference map, regions with larger pixel values indicate greater changes. By setting a threshold, the changed and unchanged regions can be segmented. This method has a certain robustness to illumination changes and noise, but the effect of image change detection in complex backgrounds is not good.

[0006] The feature extraction-based method uses features in the image sequence for change detection. Features can be features in aspects such as structure, texture, color, or motion. This method first extracts features from the image sequence and then uses machine learning algorithms for classification. In the training stage, a classifier is established by learning known changed and unchanged samples. In the testing stage, the image sequence to be detected is input into the classifier to obtain the changed and unchanged results. The advantage of this method is that it can adapt to complex backgrounds and illumination changes, but it requires a large number of samples for training and has a high computational complexity.

[0007] Existing image change detection methods generally assume that the corresponding pixels of the two input images are also corresponding in the imaging target space, that is, the same position in the two images reflects the information of the same position in the imaging target space. However, in actual use, due to the inconsistent imaging positions and perspectives of the two images, the pixel coordinates corresponding to the same position in the imaging target space are not the same in the two images. Before applying traditional image change detection methods, it is necessary to additionally execute an image registration algorithm to correct the spatial inconsistency of the two images. Usually, both image registration and change detection require processing related to image feature extraction and feature matching. Therefore, independent execution of image registration and change detection will result in duplicate processing problems, leading to low processing efficiency of the entire method. Summary of the Invention

[0008] Aiming at the above deficiencies in the prior art, the method for detecting image changes in a deep neural network that fuses confidence correspondence estimation provided by the present invention solves the problem that duplicate processing exists when image registration and change detection in existing change detection are independently performed, resulting in low processing efficiency of the entire method.

[0009] To achieve the above invention purpose, the technical solution adopted by the present invention is as follows:

[0010] Provide a method for detecting image changes in a deep neural network that fuses confidence correspondence estimation. This method is implemented by two cascaded deep neural networks, namely a deep confidence correspondence estimation network and a change detection network. The method includes the following steps:

[0011] S1. Input the image to be detected A and the image to be detected B into the feature extraction part of the deep confidence correspondence estimation network for feature extraction, and obtain the full-size feature map of the image to be detected A and the pyramid features of the image to be detected B , full-size feature map ;

[0012] S2. After performing relevant operations on the full-size feature map and the full-size feature map , obtain the feature correlation volume V. Input the feature correlation volume V into the correspondence estimation part and the corresponding uncertainty feature extraction part of the deep confidence correspondence estimation network respectively, and obtain the correspondence M and the corresponding uncertainty feature F between the image to be detected A and the image to be detected B P ;

[0013] S3. Input the correspondence M and the corresponding uncertainty feature F PInput the probability regression part of the depth confidence correspondence estimation network to obtain the parameters of the corresponding uncertainty probability model; then, according to the parameters of the corresponding uncertainty probability model, calculate the confidence map T of the correspondence M between the image A to be detected and the image B to be detected.

[0014] S4. Stack the corresponding uncertainty feature F P and the feature after stacking the confidence map T and the pyramid features of the image B to be detected Input them into the change detection network, and after the change detection network processes, obtain the change detection result between the image A to be detected and the image B to be detected.

[0015] Furthermore, the method for performing correlation operations on the image features of the two images to be detected includes:

[0016] A1. For the pixel point at any position (i, j) in the image B to be detected, find all the pixel points in the surrounding neighborhood of the pixel point at the corresponding position (i, j) in the image A to be detected. The neighborhood size is , where is a constant and is selected according to the application background;

[0017] A2. Perform correlation operations on the feature vectors in the full-size feature map of the pixel point at position (i, j) and the feature vectors in the full-size feature map of all the pixel points found in step A1. The matrix composed of all the correlation operation results corresponding to the pixel point at position (i, j) is used as the correlation sub-body ;

[0018] A3. Execute steps A1 and A2 for the pixel points at the remaining positions in the image B to be detected to obtain the correlation sub-bodies corresponding to the pixel points at all positions in the image B to be detected;

[0019] A4. Use the correlation sub-bodies corresponding to the pixel points at all positions in the image B to be detected to form the feature correlation body V of the input uncertainty feature extraction part and the correspondence estimation part.

[0020] Furthermore, the method for obtaining the corresponding uncertainty feature by using the corresponding uncertainty feature extraction part includes:

[0021] Expand the feature correlation body V to 5 dimensions , and input it into the corresponding uncertainty feature extraction part composed of a multi-level convolutional network;

[0022] The feature correlation body V of dimension passes through several convolutional layers of the corresponding uncertainty feature extraction part and outputs features of dimension , is the dimension corresponding to the uncertainty feature;

[0023] Remove the middle two dimensions with dimension 1 of the feature with dimension and reconstruct it into the corresponding uncertainty feature with dimension ; h and w are the heights and widths of the images A and B to be detected, respectively. ; h and w are the heights and widths of the images A and B to be detected, respectively.

[0024] Furthermore, the method for obtaining the correspondence between two images A and B to be detected by using the correspondence estimation part includes:

[0025] Flatten the last two dimensions of the feature correlation volume to obtain a 3D tensor with dimension ; h and w are the heights and widths of the images A and B to be detected, respectively;

[0026] Pass the 3D tensor through the correspondence estimation part composed of a multi-level convolutional neural network with residual connections to obtain the correspondence with dimension : :

[0027] The corresponding relationship of the corresponding pixel B in the image B to be detected in the image A to be detected is , is the position of the corresponding pixel of B in the image A to be detected;

[0028] The multi-level convolutional neural network with residual connections in the corresponding relationship estimation part includes a convolutional layer C0, a Relu layer R0, a convolutional + Relu layer C1, and a convolutional layer C2 connected in sequence. The output of the convolutional layer C0 is superimposed with the output of the convolutional layer C2 after passing through the residual connection layer S0 to obtain a superimposed feature , and the superimposed feature is input into the Relu layer R1, the convolutional + Relu layer C3, and the convolutional layer C4 connected in sequence; the superimposed feature is superimposed with the output of the convolutional layer C4 after passing through the residual connection layer S1 to obtain a superimposed feature , and the superimposed feature passes through the Relu layer R2 and the convolutional layer C5 to output the corresponding relationship.

[0029] Furthermore, step S3 further includes:

[0030] S31. Stack the correspondence M and the corresponding uncertainty feature F P and input them into the probability regression part with a multi-level convolutional neural network as the architecture;

[0031] S32. The stacked features pass through multiple convolutional layers to obtain the mixed probability model parameters , where are the unnormalized weights of two independent probability models in the mixture probability model, , are the variances of two independent probability models in the mixture probability model; for any pixel in the image B to be detected, the corresponding mixture probability model parameters are , , , and the element in the corresponding relationship M is , which is taken as the mean of the probability density function , that is ;

[0032] S33, the unnormalized weights of two independent probability models , obtain the corresponding mixture probability model after passing through the softmax layer the normalized model weights , , where is a random variable, representing the possible values of the corresponding relationship between two images to be detected at the pixel point in the image B to be detected, and it follows the mixture probability density distribution: , where is the independent probability density function, , are the probability distribution parameters;

[0033] S34. Calculate the confidence of the corresponding relationship between two images to be detected at the pixel point in the image B to be detected according to the probability distribution parameters of the model:

[0034]

[0035] where, is a real number, which is taken according to the application background; is , the Euclidean distance between.

[0036] Furthermore, when the probability density function adopts the Laplace distribution, the expression of the confidence of the corresponding relationship between two images to be detected at the pixel point in the image B to be detected is:

[0037]

[0038] Among them, k is a variable, and its value is 1 and 2, is an exponential function.

[0039] Furthermore, the feature extraction part uses VGG-16 as the backbone network; when extracting features, the outputs of the convolutional layers before each pooling layer in the VGG-16 backbone network are taken to form the pyramid features of the image A to be detected and the image B to be detected and the pyramid features , which contain level I-V image features; then, it is upsampled by the nearest neighbor interpolation method to the same height and width as the input image and stacked to form a feature map with a dimension of , and the full-size feature maps of the image A to be detected and the image B to be detected are obtained respectively and the full-size feature map ; after the image A to be detected undergoes feature extraction, the pyramid feature and the full-size feature map are obtained; after the image B to be detected undergoes feature extraction, the pyramid feature and the full-size feature map are obtained.

[0040] Furthermore, the loss function of the deep belief correspondence estimation network is expressed as:

[0041]

[0042] Among them, are network parameters; is the training set for training the deep belief correspondence estimation network, , X is the input image pair composed of image A and image B, is the true correspondence of X; N is the total number of input image pairs; is the probability density function; and are both the normalized independent probability density model weights corresponding to the estimated image B at; and are both the probability model variances corresponding to the estimated image B at; is the correspondence corresponding to the estimated image B at; is the true correspondence corresponding to the pixel in.

[0043] Furthermore, the change detection network in step S4 adopts a U-Net-like architecture with an encoder-decoder structure. During the process of gradually reducing the encoder, pyramid features of the image B to be detected are introduced according to the scale size. The I-V level image features in are obtained. At the end of the decoder, the probability value that each pixel in the image B to be detected belongs to the changed area is obtained through the

[0044] convolutional layer. Furthermore, the expression of the loss function of the change detection network

[0045]

[0046] is as follows: where is the training set for training the change detection network, , is the pyramid image features, uncertainty features, and confidence of the correspondence of the input image pair of image A and image B, G is the changed area label map; N is the total number of input image pairs; is the label value of the pixel at in image B of the nth image pair as the changed area label value, which is 1 for change and 0 for no change; is the probability that the pixel at in image B of the image pair predicted by the network is the changed area;

[0047] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0048] 1. When performing change monitoring in this solution, the correspondence between two images is fused, so there is no need to pre-register the two input images before detection, thus improving the detection efficiency; the change detection process fuses and utilizes the correspondence information between images extracted by the correspondence estimation network, improving the detection accuracy of the changed area.

[0049] 2. The image feature C fused in the change detection process overcomes the influence of unclear correspondence in weak texture areas; the change detection process adopts an end-to-end deep neural network structure, overcoming the negative impact brought by manually selecting image features in traditional methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a flowchart of the image change detection method of the deep neural network for fusing confidence correspondence estimation.

[0051] Figure 2Detailed overall technical solution flow chart of the deep neural network image change detection method integrating confidence correspondence estimation.

[0052] Figure 3 This is a schematic diagram of using VGG-16 as the skeleton network for feature extraction.

[0053] Figure 4 This is a schematic diagram of the architecture of the uncertainty feature extraction part.

[0054] Figure 5 Schematic diagram of the architecture of the correspondence estimation part.

[0055] Figure 6 This is a schematic diagram of the architecture of the probability parameter regression part.

[0056] Figure 7 Schematic diagram of the architecture of the changing region segmentation network. DETAILED DESCRIPTION

[0057] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0058] refer to Figure 1 , Figure 1 A deep neural network image change detection method integrating confidence correspondence estimation is shown; Figure 1 As shown, the method S is implemented by two cascaded deep sub-neural networks, namely a deep confidence correspondence estimation network and a change detection network. The method S includes steps S1 to S4. The detailed overall technical solution flow chart of the method can be referred to Figure 2 .

[0059] In step S1, the image A to be detected and the image B to be detected are input into the feature extraction part of the depth confidence correspondence estimation network for feature extraction to obtain the pyramid feature of the image A to be detected. , full-size feature map And the pyramid features of the image B to be detected , full-size feature map Among them, the image A to be detected is an early image, and the image B to be detected is a late image, that is, the acquisition time of the image A to be detected is earlier than that of the image B to be detected.

[0060] like Figure 3As shown, the feature extraction part uses VGG-16 as the backbone network; when extracting features, the outputs of the convolutional layers before each pooling layer in the VGG-16 backbone network are taken to form the pyramid features of the image A to be detected and the image B to be detected and the pyramid features I-V level image features; then, they are upsampled by the nearest neighbor interpolation method to the same height and width as the input image and stacked to form a feature map with a dimension of to obtain the full-size feature maps of the image A to be detected and the image B to be detected respectively and the full-size feature map ; after feature extraction, the image A to be detected obtains the pyramid feature and the full-size feature map ; after feature extraction, the image B to be detected obtains the pyramid feature and the full-size feature map .

[0061] In step S2, after performing correlation operations on the full-size feature map and the full-size feature map to obtain the feature correlation volume V, the feature correlation volume V is respectively input into the correspondence estimation part and the corresponding uncertainty feature extraction part of the deep belief correspondence estimation network to obtain the correspondence M and the corresponding uncertainty feature F between the image A to be detected and the image B to be detected P .

[0062] In an embodiment of the present invention, the method for performing correlation operations on the image features of two images to be detected includes:

[0063] A1. For the pixel point at any position (i, j) in the image B to be detected, find all the pixel points in the neighborhood of the pixel point at the corresponding position (i, j) in the image A to be detected, and the neighborhood size is , is a constant and is selected according to the application background; in this embodiment, takes 17, and the correspondence offset is constrained within ±8 pixels.

[0064] A2. Perform correlation operations on the feature vectors in the full-size feature map of the pixel point at the position (i, j) and the feature vectors in the full-size feature map of all the pixel points found in step A1, and the matrix composed of all the correlation operation results corresponding to the pixel point at the position (i, j) is used as the correlation sub ;

[0065] A3. Execute steps A1 and A2 for the pixel points at the remaining positions in the image B to be detected to obtain the correlation sub-volumes corresponding to the pixel points at all positions in the image B to be detected

[0066] A4. Use the relevant sub-volumes corresponding to the pixel points at all positions in the image B to be detected to form the feature correlation volume V that inputs the corresponding uncertainty feature extraction part and the corresponding relationship estimation part.

[0067] When implementing, the method preferably adopted by the present solution to obtain the corresponding uncertainty features by the corresponding uncertainty feature extraction part includes:

[0068] Expand the feature correlation volume V to 5 dimensions , and input it into the corresponding uncertainty feature extraction part composed of a multi-level convolutional network;

[0069] Dimension The feature correlation volume V of... dimensions passes through several convolutional layers of the corresponding uncertainty feature extraction part and then outputs features of dimension ;... is the dimension of the corresponding uncertainty feature. In this embodiment, take 64;

[0070] Remove the middle two dimensions with dimension 1 of the features of dimension to reconstruct them into the corresponding uncertainty features of ; h and w are the heights and widths of the images A and B to be detected respectively. As shown, in the four sequentially connected convolutional layers of the uncertainty feature extraction part, the convolutional kernel of convolutional layer 1 is Figure 4 , Stride is 2, Pad is 0, the number of channels is 8, and the output dimension is ; the convolutional kernel of convolutional layer 2 is , Stride is 1, Pad is 0, the number of channels is 16, and the output dimension is ;

[0071] The convolutional kernel of convolutional layer 3 is , Stride is 1, Pad is 0, the number of channels is 32, and the output dimension is ; the convolutional kernel of convolutional layer 4 is , Stride is 1, Pad is 0, the number of channels is 64, and the output dimension is .

[0072] When implementing, the method preferably adopted by the present solution to obtain the corresponding relationship between the two images A and B to be detected by the corresponding relationship estimation part includes:

[0073] Flatten the last two dimensions of the feature correlation volume to obtain a 3D tensor with dimension ; h and w are the heights and widths of the images A and B to be detected respectively; ​​

[0074] Use a multi-level convolutional neural network with residual connections for the correspondence estimation part of the tensor to obtain a correspondence of dimension : :

[0075] For any pixel B in the image B to be detected The corresponding relationship of the corresponding pixel point in the image A to be detected is , is the position of the corresponding pixel point of B in the image A to be detected;

[0076] As Figure 5 shown, the correspondence estimation part includes a convolutional layer C0, a Relu layer R0, a convolutional + Relu layer C1, and a convolutional layer C2 connected in sequence. The output of the convolutional layer C0 is superimposed with the output of the convolutional layer C2 after passing through the residual connection layer S0 to obtain a superimposed feature . The superimposed feature is input into a Relu layer R1, a convolutional + Relu layer C3, and a convolutional layer C4 connected in sequence; the superimposed feature is superimposed with the output of the convolutional layer C4 after passing through the residual connection layer S1 to obtain a superimposed feature . The superimposed feature passes through the Relu layer R2 and the convolutional layer C5 to output the correspondence.

[0077] The convolutional kernels of the convolutional layer C0, the convolutional layer C2, the convolutional layer C4, and the convolutional layer C5 are , with a Stride of 1 and a Padding of 1; the convolutional kernels of the convolutional + Relu layer C1 and the convolutional + Relu layer are , with a Stride of 1 and a Padding of 1; the convolutional kernels of the residual connection layer S0 and the residual connection layer S1 are , with a Stride of 1 and a Padding of 0.

[0078] In step S3, input the correspondence M and the correspondence uncertainty feature F P into the probability regression part of the deep belief correspondence estimation network to obtain the parameters of the correspondence uncertainty probability model; then, according to the parameters of the correspondence uncertainty probability model, calculate the confidence map T of the correspondence M between the image A to be detected and the image B to be detected.

[0079] In an embodiment of the present invention, step S3 further includes:

[0080] S31. Stack the correspondence M and the correspondence uncertainty feature F P and then input them into the probability regression part with a multi-level convolutional neural network as the architecture;

[0081] S32. The stacked features pass through multiple convolutional layers to obtain the parameters of the mixture probability model , where are the unnormalized weights of the two independent probability models in the mixture probability model, , are the variances of the two independent probability models in the mixture probability model respectively; for any pixel in the image B to be detected, the corresponding mixture probability model parameters are , , , and the elements of the corresponding relationship M are , taken as the mean value of the probability density function , that is ;

[0082] S33. The unnormalized model weights , pass through the softmax layer to obtain the corresponding mixture probability model The normalized model weights , , where is a random variable, representing the possible values of the corresponding relationship between the two images to be detected at the pixel point in the image B to be detected, and it follows the mixture probability density distribution: , where is the independent probability density function, , are the probability distribution parameters;

[0083] S34. According to the probability distribution parameters of the model, calculate the confidence of the corresponding relationship between the two images to be detected at the pixel point in the image B to be detected:

[0084]

[0085] Among them, is a real number, and its value is determined according to the application background; is , the Euclidean distance between.

[0086] As Figure 6 shown, C0 and C1 in the probability regression network are convolutional + Relu layers, and their convolutional kernels are , Stride is 1, and Padding is 1; C2 is a convolutional layer, and its convolutional kernel is , Stride is 1 and Padding is 1.

[0087] When the probability density function adopts the Laplace distribution, for the pixel point at the position in the image B to be detected, the corresponding relationship between the two images to be detected has the following confidence expression:

[0088]

[0089] where k is a variable with values of 1 and 2, and it is an exponential function.

[0090] In step S4, the corresponding uncertainty feature F P and the feature after stacking the confidence map T and the pyramid feature of the image B to be detected are input into the change detection network, and after being processed by the change detection network, the change detection result between the image A to be detected and the image B to be detected is obtained.

[0091] The detailed implementation process of step S4 is as follows: The change detection network includes two parts: the hybrid feature integration SN 21 and the change region segmentation network SN 22 . SN 21 stacks the corresponding uncertainty feature F P and the confidence map T, and then, the stacked feature and the pyramid feature of the image B to be detected are input into the change region segmentation network SN 22 , and through inference calculation, the change region of the image B to be detected relative to the image A to be detected is obtained, thereby obtaining the change detection result.

[0092] The above change region segmentation network SN 22 is a semantic segmentation network composed of a multi-level convolutional neural network. The input of this network is the stacked feature and the pyramid feature of the image B to be detected , and it outputs the probability that each pixel in the image B to be detected belongs to the change region. In this embodiment, the change detection network adopts a U-Net-like architecture, and the network structure is as Figure 7 shown. It adopts an encoder-decoder structure. During the process of the encoder gradually decreasing, corresponding image pyramid features will be introduced according to the scale size of the I-V level image features in Figure 3 (the I-V level image features shown in ); at the end of the decoder, the probability value that each pixel in the image to be detected belongs to the change region is obtained through Figure 7 the convolutional layer. The number of channels of each convolutional layer in the network is as

[0093] During implementation, the loss function of the deep belief correspondence estimation network is preferably expressed as:

[0094]

[0095] where are network parameters; is the training set for training the deep belief correspondence estimation network, , X is the input image pair composed of image A and image B, is the true correspondence of X; N is the total number of input image pairs; and are both the normalized independent probability density model weights corresponding to the estimated image B at; and are both the probability model variances corresponding to the estimated image B at; is the correspondence of the estimated image B at; is the true correspondence corresponding to the pixel in.

[0096] When the probability density function in the loss function adopts the Laplace probability density distribution, the expression of the loss function is:

[0097]

[0098] where and are both the unnormalized independent probability model weights corresponding to the estimated image B at; k is a variable, equal to 1 or 2; e is the natural logarithm; is the 1-norm.

[0099] The expression of the loss function of the change detection network is:

[0100]

[0101] where is the training set for training the change detection network, , For the input images, the pyramid image features, uncertainty features, and confidence of the correspondence for images A and B, G is the change region label map; N is the total number of input image pairs For the image in the nth image pair at the pixel value is the change region label value, with a change being 1 and no change being 0; is the probability that the pixel at in image B of the image pair predicted by the network is a change region; is the weight corresponding to the nth input image pair, equal to the number of pixels in the sample that do not belong to the change region divided by the number of pixels that belong to the change region.

[0102] In summary, this solution organically integrates image correspondence estimation, correspondence confidence estimation, and change detection, unifying image registration and change detection into a single network framework, while improving the effect of image change detection.

Claims

1. A method for detecting image changes in a deep neural network that integrates confidence correspondence estimation, characterized in that, This method is implemented by using two cascaded deep neural networks, namely a deep belief correspondence estimation network and a change detection network. The method includes the following steps: S1. Input the image A to be detected and the image B to be detected into the feature extraction part of the deep belief correspondence estimation network for feature extraction, and obtain the full-size feature map of the image A to be detected and the pyramid features of the image B to be detected , the full-size feature map ; S2. For the full-size feature map and the full-size feature map After performing relevant operations, the feature correlation volume V is obtained. The feature correlation volume V is respectively input into the correspondence estimation part and the corresponding uncertainty feature extraction part of the deep belief correspondence estimation network to obtain the correspondence M and the corresponding uncertainty feature F between the image A to be detected and the image B to be detected P ; S3. Input the correspondence M and the corresponding uncertainty feature F P into the probability regression part of the deep belief correspondence estimation network to obtain the parameters of the corresponding uncertainty probability model. Then, according to the parameters of the corresponding uncertainty probability model, calculate the confidence map T of the correspondence M between the image A to be detected and the image B to be detected; S4. Stack the features corresponding to the uncertainty feature F P and the confidence map T, and the pyramid features of the image B to be detected and input them into the change detection network. After processing by the change detection network, the change detection result between the image A to be detected and the image B to be detected is obtained; The method for performing correlation operations on the image features of two images to be detected includes: A1. For the pixel at any position (i, j) in the image B to be detected, find all the pixels in the neighborhood of the pixel at the corresponding position (i, j) in the image A to be detected. The neighborhood size is , is a constant; A2. The full-size feature map of the pixel at position (i, j) and the feature vectors in the full-size feature maps of all the pixels found in step A1 are subjected to a correlation operation. The matrix formed by all the correlation operation results corresponding to the pixel at position (i, j) is used as the correlation sub-volume ; A3. Execute steps A1 and A2 for the pixel points at the remaining positions in the image to be detected B, and obtain the correlation sub-volumes corresponding to the pixel points at all positions in the image to be detected B. A4. Use the correlation sub-volumes corresponding to the pixel points at all positions in the image to be detected B to form the feature correlation volume V input to the uncertainty feature extraction part and the correspondence estimation part.

2. The method for detecting image changes in a deep neural network according to claim 1, wherein The method for obtaining the corresponding uncertainty features by using the corresponding uncertainty feature extraction part includes: Expand the feature-related body V to five dimensions , and input it into the corresponding uncertainty feature extraction part composed of a multi-level convolutional network; Dimension The feature-related body V of outputs features with a dimension of after passing through several convolutional layers in the corresponding uncertainty feature extraction part; Remove the middle two dimensions of dimension 1 of the feature with dimension so that it is reconstructed into corresponding uncertainty feature of ; h and w are the heights and widths of the images A and B to be detected, respectively.

3. The method for detecting image changes in a deep neural network according to claim 1, wherein The method for obtaining the correspondence between two images to be detected A and B by using the correspondence estimation part includes: Flatten the last two dimensions of the feature-related body to obtain a 3D tensor with a dimension of ; h and w are the heights and widths of the images A and B to be detected, respectively. The 3D tensor is passed through a correspondence estimation part composed of a multi-level convolutional neural network with residual connections to obtain a correspondence of dimension : ​ Any pixel B in the image B to be detected The corresponding relationship of the homonymous pixel point in the image A to be detected is , is B The position of the homonymous pixel point of B in the image A to be detected; The multi-level convolutional neural network with residual connections in the corresponding relationship estimation part includes a convolutional layer C0, a Relu layer R0, a convolutional + Relu layer C1, and a convolutional layer C2 connected in sequence. The output of the convolutional layer C0 is superimposed with the output of the convolutional layer C2 after passing through the residual connection layer S0 to obtain a superimposed feature , the superimposed feature is input into a Relu layer R1, a convolutional + Relu layer C3, and a convolutional layer C4 connected in sequence; the superimposed feature is superimposed with the output of the convolutional layer C4 after passing through the residual connection layer S1 to obtain a superimposed feature , the superimposed feature is output after passing through the Relu layer R2 and the convolutional layer C5 to obtain the corresponding relationship .

4. The method for detecting image changes in a deep neural network according to claim 3, wherein Step S3 further includes: S31. Stack the correspondence M and the corresponding uncertainty feature F P and input them into the probability regression part with a multi-level convolutional neural network as the architecture; S32. After the stacked features pass through multiple convolutional layers, the parameters of the mixture probability model are obtained , where are the unnormalized weights of the two independent probability models in the mixture probability model, , are the variances of the two independent probability models in the mixture probability model respectively; for any pixel in the image B to be detected, its corresponding mixture probability model parameters are , , , and the element in the corresponding relationship M corresponding to it is taken as the mean value of the probability density function, that is ; S33, Unnormalized weights of two independent probability models , After passing through the softmax layer, the corresponding mixture probability model is obtained Normalized model weights , , where is a random variable representing the possible values of the correspondence between two images to be detected at the pixel point in the image B to be detected, and it follows a mixture probability density distribution: , where is an independent probability density function, , is a probability distribution parameter; S34. Calculate, according to the probability distribution parameters of the model , the correspondence between two images to be detected at the position of the pixel point in the image B to be detected, and the confidence level is : Among them, is a real number and takes values according to the application background; is and the Euclidean distance between them.

5. The method for detecting image changes in a deep neural network according to claim 4, characterized in that, When the probability density function adopts the Laplace distribution, the corresponding relationship between two images to be detected at the pixel point position in the image B to be detected confidence The expression is: where k is a variable with values of 1 and 2, is an exponential function.

6. The method for detecting image changes in a deep neural network according to any one of claims 1-5, characterized in that The feature extraction part uses VGG-16 as the backbone network; when extracting features, the outputs of the convolutional layers before each pooling layer in the VGG-16 backbone network are taken to form the pyramid features of the image A to be detected and the image B to be detected and the pyramid features , which contain level I-V image features; then, they are upsampled by the nearest neighbor interpolation method to the same height and width as the input image and stacked to form a feature map with a dimension of , and the full-size feature maps of the image A to be detected and the image B to be detected are obtained respectively and ; after the image A to be detected undergoes feature extraction, the pyramid feature and the full-size feature map are obtained; after the image B to be detected undergoes feature extraction, the pyramid feature and the full-size feature map are obtained; h and w are the heights and widths of the images A and B to be detected respectively.

7. The method for detecting image changes in a deep neural network according to any one of claims 1-5, characterized in that, Loss function of the deep belief correspondence estimation network The expression is as follows: Among them, is a network parameter; is a training set for training the deep belief correspondence estimation network, , where X is an input image pair composed of image A and image B, is the true correspondence of X; N is the total number of input image pairs; is an independent probability density function; and are both the normalized independent probability density model weights corresponding to the position of image B estimated from the nth input image pair ; and are both the variances of the independent probability density model corresponding to the position of image B estimated from the nth input image pair ; is the correspondence of image B estimated from the nth input image pair at the position of; is the true correspondence corresponding to the pixel in; 8. The method for detecting image changes in a deep neural network according to claim 6, wherein The change detection network in step S4 adopts a U-Net-like architecture, which uses an encoder-decoder structure; during the process of gradually reducing the encoder, pyramid features of the to-be-detected image B are introduced according to the scale size. The image features of levels I-V in are obtained; at the end of the decoder, the probability value that each pixel in the to-be-detected image B belongs to the changed area is obtained through the convolutional layer of 9. The method for detecting image changes in a deep neural network according to any one of claims 1-5, characterized in that, The loss function of the change detection network is expressed as: Among them, is the training set for training the change detection network, , is the mixed feature composed of the pyramid image features, uncertainty features and confidence of the correspondence of the input image pair for image A and image B, G is the change region label map; N is the total number of input image pairs; is the pixel at the of image B in the nth image pair, where the change region label value is 1 for change and 0 for no change; is the probability value of the pixel at the of image B in the image pair predicted by the network; is the weight corresponding to the nth input image pair, which is equal to the number of pixels in the sample that do not belong to the change region divided by the number of pixels that belong to the change region.

Images