Non-aligned heterogeneous remote sensing image change detection method based on image transformation and optical flow

Abstract

The invention relates to a non-aligned heterogeneous remote sensing image change detection method based on image transformation and optical flow. The method comprises the following steps: acquiring different-source images at different moments; the method comprises the following steps: transforming an image before an event into an image domain after the event, dividing the image to obtain a block group matrix, and constructing a structure consistency regular term by taking image blocks as nodes; and decomposing the transformed image, registering and calculating an optical flow vector by using an optical flow method, introducing a regular term to construct an image decomposition model, introducing a Lagrange multiplier to solve a minimization model, and solving to obtain a multi-scale difference image. And finally, fusion segmentation based on a Markov random field is used to obtain a final change graph. By adopting the method, the heterogenous change detection performance under the registration error condition can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image detection, and particularly relates to a non-aligned multi-source remote sensing image change detection method based on image transformation and optical flow. BACKGROUND

[0002] Multi-source change detection refers to a technology of extracting ground cover change information by comparing multiple images of the same area acquired by different sensors at different times. With the continuous development of remote sensing related technologies, multi-source change detection has become a research hotspot that is increasingly concerned, and it can break through the limitation of traditional multi-temporal change detection on multi-temporal remote sensing images from the same sensor. Multi-source change detection is particularly crucial in emergency response of natural disasters. Due to the limited imaging conditions, such as the adverse weather conditions such as heavy rain, cloud, thick fog, etc. often accompanied by disasters, and the limitation of satellite revisit period, it is difficult to obtain multi-temporal images from the same sensor in time. In this case, multi-source change detection can use post-disaster emergency acquired SAR images and pre-disaster optical images to realize rapid detection and disaster analysis of the disaster area.

[0003] Since the images used in multi-source change detection come from different sensors and have different imaging conditions, the expression of the same ground object in multi-source images often has significant differences, which brings great challenges to multi-source change detection.

[0004] Firstly, the heterogeneity between multi-temporal images makes it impossible to directly compare the differences between images to extract change information as in the case of homogenous change detection. Therefore, the core problem of heterogenous change detection is how to establish a comparable relationship between images, which is the focus of current research. According to the way of establishing image comparability, the existing methods of heterogenous change detection can be roughly divided into the following five categories: methods based on image classification - dividing the heterogenous images into the same land cover categories respectively, and comparing the classification results to determine the changes. Methods based on similarity measurement - usually assuming that there is some kind of relationship between the heterogenous images that is independent of the imaging conditions, and constructing a similarity measurement function based on this to identify the change area. Methods based on deep feature transformation - by using labeled data to train twin or pseudo-twin networks, the heterogenous images are mapped into a shared latent feature space for comparison. Methods based on image translation - using traditional regression functions or generative adversarial networks and their variants to convert one image to another image domain, and through style transfer to realize image isomorphism, thereby realizing change detection. Methods based on energy model - generally using Markov random field (MRF) to model the problem of heterogenous change detection, and by constructing reasonable constraints to flexibly transform the change label assignment task into an energy minimization problem. Secondly, the significant differences in imaging conditions between multi-temporal images also bring great challenges to the accurate registration of heterogenous images. It is well known that image registration is a key preprocessing step in change detection. In previous studies, it is usually assumed that the multi-temporal images to be compared have been accurately registered, i.e. the pixels at the same position in the image correspond to the same geographical area. At the same time, image registration and change detection are usually considered as two independent research tasks.

[0005] Currently, the registration technique of homologous images is relatively mature, which makes the influence of registration error often ignored in homologous change detection tasks. However, for the heterologous images from different sensors and imaging conditions, it is a challenging task to achieve high-precision registration itself. At present, a large number of studies are still focused on improving the accuracy of heterologous image registration, especially in the heterologous images involving large range, high resolution and complex scene, it is still a major problem to achieve pixel-level matching accuracy. Therefore, it is particularly important to consider the influence of registration error in the heterologous change detection task. Unfortunately, the current research of most heterologous change detection methods mainly focuses on the first aspect, that is, making the heterologous images comparable, and often ignores a practical problem: residual registration error, which may seriously affect the final change detection performance. At present, only a small number of studies attempt to consider the change detection and image registration tasks jointly, but these studies only focus on homologous change detection and do not involve more complex heterologous change detection. For example, in the homologous change detection of SAR images, a joint change detection and registration method based on sparsity is proposed, which determines the optimal matching by constructing an image block dictionary and using sparse representation, and uses the reconstruction error as the criterion of change. An optimization framework is studied, which decomposes the observed images into three parts: registration images based on affine transformation, change images based on sparse representation and image noise. For the homologous change detection of optical images, an energy model based on Markov random field (MRF) is proposed, which includes three energy terms: change detection, registration and coupling. For optical aerial images with registration error, an end-to-end network is trained to achieve joint processing by extracting general features that can serve both change detection and image registration. On the other hand, some researchers found that most deep features extracted by networks for change detection and image registration have high cosine similarity, so they proposed a unified network to perform change detection and image registration tasks simultaneously, aiming to reduce training and memory overhead. However, this network does not consider the change detection problem under registration error, nor does it involve the processing of heterologous images. Obviously, the existing research mainly focuses on the homologous change detection task with registration error, and still ignores the more challenging problem of heterologous change detection under registration error. SUMMARY

[0006] Therefore, it is necessary to provide a non-aligned heterologous remote sensing image change detection method based on image transformation and optical flow, which can improve the change detection performance under registration error.

[0007] A non-aligned heterologous remote sensing image change detection method based on image transformation and optical flow, the method comprising: acquiring heterologous images at different times; the heterologous images include pre-event images and post-event images; transforming the pre-event image into a domain of the post-event image to obtain a transformed image; performing image partition on the pre-event image and the transformed image to obtain corresponding image block group matrices; constructing a probability graph as a graph structure with each image block in the image block matrix as a graph node, and setting a regularization term of structural consistency according to a relationship between the images based on the graph structure; decomposing the transformed image into a registered image and a change map, and performing registration on the registered image and the post-event image and calculating an optical flow vector in the registration process according to an optical flow method; introducing the regularization term of structural consistency, the optical flow vector and a preset prior sparsity regularization term into a decomposition process of the registered image to construct an image decomposition model, introducing a Lagrange multiplier into the image decomposition model to obtain a minimization model, and solving the minimization model to obtain a multi-scale difference map; performing fusion segmentation on the multi-scale difference map by using a fusion segmentation method based on a Markov random field to obtain a final change map.

[0008] The above-mentioned non-aligned multi-source remote sensing image change detection method based on image transformation and optical flow combines the image transformation of the graph model and the Lucas-Kanade optical flow method, the former captures the structural correlation by constructing a probability graph of image blocks, and the latter dynamically calculates the optical flow vector to track the pixel motion, and the two realize the unsupervised change detection of multi-source images under the condition of registration error. The method improves the practical applicability of multi-source change detection, because pixel-level accurate registration cannot always be achieved in practical applications. The image transformation, registration and change estimation are integrated into a unified decomposition model, the structural consistency, optical flow and change sparsity constraints are introduced to form a collaborative network: the structural constraint retains the feature of the ground object, the optical flow constraint corrects the interference of the registration error on the corresponding pixels, and the change sparsity constraint anchors the change region, promotes the sub-tasks from discrete processing to collaborative optimization, and weakens the error influence. The multi-scale coarse-to-fine strategy and the MRF fusion segmentation are adopted, so that the Lucas-Kanade optical flow method can effectively handle the large displacement problem, realize the gradual refinement of the matching and change detection results, and further improve the performance of multi-source change detection by fusing the multi-scale change images. BRIEF DESCRIPTION OF DRAWINGS

[0009] Figure 1 It is a flowchart of the non-aligned multi-source remote sensing image change detection method based on image transformation and optical flow in one embodiment; Figure 2 It is a framework flowchart of the non-aligned multi-source remote sensing image change detection method based on image transformation and optical flow in one embodiment; Figure 3 It is a flowchart of the multi-scale strategy solving in one embodiment. DETAILED DESCRIPTION

[0010] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0011] In one embodiment, Figure 1 As shown, a non-aligned heterogeneous remote sensing image change detection method based on image transformation and optical flow is provided, comprising the following steps: Step 102, obtaining heterogeneous images at different moments; the heterogeneous images include pre-event images and post-event images; transforming the pre-event image into the domain of the post-event image to obtain a transformed image; performing image partitioning on the pre-event image and the post-event image to obtain a corresponding image block group matrix; constructing a probability graph as a graph structure with each image block in the image block matrix as a graph node, and setting a regularization term for structural consistency based on the relationship between images characterized by the graph structure.

[0012] Given a pair of and Acquired heterogeneous images, pre-event images belong Domain, post-event image belong domain, where 、 、 and Respectively and By utilizing the geographic coordinate reference inherent in remote sensing images, the heterogeneous images have been roughly aligned without large translation, rotation or scale differences. Figure 2 shown.

[0013] In change detection tasks, multi-temporal images must be comparable and spatially aligned within the same image or feature domain. In optical flow-based image registration, multi-temporal images must satisfy the assumptions of constant brightness and small, smooth motion. However, in heterogeneous change detection tasks, both assumptions are violated: multi-temporal images are acquired by different sensors under different acquisition conditions, and there are drastic changes between images. To meet these constraints, the joint framework proposed in this application uses an image decomposition model as its foundation.

[0014] (1) GiToF will match the after-the-fact image Decompose into unchanged transformed images and the changing images caused by events ,Right now , and three regularization terms are introduced for this decomposition: 1) Regularization term based on structural consistency , used to constrain the unchanged transformed image With the original pre-image Maintain the same topology in the similarity graph model; 2) Regularization term based on Lucas-Kanade optical flow , used to solve the displacement between the original image and the matched subsequent image ,satisfy ; 3) Regularization term based on prior sparsity , which is based on the fact that only a small number of areas in real scenes change, while most areas remain unchanged.

[0015] First, transform the pre-image into the post-image domain, which is defined as ,in represents the image transformation function. The basic idea behind image transformation methods based on structural consistency is that when imaging the same scene using different sensor types, although the resulting images may differ significantly in appearance, the topological relationships between objects in the images generally remain consistent. Structural consistency constraints can be implemented by constructing a graphical model that represents the structure of the original image and ensuring that the transformed image retains the same structural features within this graphical model.

[0016] First, the pre-image Divide into sizes Small square non-overlapping image patches are used to fill in the parts that cannot be divided by the edges. Then, these image patches are vectorized and stacked into an image patch group matrix, denoted as ,in , Indicates rounding up. For the transformed image , the same operation is also used to divide it into image blocks and obtain the corresponding image block group matrix, recorded as The operation of generating a block matrix from an image is denoted as , and the operation of restoring the image block matrix to the image is recorded as .

[0017] Constructing a probability plot To depict the prior image The structural features of each image block are as a graph node, denoted as , and through the edge weight Connect it with the adjacent image blocks.

[0018] (2) in, Indicates the first image blocks and the The distance between image blocks, is a tuning parameter used to adjust the number of neighbors. The closed-form optimal solution to problem (2) is: (3) in, Represents a vector Middle Small distance value, symbol Indicates the The index of the small value. It can be found that the figure It is actually a The k-nearest neighbor (KNN) graph model.

[0019] Pre-image With the transformed image The structural consistency between them requires that they share the same topological structure relationship. That is, if the graph The two blocks connected in and Very similar (corresponding edge weights is very large), then transform the image The corresponding block and should also be very similar. Therefore, we can get the following regularization term: (4) This is similar to the first term in the graph construction model (2). Define the graph Laplacian matrix as ,in is a diagonal matrix with diagonal elements , the regularization term (4) based on structural consistency can be rewritten as: (5) in, represents the trace of the matrix.

[0020] Step 104 : decompose the transformed image into a registered image and a change map, register the registered image and the post-event image according to an optical flow method, and calculate the optical flow vector in the registration process.

[0021] In optical flow based image registration tasks, multi-temporal images are required to satisfy the brightness constancy assumption and small and smooth motion assumption, but due to the heterogeneity of images and the emergence of changes, both assumptions cannot be met in the heterogeneous change detection task. Therefore, in the proposed GiToF method, first, the event pre-image is converted to , and then is decomposed into an accurately registered image and a change map , i.e. . Subsequently, registration is performed between and , denoted as , where represents displacement. Then, for each pixel, there is: (6) where and represent the horizontal and vertical components of the optical flow, respectively. Assuming that the displacement is small, the local Taylor series expansion is used, and is simplified as to simplify the notation, resulting in: (7) where and represent partial derivatives, specifically and .

[0022] The Lucas-Kanade method assumes that the displacement approximately remains unchanged within the neighborhood and calculates the optical flow vector by solving the following minimization problem using the least squares method principle: (8) where represents a standard Gaussian template of size , . By setting and , we obtain: (9) where (10) Step 106, introduce the regularization term of structural consistency, the optical flow vector, and the pre-set prior sparsity regularization term into the decomposition process of the registered image to construct an image decomposition model; introduce the Lagrange multiplier into the image decomposition model to obtain a minimization model; solve the minimization model to obtain a multi-scale difference map.

[0023] In real change detection problems, there is a sparsity prior, i.e., only a small portion of the area changes while the majority of the area remains unchanged in real scenarios. This prior sparsity is widely used in other change detection methods. Intuitively, the norm of the difference between the two images, i.e., the number of changed pixels, should be chosen as the sparsity penalty function. However, considering the non-convexity of the norm, its convex relaxation form, i.e., the norm regularization, is used instead, denoted as: (11) When , degenerates to the norm regularization.

[0024] By introducing the regularization terms (5), (8), (11), and substituting them into equation (1), while introducing the Lagrange multiplier , the augmented Lagrangian function is obtained as (12) where is the penalty parameter, and are the image block group matrices corresponding to , which can be converted to each other by and . The Alternating Direction Method of Multipliers (ADMM) is adopted to solve the minimization model (12). Given the current value of the $k$th iteration, the following five sub-problems can be obtained.

[0025] 1) The first sub-problem can be written as (13) which has a closed-form solution as follows (14) where denotes the identity matrix.

[0026] 2) The second sub-problem can be written as (15) and its closed-form solution is (16) i.e., for each pixel, we have (17) 3)​ -Subproblem, based on the Lucas-Kanade optical flow method, You can do this by Substitute into formula (9) and solve it.

[0027] 4) -subproblem, which can be written as (18) in, . The closed-form solution is as follows (19) 5) - subproblem, the Lagrange multiplier can be updated by the following formula (20) GiToF uses The multi-scale strategy from coarse to fine at each level brings four key advantages: First, this strategy enables the Lucas-Kanade optical flow method to effectively deal with large displacement problems; Second, it exploits the fact that change detection is relatively robust to registration errors at coarse scales, making initialization easier and more accurate; Third, the strategy can gradually refine the results of registration, image transformation, and change detection; Fourth, it can provide multi-scale difference maps, which can be fused to obtain more accurate change detection results.

[0028] Specifically, GiToF first constructs a pyramid structure of the downsampled image, then solves the minimization problem of Equation (12) at a coarser level to roughly estimate the displacement, transformed image, and difference map, and then gradually refines these results at a finer scale, such as Figure 3 shown.

[0029] Step 108 : Using a Markov random field-based fusion segmentation method to fuse and segment the multi-scale difference map to obtain a final change map.

[0030] The multi-scale difference map is obtained by minimizing the model (12) , you can get a multi-scale difference map, which is calculated as follows: (twenty one) This application adopts a fusion segmentation method based on Markov random field (MRF) to replace the traditional "fusion and segmentation" strategy. In order to reduce the influence of salt and pepper noise, superpixels are used as the basic unit in the change map. The method is to and the registered post-event image Joint segmentation is performed to obtain superpixels and the corresponding joint segmentation map The problem of computing the binary change map can be viewed as a label assignment problem for superpixels, i.e., for each superpixel , the label indicates no change, indicates change. According to the theory of Markov Random Fields, this label assignment problem can be modeled as an energy minimization problem: (22) where and denote the change energy function and the spatial energy function, respectively, is a weight parameter.

[0031] To construct the change energy function , first upsample the multi-scale difference maps , to the size of , then compute the average change intensity of the th superpixel at the th layer scale, which is specified as follows: (23) Then, normalize the change intensity to , where is the threshold obtained by applying the Otsu thresholding method to at the th layer scale. Define the change energy function as follows: (24) where is the (25) where is a parameter that controls the rate of imbalance penalty and can consider the imbalance between the changed and unchanged classes in the heterogeneous change detection problem. For the function , directly adopt the common spatial constraint form to encourage spatially neighboring superpixels to be assigned the same label, so as to be able to fuse the context information of each superpixel. With the help of these energy terms, the model (22) can be efficiently solved by the graph cut algorithm to obtain the final change map.

[0032] The GiToF proposed in this application assumes that the registered post-event image can be decomposed into an unchanged image and a changed image, and introduces three regularization terms to the decomposition: using structural consistency to ensure that the unchanged image maintains the same structure as the original pre-event image in terms of graph structure; imposing Lucas-Kanade optical flow constraints between the original and registered post-event images; and limiting the sparsity of the changed image through prior sparsity constraints. With the help of this joint model, GiToF helps the image transformation process overcome the influence of registration errors and changed areas, while assisting the optical flow method to meet the brightness consistency assumption and the small and smooth motion assumption. By solving this joint model, a multi-scale change map can be obtained, and then the final change detection result is obtained by using fusion segmentation based on Markov random fields, and the effectiveness of this application is verified by experimental results.

[0033] This application proposes a change detection method for non-aligned heterogeneous remote sensing images based on image transformation and optical flow. By combining graph-based image transformation with the Lucas-Kanade optical flow method, the former captures structural associations by constructing a probability graph from image patches, while the latter dynamically calculates optical flow vectors to track pixel motion. This method achieves unsupervised change detection for heterogeneous images in the presence of registration errors. This method improves the practical applicability of heterogeneous change detection, as pixel-level accurate registration is not always possible in practical applications. By integrating image transformation, registration, and change estimation into a unified decomposition model, structural consistency, optical flow, and change sparsity constraints are introduced to form a collaborative network: the structural constraint preserves ground feature features, the optical flow constraint corrects the interference of registration errors on pixel correspondences, and the change sparsity constraint anchors the changed regions, moving the subtasks from discrete processing to collaborative optimization and mitigating the impact of errors. By employing a multi-scale coarse-to-fine strategy and MRF fusion segmentation, the Lucas-Kanade optical flow method can effectively handle large displacements, achieving progressive refinement of matching and change detection results. By fusing multi-scale change images, the performance of heterogeneous change detection is further improved.

[0034] In one embodiment, constructing a probability graph as a graph structure with each image block in the image block matrix as a graph node includes: Constructing a probability plot Describing the image before the event The structural features of each image block are as a graph node, denoted as , and through the edge weight Connect it with the adjacent image blocks; image blocks and the The edge weight between image patches is

[0035] in, Represents a vector the first small distance value, the symbol the first index of the small value, the vector the first small distance value, the number of neighbors of each vertex in the probability graph, the neighbor number of each vertex in the probability graph, the vector the first small distance value.

[0036] In one embodiment, the regularization term of structural consistency is set according to the relationship between images described by the graph structure, including: The regularization term of structural consistency is set according to the relationship between images described by the graph structure as follows:

[0037] Also can be written as:

[0038] Wherein, denotes the transformed image, denotes the pre-event image, denotes the total number of image blocks, denotes the image block of the transformed image, and both denote the serial number of the image block, denotes the edge weight between the first image block and the first image block in the pre-event image.

[0039] In one embodiment, the registered image and the post-event image are registered according to the optical flow method, and the optical flow vector in the registration process is calculated, including: The registered image and the post-event image are registered, denoted as , wherein denotes the displacement, and denote the horizontal and vertical optical flow components, respectively; The optical flow vector in the registration process is calculated as follows:

[0040] Wherein, denotes the number of channels of the post-event image, denotes the channel serial number, denotes the size The standard Gaussian template, represents the optical flow component in the horizontal direction, represents the horizontal partial derivative of the image after the event, represents the optical flow component in the vertical direction, represents the pixel difference between the original image and the registered image, represents the vertical partial derivative of the image after the event, represents the pixel value of the image after the original event, Indicates the value of the image after the matching event, Represents different pixel coordinates.

[0041] In one embodiment, a structural consistency regularization term, an optical flow vector, and a preset prior sparsity regularization term are introduced into the decomposition process of the registered image to construct an image decomposition model, including: The regularization term of structural consistency, optical flow vector and pre-set prior sparsity regularization term are introduced into the decomposition process of the registered image to construct the image decomposition model:

[0042] in, represents the pre-set prior sparsity regularization term, Represents the image after registration , Represents a change graph, represents the transformed image, Represents a post-event image.

[0043] In one embodiment, Lagrange multipliers are introduced into the image decomposition model to obtain a minimization model, including: Introducing the Lagrange multiplier into the image decomposition model, the minimization model is obtained as follows:

[0044] in, , is the angle matrix, T represents the transpose operation, is the penalty parameter, They correspond to The image block matrix, represents the Lagrange multiplier.

[0045] In one embodiment, solving the minimization model to obtain a multi-scale difference map includes: Use A coarse-to-fine multi-scale strategy is adopted to construct a pyramid structure of the downsampled image. The ADMM algorithm is used to solve the minimization model at each scale to obtain a multi-scale difference map.

[0046] In one embodiment, a Markov random field-based fusion segmentation method is used to fuse and segment the multi-scale difference map to obtain a final change map, including: Taking superpixels as the basic unit in the change map, the pre-event image and the registered image Perform joint segmentation to obtain superpixels and the corresponding joint segmentation map ; The computation of the change map can be viewed as a superpixel label assignment problem, that is, for each superpixel ,Label Indicates no change. Indicates a change; After upsampling the multi-scale difference map, the Layer scale The average change intensity of superpixels; the change energy function is defined according to the average change intensity; According to the Markov random field theory, the label assignment problem is modeled as an energy minimization problem based on the variable energy function:

[0047] in, and represent the variation energy function and the spatial energy function respectively, is the weight parameter; The energy problem is solved by the graph cut algorithm to obtain the final change graph.

[0048] In one embodiment, a change energy function is defined according to the average change intensity, including: The change energy function is defined according to the average change intensity:

[0049] in, Indicates the Layer scale Superpixels The average change intensity of represents a multi-scale difference map, is a defined function, a and b represent the variables of the function, is a parameter that controls the imbalance penalty rate, and L represents the scale of the difference map.

[0050] It should be understood that althoughFigure 1 The steps in the flowcharts of the above embodiments are displayed in sequence according to the arrows, but the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the steps are not necessarily executed in strict order, and the steps can be executed in other orders. Moreover, Figure 1 At least a part of the steps in the flowcharts of the above embodiments can include a plurality of sub-steps or a plurality of stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of the sub-steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least a part of other steps or sub-steps or stages of other steps.

[0051] The technical features of the above embodiments can be combined in any manner. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described, but it should be understood that any combination of the technical features is within the scope of the present disclosure as long as there is no contradiction.

[0052] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the present application. It should be noted that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present application, and these are within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A change detection method for non-aligned heterogeneous remote sensing images based on image transformation and optical flow, characterized in that: The method comprises: Acquiring heterogeneous images at different moments; the heterogeneous images include pre-event images and post-event images; The pre-event image is transformed into the domain of the post-event image to obtain a transformed image; the pre-event image and the post-event image are partitioned to obtain a corresponding image block group matrix; a probability graph is constructed as a graph structure using each image block in the image block matrix as a graph node, and a regularization term for structural consistency is set based on the relationship between images described in the graph structure; Decomposing the transformed image into a registered image and a change map, registering the registered image and the post-event image according to an optical flow method and calculating the optical flow vector in the registration process; Introducing the structural consistency regularization term, the optical flow vector, and the preset prior sparsity regularization term into the decomposition process of the registered image to construct an image decomposition model; introducing Lagrange multipliers into the image decomposition model to obtain a minimization model; solving the minimization model to obtain a multi-scale difference map; The multi-scale difference map is fused and segmented by a fusion segmentation method based on Markov random fields to obtain a final change map.

2. The method according to claim 1, wherein A probability graph is constructed as a graph structure using each image block in the image block matrix as a graph node, including: Constructing a probability plot Describing the image before the event The structural features of each image block are as a graph node, denoted as , and through the edge weight Connect it with the adjacent image blocks; image blocks and the The edge weight between image patches is in, Represents a vector Middle Small distance value, symbol Indicates the The index of the small value, Represents a vector Middle Small distance values, represents the number of neighbors of each vertex in the probability graph, represents the neighbor number of each vertex in the probability graph, Represents a vector Middle Small distance value.

3. The method according to claim 1, characterized in that The regularization term for structural consistency is set based on the relationship between images described in the graph structure, including: The regularization term for setting structural consistency based on the relationship between images described in the graph structure is: Can also be written as: in, represents the transformed image, represents the image before the event, represents the total number of image blocks, represents the image patch after the transformation, and Both represent the sequence number of the image block, Indicates the first image blocks and the The edge weights between image patches.

4. The method according to claim 3, characterized in that The registered image and the post-event image are registered according to the optical flow method and the optical flow vector in the registration process is calculated, including: After registration, the image and post-event images The registration is performed between ,in represents displacement, and Represents the optical flow components in the horizontal and vertical directions respectively; The optical flow vector in the registration process is calculated as: in, Indicates the number of channels of the image after the event, Indicates the channel number, Indicates size The standard Gaussian template, represents the optical flow component in the horizontal direction, represents the horizontal partial derivative of the image after the event, represents the optical flow component in the vertical direction, Represents the pixel difference between the original image and the registered image, represents the vertical partial derivative of the image after the event, represents the pixel value of the image after the original event, Indicates the value of the image after the matching event. Represents different pixel coordinates.

5. The method according to claim 4, wherein The structural consistency regularization term, the optical flow vector, and the preset prior sparsity regularization term are introduced into the decomposition process of the registered image to construct an image decomposition model, including: The structural consistency regularization term, optical flow vector and pre-set prior sparsity regularization term are introduced into the decomposition process of the registered image to construct an image decomposition model: in, represents the pre-set prior sparsity regularization term, Represents the image after registration , Represents a change graph, represents the transformed image, Represents a post-event image.

6. The method according to claim 5, characterized in that The Lagrange multiplier is introduced into the image decomposition model to obtain a minimization model, including: Introducing the Lagrange multiplier into the image decomposition model, the minimization model is obtained as follows: in, , is the angle matrix, T represents the transpose operation, is the penalty parameter, They correspond to The image block matrix, represents the Lagrange multiplier.

7. The method according to claim 6, characterized in that Solve the minimization model to obtain a multi-scale difference map, including: Use A coarse-to-fine multi-scale strategy is adopted at different levels to construct a pyramid structure of the downsampled image. The ADMM algorithm is used to solve the minimization model at each scale to obtain a multi-scale difference map.

8. The method according to claim 1, characterized in that The multi-scale difference map is fused and segmented using a fusion segmentation method based on Markov random fields to obtain a final change map, including: Taking superpixels as the basic unit in the change map, the pre-event image and the registered image Perform joint segmentation to obtain superpixels and the corresponding joint segmentation map ; The computation of the change map can be viewed as a superpixel label assignment problem, that is, for each superpixel ,Label Indicates no change. Indicates a change; The multi-scale difference map is upsampled and the At the layer scale The average change intensity of super pixels; defining a change energy function according to the average change intensity; According to the Markov random field theory, the label assignment problem is modeled as an energy minimization problem based on the variable energy function: in, and represent the variation energy function and the spatial energy function respectively, is the weight parameter; The energy problem is solved by a graph cut algorithm to obtain a final change graph.

9. The method according to claim 8, wherein A change energy function is defined according to the average change intensity, including: The change energy function is defined according to the average change intensity as follows: in, Indicates the At the layer scale Superpixels The average change intensity of represents a multi-scale difference map, is a defined function, a and b represent the variables of the function, is a parameter that controls the imbalance penalty rate, and L represents the scale of the difference map.

Images