Image change detection method, device and equipment based on key point graph structural features

Abstract

The invention relates to the field of remote sensing image processing, and provides an image change detection method, device and equipment based on key point graph structure features. The method comprises: obtaining a pre-phase SAR image and a post-phase SAR image of the same location, and extracting an initial difference image of an overlapping area; classifying the initial difference image, identifying a change area, and obtaining a rough change image; extracting a key point set of the pre-phase SAR image or the post-phase SAR image based on the rough change image, and constructing a pre-phase graph structure and a post-phase graph structure corresponding to the pre-phase SAR image and the post-phase SAR image respectively according to the key point set; cross-projecting the pre-phase graph structure and the post-phase graph structure, calculating a forward difference graph and a post-phase difference graph, and fusing and classifying them to obtain a point change graph; performing a joint segmentation operation on the pre-phase SAR image and the post-phase SAR image to generate a superpixel mask; and refining the point change graph based on the superpixel mask to generate a binary change graph.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing image processing, and in particular to an image change detection method, device, equipment and medium based on key point graph structural features. Background Art

[0002] Synthetic aperture radar (SAR) image change detection is a crucial task in remote sensing technology, and it has shown extremely high application value in military, civilian and many other fields. However, the SAR image change detection task is significantly affected by two key factors: one is the speckle noise caused by the coherent imaging mechanism, and the other is the geometric distortion caused by the side-view imaging mechanism. When detecting changes between high-resolution SAR images, traditional methods often show fragmented results, while most deep learning-based methods adopt unsupervised learning methods, which have weak generalization capabilities for high-resolution SAR images, resulting in blocky change detection results. In addition, the pixel-by-pixel or block-by-block detection methods are computationally expensive and cannot meet the needs of rapid extraction of change information in the future high-resolution and high-frequency observation background. Taking all factors into consideration, the multi-temporal high-resolution SAR image change detection task faces many challenges, including complex interference factors, diverse uncertainties, and requirements for completeness and timeliness, which together limit the accurate extraction of SAR image change information. Summary of the invention

[0003] 1. Technical issues to be resolved

[0004] In view of the above problems, the present invention provides an image change detection method, device, equipment and medium based on key point graph structural features, so as to at least partially solve the technical problems of traditional image change detection methods such as low accuracy, high cost, slow extraction speed and poor robustness.

[0005] (II) Technical solution

[0006] On the one hand, the present invention provides an image change detection method based on key point graph structural features, comprising: acquiring a pre-phase SAR image and a post-phase SAR image of the same location, and extracting an initial difference image of an overlapping area; classifying the initial difference image, identifying a change area, and obtaining a rough change image; based on the rough change image, extracting a key point set of the pre-phase SAR image or the post-phase SAR image, and constructing a pre-phase graph structure and a post-phase graph structure corresponding to the pre-phase SAR image and the post-phase SAR image respectively according to the key point set; cross-projecting the pre-phase graph structure and the post-phase graph structure, and calculating a forward difference graph and a backward difference graph, and fusing and classifying the forward difference graph and the backward difference graph to obtain a point change graph; performing a joint segmentation operation on the pre-phase SAR image and the post-phase SAR image to generate a spatiotemporally consistent superpixel mask; based on the superpixel mask, refining the point change graph to generate a binary change graph.

[0007] According to an embodiment of the present invention, the cross-projecting of the front phase image structure and the rear phase image structure, calculating a forward difference image and a rear phase difference image, and fusing and classifying the forward difference image and the rear phase difference image to obtain a point change image includes: forward mapping the front phase image structure to the rear phase SAR image to obtain a forward mapping image, and backward mapping the rear phase image structure to the front phase SAR image to obtain a backward mapping image; measuring the structural consistency of the forward mapping image and the rear phase image structure to obtain a forward difference image, and measuring the structural consistency of the backward mapping image and the front phase image structure to obtain a backward difference image; fusing the forward difference image and the rear phase difference image to obtain a point difference image; and classifying each key point in the key point set of the point difference image to obtain a point change image.

[0008] According to an embodiment of the present invention, classifying the initial difference image, identifying the changed area, and obtaining a rough changed image includes: based on the initial difference image, using the maximum inter-class variance to determine a segmentation threshold; using the segmentation threshold to classify the initial difference image to identify changed pixels and non-changed pixels, and obtain a rough changed image.

[0009] According to an embodiment of the present invention, extracting the key point set of the front-phase SAR image or the back-phase SAR image based on the coarse change image includes: based on the coarse change image, traversing the front-phase SAR image or the back-phase SAR image through a pre-set key point detection window, extracting the key points, and obtaining the corresponding key point set.

[0010] According to an embodiment of the present invention, based on the coarse change image, traversing the front-phase SAR image or the back-phase SAR image through a preset key point detection window, extracting the key points, and obtaining the corresponding key point set also includes: when a pixel of the front-phase SAR image or the back-phase SAR image is identified as a changed pixel by the coarse change image, reducing the key point detection window, and re-detecting to obtain the key point set.

[0011] According to an embodiment of the present invention, constructing the front-phase graph structure and the back-phase graph structure corresponding to the front-phase SAR image and the back-phase SAR image respectively according to the key point set includes: using a distance measurement method to calculate the distance between each key point in the key point set and other key points, and selecting K key points with the closest distance as adjacent key points; according to the radiation characteristics of the front-phase SAR image and the back-phase SAR image, respectively calculating the connection edge weights between each key point and the adjacent key point, and obtaining the similarity between each key point and the adjacent key point; based on the similarity between each key point and the adjacent key point, constructing the front-phase graph structure and the back-phase graph structure.

[0012] According to an embodiment of the present invention, the point change map is refined based on the superpixel mask to generate a binary change map, including: based on the point change map, all pixels within the key point detection window centered on the key point are expanded to generate an expanded point change map; using the superpixel mask, performing regional analysis on the expanded point change map to determine the changed area and the non-changed area, and obtaining the binary change map corresponding to the changed area.

[0013] The second aspect of the present invention provides an image change detection device based on key point graph structural features, comprising: an acquisition module, used to acquire a pre-phase SAR image and a post-phase SAR image of the same location, and extract an initial difference image of the overlapping area; a classification module, used to classify the initial difference image, identify the change area, and obtain a rough change image; an extraction module, used to extract a key point set of the pre-phase SAR image or the post-phase SAR image based on the rough change image, and construct the pre-phase SAR image and the post-phase SAR image according to the key point set. A front-phase image structure and a back-phase image structure corresponding to the time-phase SAR image; a fusion module, used for cross-projecting the front-phase image structure and the back-phase image structure, and calculating a forward difference map and a back-phase difference map, and fusing and classifying the forward difference map and the back-phase difference map to obtain a point change map; a segmentation module, used for performing a joint segmentation operation on the front-phase SAR image and the back-phase SAR image to generate a super-pixel mask with spatiotemporal consistency; a generation module, used for refining the point change map based on the super-pixel mask to generate a binary change map.

[0014] The third aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, each step of the image change detection method based on key point graph structural features is implemented.

[0015] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the various steps in the image change detection method based on key point graph structural features are implemented.

[0016] (III) Beneficial effects

[0017] The image change detection method, device, equipment and medium based on key point graph structural features provided by the present invention, by comprehensively considering the coherent and side-view imaging mechanisms of SAR sensors and the complexity increased by high resolution, adopts robust graph structural features and joint segmentation and refinement technology, effectively improving the accuracy and efficiency of change detection. At the same time, the use of key points replaces the traditional pixel-by-pixel analysis, significantly reducing the computational cost. In addition, the robustness of the difference map is enhanced by measuring changes using structural information, while the application of joint segmentation masks further refines the change detection results. These strategies jointly solve the problem of false alarms that may be caused by speckle noise and geometric distortion, as well as the problem of incomplete change information detection, making the extraction of change information in high-resolution complex scenes more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] For a more complete understanding of the present invention and its advantages, reference will now be made to the following description taken in conjunction with the accompanying drawings, in which:

[0019] Figure 1 A schematic diagram of a SAR payload coherent imaging mechanism provided by an embodiment of the present invention is schematically shown;

[0020] Figure 2 The flowchart of the image change detection method based on key point graph structural features provided by an embodiment of the present invention is schematically shown;

[0021] Figure 3 A schematic diagram of constructing a KNN graph structure provided by an embodiment of the present invention is schematically shown;

[0022] Figure 4 The structural block diagram of the image change detection device based on the key point graph structural features provided by an embodiment of the present invention is schematically shown;

[0023] Figure 5 The hardware structure diagram of an electronic device is schematically shown. DETAILED DESCRIPTION

[0024] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0025] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0026] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0027] Some block diagrams and / or flow charts are shown in the accompanying drawings. It should be understood that some blocks or combinations thereof in the block diagrams and / or flow charts may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that these instructions, when executed by the processor, may create a device for implementing the functions / operations described in these block diagrams and / or flow charts.

[0028] The inventors have found through research that in the related art, due to the unique imaging mechanism of the SAR payload, the images obtained are significantly different from those of the optical images in terms of visual features. This difference is mainly manifested in that the representation of the SAR image does not intuitively reflect the appearance of the actual object, that is, "what you see is not what you get". For example, in the comparison between the SAR image and the optical image of the same scene, the representation of some prominent ground objects in the SAR image is completely different from that in the optical image. This significant difference means that the change detection method originally applicable to optical satellite images may not be applicable to SAR images at all. In addition, the grayscale representation of SAR images is also affected by the coherent imaging and side-view imaging mechanisms, which leads to significant speckle noise and geometric distortion in the image. These interference factors not only increase the complexity of change information extraction, but also seriously hinder the accuracy and reliability of the change detection task.

[0029] Specifically, according to the working principle of SAR payload, the received echo signal is the coherent superposition of scattered signals from multiple scatterers within the resolution unit. This coherent superposition causes the SAR image to show significant fluctuations and high randomness in scattering intensity. Figure 1The coherent imaging mechanism of SAR payload is clearly demonstrated, revealing the three main characteristics of SAR coherent imaging: (a) the basic resolution unit contains many scatterers; (b) the total backscatter of the resolution unit is composed of the vector superposition of the scattered signals of each scatterer in the unit; (c) the backscatter of SAR is random. These characteristics together constitute the speckle noise on the SAR image. Its violent and random fluctuations make it extremely difficult to accurately describe the change information of each pixel, so that the results of conventional change detection methods show discreteness and incompleteness, limiting the accuracy of subsequent interpretation work. On the other hand, due to the side-view imaging mechanism of SAR, its imaging process can only capture the side information of ground targets, resulting in the payload being extremely sensitive to factors such as incident angle, azimuth and target height, and easily generating geometric distortion phenomena near buildings, such as overlap, corner lines and shadows. These geometric distortion phenomena are difficult to be effectively detected and eliminated when extracting change information, often resulting in a large number of false alarms and missed detections. Although some studies have attempted to mitigate the impact of geometric distortion on change detection by establishing a backscattering model for single buildings with flat roofs, this method often fails to ensure the integrity of the changed area and has limited practicality, especially in areas with dense buildings. In high-resolution SAR images, the interference of these limiting factors is even more serious.

[0030] In summary, the interference of speckle noise and the influence of geometric distortion make many high-frequency components appear on SAR images, resulting in unstable radiation values ​​of SAR images and high randomness. When change detection is performed directly based on the radiation characteristics of multi-phase SAR images, speckle noise often significantly reduces the accuracy of change detection. Structures are stable relative to noise, and they characterize the topological relationship of the image. Graph structure is an effective means to express the structural characteristics of the image, which can reflect the intrinsic relationship of the image at a higher level. It shows stability under large changes in imaging conditions and noise intensity, and is suitable for change detection tasks in high-resolution SAR images. In order to address the shortcomings of existing methods in high-resolution SAR image change detection, the present invention provides an image change detection method based on the structural characteristics of image key points, and uses joint segmentation technology to refine the detection results.

[0031] Figure 2 The flowchart of the image change detection method based on key point graph structural features provided by an embodiment of the present invention is schematically shown.

[0032] like Figure 2 As shown, the flowchart of the image change detection method based on key point graph structural features includes S1 to S6.

[0033] In operation S1 , a pre-phase SAR image and a post-phase SAR image of the same location are acquired, and an initial difference image of the overlapping area is extracted.

[0034] In operation S2, the initial difference image is classified to identify the changed area and obtain a rough changed image.

[0035] In operation S3, based on the rough change image, a key point set of the front-phase SAR image or the back-phase SAR image is extracted, and a front-phase graph structure and a back-phase graph structure corresponding to the front-phase SAR image and the back-phase SAR image are respectively constructed according to the key point set.

[0036] In operation S4, the front phase image structure and the back phase image structure are cross-projected, and a front difference image and a back phase difference image are calculated, and the front difference image and the back phase difference image are fused and classified to obtain a point change image.

[0037] In operation S5, a joint segmentation operation is performed on the front-phase SAR image and the back-phase SAR image to generate a spatiotemporally consistent superpixel mask.

[0038] In operation S6, the point-like change map is refined based on the superpixel mask to generate a binary change map.

[0039] Specifically, for the dual-phase SAR images (pre-phase SAR image X and post-phase SAR image Y) taken at different times and in the same place, a neighborhood ratio detector is used to extract the initial difference image DI of the overlapping area. init , and then the threshold segmentation is performed by the maximum between-class variance method (OTSU) to obtain a rough change image to guide the subsequent key point extraction process. Then, by setting the key point detection window and adopting the local extreme value strategy, the key points are extracted from the front or back phase SAR image. Then, after obtaining the key point set, for each key point, the most similar K key points are found in the set to construct the KNN graph structure of the key point. In order to ensure the accuracy of the KNN graph structure and extract reliable change information, in some exemplary embodiments, a distance metric considering the statistical characteristics of SAR is adopted. Subsequently, the same K nearest neighbor point search and graph structure construction process is performed on the other phase image. After the graph structure is established in the image domain of the two phases, the change intensity of the key point is evaluated by mapping the graph structure of one phase domain to the other phase domain and measuring the consistency of the structure. Based on the change intensity of each key point obtained, the Kmeans clustering algorithm is applied to divide the key points into Q categories, and the top q categories with the most significant changes are selected as the real change key points, and the rest are selected as non-change key points, thereby forming a point change graph. Finally, the multi-temporal SAR images are segmented through a joint segmentation operation to extract the superpixel mask that is consistent with the previous and next phase images. On this basis, the point change map is refined and the final binary change map is generated, which can accurately reflect the changes of each pixel.

[0040] For example, assuming that the dual-phase SAR image is represented as and , where H and W represent the number of rows and columns of the image after precise registration. x(m,n) and y(m,n) represent the grayscale values ​​of image X and Y at position (m,n).

[0041] According to the following formula, the initial difference image is calculated using the neighborhood ratio detector, that is,

[0042]

[0043] in, Represents the weight factor of the central pixel ratio and the neighborhood ratio, represents the local rectangular neighborhood at the center pixel (i, j), and Respectively represent the rectangular neighborhood The standard deviation and mean of . α is a measure of local heterogeneity. When α is small, The region corresponds to a homogeneous region, and the second part in formula (1) plays a dominant role. When α is large, The region is considered to be a heterogeneous region, and the first part of formula (1) plays a dominant role. Through the above operation, the speckle suppression ability and detail preservation ability can be balanced to obtain a more accurate difference map.

[0044] It can be understood that the change detection method based on key point graph structural features provided by the embodiment of the present invention can effectively extract change information between multi-phase high-resolution SAR images, generate a clear, complete and well-detailed binary change map, and provide high-quality change analysis products for subsequent applications.

[0045] Furthermore, the classifying the initial difference image, identifying the changed area, and obtaining the rough changed image includes: determining a segmentation threshold based on the initial difference image using the maximum inter-class variance; and classifying the initial difference image using the segmentation threshold to identify changed pixels and non-changed pixels to obtain the rough changed image.

[0046] For example, based on the initial difference image DI obtained init , use the maximum between-class variance (OTSU) to determine an optimal segmentation threshold T opt The threshold T opt The initial difference image DI init Divided into two categories: Change pixel set and the set of non-changing pixels . Non-changing pixels are represented by 0, and changing pixels are represented by 1, that is, The proportion of changed pixels is recorded as p0, and the average change intensity is recorded as μ c ; Correspondingly, the proportion of unchanged pixels is recorded as p1, and its average change intensity is recorded as μ uc The entire initial difference image DI init The total average change intensity is denoted as μ total Therefore, we can get the following between-class variance, expressed as

[0047]

[0048] Given that DI init It can be normalized to the interval [0,255], so the optimal threshold T opt The value range of is also limited to [0,255]. opt The optimal threshold can be determined by setting it to every possible value in this range and applying the following formula:

[0049]

[0050] According to the optimal threshold T opt DI init Divide according to the following formula to get the rough change map CM coarse .

[0051]

[0052] In an embodiment of the present invention, extracting the key point set of the front-phase SAR image or the back-phase SAR image based on the coarse change image includes: based on the coarse change image, traversing the front-phase SAR image or the back-phase SAR image through a pre-set key point detection window, extracting the key points, and obtaining the corresponding key point set.

[0053] Furthermore, based on the coarse change image, traversing the front-phase SAR image or the back-phase SAR image through a pre-set key point detection window, extracting the key points, and obtaining the corresponding key point set also includes: when a pixel of the front-phase SAR image or the back-phase SAR image is identified as a changed pixel by the coarse change image, reducing the key point detection window, and re-detecting to obtain the key point set.

[0054] Using the rough change map CM coarse As a guide and aid, extract key points from the previous phase SAR image X or the next phase SAR image Y, and obtain the key point set If the central pixel of the key point detection window is identified as a change category (i.e., a change pixel) in the roughly changed image, the window size is reduced and the detection is performed again.

[0055] For example, in the rough change diagram CM coarse On this basis, key points are extracted from the SAR image X. The key point detection window size is set to ω×ω, and by traversing the image X, key points can be identified according to the following criteria:

[0056]

[0057] in, represents the set of key points extracted by the above method, and s represents CM coarse Auxiliary window scaling factor.

[0058] It can be understood that based on the rough change map CM coarse The guided dense key point extraction method can not only comprehensively capture the global features of the image, but also specifically enhance the description ability of potential change areas by adjusting the density and number of key points, thereby achieving more accurate and robust change detection.

[0059] Furthermore, the constructing of the pre-phase graph structure and the post-phase graph structure corresponding to the pre-phase SAR image and the post-phase SAR image respectively according to the key point set includes: using a distance measurement method to calculate the distance between each key point in the key point set and other key points, and selecting the K key points with the closest distance as adjacent key points; according to the radiation characteristics of the pre-phase SAR image and the post-phase SAR image, respectively calculating the connection edge weights of each key point and the adjacent key points to obtain the similarity between each key point and the adjacent key points; based on the similarity between each key point and the adjacent key points, constructing the pre-phase graph structure and the post-phase graph structure.

[0060] For example, for the acquired key point set For each key point in , a robust distance measurement method suitable for SAR is used. For each key point, find K key points that are most similar to it as adjacent key points. The i-th key point p in i ,in (N kpt is the total number of key points), and its KNN graph structure (pre-phase graph structure) in the image domain X is defined as ,in:

[0061]

[0062] in Indicates key points The pre-phase diagram structure with the core as the core, Representation diagram All nodes included, Representation and key points The set of key points with the most similar features, Represents the weight of the connection edge, quantifying the key points The similarity between adjacent key points is calculated by the distance function Calculated, where γ is a preset compression parameter. This distance function measures the similarity between two key points, and its calculation is based on the similarity between the image blocks around the two key points. Assuming and The image patch centered at and , the size of the image block is ,but It can be calculated by the following formula

[0063]

[0064] in represents the similarity function, and Represents image blocks and The grayscale value of the kth element in .

[0065] In the same way, the corresponding post-phase image structure can be constructed on the post-phase image Y, expressed as It should be noted that although the previous phase image X and the next phase image Y share the same set of key points, namely , but the edge weights between these keypoints and their adjacent keypoints are calculated based on their respective radiation features in the X and Y domains. This means that and The topological structure is inconsistent. Key points in the image domain X and Y and The graph structure is as follows Figure 3 As shown in Figure 2. Since X and Y have different image features, the i-th key point and With different graph structures.

[0066] Furthermore, the cross-projection of the front phase image structure and the back phase image structure, and the calculation of the forward difference image and the back phase difference image, as well as the fusion and classification of the forward difference image and the back phase difference image to obtain the point change image include: forward mapping the front phase image structure to the back phase SAR image to obtain the forward mapping image, and backward mapping the back phase image structure to the front phase SAR image to obtain the backward mapping image; measuring the structural consistency of the forward mapping image and the back phase image structure to obtain the forward difference image, and measuring the structural consistency of the backward mapping image and the front phase image structure to obtain the backward difference image; fusing the forward difference image and the back phase difference image to obtain the point difference image; and classifying each key point in the key point set of the point difference image to obtain the point change image.

[0067] Specifically, through cross-projection, the pre-phase graph structure established in the X domain is forward mapped to the Y domain to obtain the forward mapping graph . Backward map the post-phase graph structure in the Y domain to the X domain to obtain the backward map On this basis, by measuring and G Y The forward difference map DI is obtained by the structural consistency of forward , through the metric G X and The backward difference map DI is obtained by the structural consistency of backward , by fusing the two, we get the point difference map DI point Then DI point Classification and analysis to obtain point-like change map CM point .

[0068] For example, based on the key point set And the KNN structure constructed by the key points in the front phase domain X and the back phase domain Y are respectively expressed as and ,in , indicating that there are N kpt Each key point graph structure consists of a set of nodes. , edge set and edge weights composition. Expresses the key point i The connection relationship with other key points is the basis of graph structure expression. and They share the same set of key points, but the connections in each graph are different. In order to measure the level of change of each key point, the graph structure needs to be cross-projected so that the point difference map can be calculated in each phase domain.

[0069] Cross projection includes forward projection and backward projection. The connection relationship of is mapped to Y, and the connection edge weights are recalculated in Y to obtain the forward projection graph structure , , and They are expressed as follows:

[0070]

[0071] in Indicates that the X domain and the key point The most similar K adjacent node positions Mapped to the Y domain, the mapped The key points and The key points in have consistent image coordinates. Represents the edge weight recalculated based on the mapped neighborhood. Similarly, in the back-projection process, The connection relationship expressed is mapped to the X domain, and the connection edge weights are recalculated to obtain the backward projection graph structure Through the above cross-mapping process, the connection relationship of key points can be compared in the image domain of the previous and next phases, so that the similarity between key points can be revealed by measuring the consistency of the structural relationship, thereby reflecting the changes of key points between the previous and next phases.

[0072] For a graph model, its Laplace matrix and random walk matrix are defined as L = DW and . Where D represents the degree matrix of the graph model, defined as

[0073]

[0074] Assume that the definition is N kpt Define the signal function or characteristic function at the key points , some important properties of the graph can be derived from the following formula:

[0075]

[0076]

[0077]

[0078] In order to evaluate the degree of change of key points in the previous phase X and the next phase Y, the features of key points are first extracted in the two image domains respectively, and the following feature set is defined:

[0079]

[0080]

[0081] in, Represents the key point p i Characteristic signal in X domain or Y domain, Represented by key point p i The grayscale mean of the sz×sz rectangular area centered on .

[0082] Using these characteristics, we can calculate according to formula (10) and The table represents the SAR image X of the previous phase and the SAR image Y of the next phase at the key point p. i The radiation information and structural information related to the local topology of the image are located at You can get , we can get on the backward mapping graph .

[0083] and , and share the same topological structure, so based on this local information, we can define a forward mapping difference map and the backward mapping difference map , as shown below:

[0084]

[0085]

[0086] Through weighted fusion and , we can get the final point difference map :

[0087]

[0088] Among them, η is the weight factor, ranging from 0 to 1. The larger the value, the more critical the point p i The higher the probability of change, the higher the probability of change. Then, use the Kmeans clustering algorithm to Perform classification processing to obtain the category index of the key points :

[0089]

[0090] in Represents the key point p iAssuming the number of categories is Q, we sort them in descending order according to the size of the cluster center value, select the key points of the first q categories as the change category and mark them as 1, and the remaining categories as the non-change category and mark them as 0, thus generating a point change map .

[0091] Furthermore, a joint segmentation operation is performed on the front-phase SAR image X and the back-phase SAR image Y to generate a spatiotemporally consistent superpixel mask .

[0092] Exemplarily, the joint segmentation is based on the minimum heterogeneity criteria of spectrum and shape, and uses a bottom-up approach to calculate the heterogeneity and merge judgment of adjacent pixels or objects. Specifically, if the heterogeneity increment caused by merging two regions to be merged is less than a predefined threshold, the two regions are merged; if the heterogeneity increment exceeds the threshold, other regions to be merged are continuously evaluated.

[0093] For the segmentation of a single phase X or Y, when considering merging adjacent regions r i and r j When , the heterogeneous increment generated by merging It can be calculated by the following formula

[0094]

[0095] in, Represents region r i and r j The heterogeneous increments generated by the merger, and denote the spectral heterogeneity increment and shape heterogeneity increment respectively, and Represent the corresponding weight factors respectively, and satisfy .

[0096] Spectral heterogeneity To ensure the homogeneity of the segmented object in radiation, the calculation formula is defined as follows:

[0097]

[0098] in represents the weight factor of the c-band, , as well as Respectively represent the merged areas and the adjacent region r i and r j The number of pixels contained, , and They represent the standard deviation within the region.

[0099] Shape heterogeneity Ensure that the object is compact in geometry to avoid strip interference. The calculation formula is defined as follows:

[0100]

[0101] in and Represents smoothness and compactness respectively, and the specific definitions are as follows

[0102]

[0103]

[0104] in, , and Respectively represent the area , and The circumference of , and Respectively represent the perimeter of the minimum circumscribed rectangle of the region, and the smoothness and compactness weight factors satisfy the relationship

[0105] Assume that the predefined threshold is ,if , then r i and r j Merge into ;if , no merging is performed.

[0106] In the joint segmentation process, the above region merging is performed on X and Y respectively, and then the two merging processes are coordinated through joint decision making. In order to make the segmentation take into account the change information, first X and Y are merged. Stacking is done in the channel dimension, Y and Stack in the channel dimension and then get a new multi-channel image and . and It contains common change information, which helps to obtain a complete segmentation result of the change area. The realization of joint segmentation depends on two key steps: 1) Joint determination and 1) The merged area that is consistent with the previous one; 2) Determine and The relationship between the heterogeneity increment and the predefined threshold.

[0107] First, jointly determine and At some point in the segmentation process, suppose Area on the image With regional collection adjacent to each other, represents the total number of adjacent regions. Similarly, we can get Image area The set of adjacent regions . Where m and n represent the indexes of adjacent regions and satisfy The joint segmentation ensures that and Each step above is consistent with merging and not merging, which means and have the same position and size, and the position, size, and number of their adjacent areas are also consistent.

[0108] In getting and After that, we can calculate Heterogeneity resulting from merging with adjacent regions, and The heterogeneity generated by merging with adjacent regions is expressed as follows

[0109]

[0110]

[0111] in express and The increase in heterogeneity caused by the merger, express and The amount of increase in heterogeneity due to the merger. and The merge has the smallest heterogeneity increment, i.e. , similarly, assuming and The merge has the smallest heterogeneity increment, i.e. .

[0112] If the image The index of the best merged region is equal to the image The index of the best merged region on the , that is, m=n, then set k=m=n, so we can get and The best merge object is expressed as If m≠n, then we need to calculate and The heterogeneous increments generated by the merger and and The heterogeneity increments generated by the merger are denoted as and Next, the index of the consistent merged region is determined by the following combination of criteria:

[0113]

[0114] in They are and The best area to be merged.

[0115] After a consistent merge area is identified, a joint decision is made to decide whether to and as well as and Merge. First, in the image and images The merge threshold set on and , the joint decision rule is expressed as follows:

[0116]

[0117] If the above conditions are met, then and Merge and Merge. Traverse the previous phase images in order from top to bottom and from left to right. Post-phase imaging , when there are no adjacent regions to merge, the joint segmentation process reaches a convergence state and finally outputs a consistent superpixel mask .

[0118] Furthermore, based on the superpixel mask, the point change map is refined to generate a binary change map, including: based on the point change map, all pixels within the key point detection window centered on the key point are expanded to generate an expanded point change map; using the superpixel mask, performing regional analysis on the expanded point change map to determine the changed area and the non-changed area, and obtaining the binary change map corresponding to the changed area.

[0119] Based on the acquired superpixel mask , for the point change diagram CM obtained point Refine it and generate the final binary change map BCM.

[0120] Exemplarily, the aforementioned sliding window method is used to detect key points in an image. Specifically, for each pixel in the image, its grayscale value within a rectangular window of size ω×ω is checked. If the grayscale value of the pixel is an extreme value in the window (that is, it is the maximum or minimum value in the window), the pixel is determined to be a key point. These key points are representative because of their extreme characteristics in local areas, and effectively capture the local grayscale changes and structural information of the image. Based on this idea, if a key point is identified as a change class, it is assumed that all pixels within the ω×ω window range where the key point is located belong to the change class. Specifically, assuming that the key point p i The position coordinates of are (x,y), then , and the expanded point-like change graph can be defined :

[0121]

[0122] in, Represents a window of size ω×ω centered at (x,y) in the image domain.

[0123] Get the expanded change graph Then, according to the segmentation mask Further analysis of the change map. For the segmentation mask Each region r in i , the number of pixels belonging to the change category C i It can be expressed as r i All pixels (x, y) in The sum of:

[0124]

[0125] T i Represents region r i The total number of pixels in area r i The proportion of pixels belonging to the change category to the total number of pixels P i for:

[0126]

[0127] Set the specific gravity threshold parameter t, if P i exceeds a predefined threshold t (i.e., P i ≥t), then the area r i Identified as a change area, i.e.

[0128]

[0129] After the above processing, complete, clean and well-edge-preserved change information can be obtained, providing accurate results for high-resolution SAR change detection.

[0130] The change detection method based on the key point graph structure feature provided by the embodiment of the present invention significantly reduces the false alarm and fragmentation problems caused by speckle noise and geometric distortion in high-resolution SAR images. By applying key points, tedious pixel-by-pixel change analysis is avoided, and the processing efficiency is greatly improved. In addition, the degree of change can be detected by measuring the consistency of the graph structure, which can effectively improve the robustness of the difference map. The consistent superpixel mask generated by the joint segmentation technology refines the point change map, greatly enhancing the accuracy and completeness of multi-phase high-resolution SAR change detection.

[0131] Figure 4 The structural block diagram of the image change detection device based on key point graph structural features provided by an embodiment of the present invention is schematically shown.

[0132] like Figure 4 As shown, the image change detection device 400 based on key point graph structural features of this embodiment includes an acquisition module 401 , a classification module 402 , an extraction module 403 , a fusion module 404 , a segmentation module 405 and a generation module 406 .

[0133] The acquisition module 401 is used to acquire the pre-phase SAR image and the post-phase SAR image of the same location, and extract the initial difference image of the overlapping area.

[0134] The classification module 402 is used to classify the initial difference image, identify the changed area, and obtain a rough changed image.

[0135] The extraction module 403 is used to extract the key point set of the front-phase SAR image or the back-phase SAR image based on the rough change image, and construct the front-phase graph structure and the back-phase graph structure corresponding to the front-phase SAR image and the back-phase SAR image respectively according to the key point set.

[0136] The fusion module 404 is used to cross-project the structure of the front phase image and the structure of the back phase image, calculate the front difference image and the back phase difference image, and fuse and classify the front difference image and the back phase difference image to obtain a point change image.

[0137] The segmentation module 405 is used to perform a joint segmentation operation on the front-phase SAR image and the back-phase SAR image to generate a spatiotemporally consistent superpixel mask.

[0138] The generation module 406 is used to refine the point-like change map based on the superpixel mask to generate a binary change map.

[0139] It is understandable that the acquisition module 401, the classification module 402, the extraction module 403, the fusion module 404, the segmentation module 405 and the generation module 406 can be combined in one module, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the acquisition module 401, the classification module 402, the extraction module 403, the fusion module 404, the segmentation module 405 and the generation module 406 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented in hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or in a suitable combination of software, hardware and firmware. Alternatively, at least one of the acquisition module 401, classification module 402, extraction module 403, fusion module 404, segmentation module 405 and generation module 406 can be at least partially implemented as a computer program module, and when the program is run by a computer, the function of the corresponding module can be executed.

[0140] Figure 5 The hardware structure diagram of an electronic device provided by an embodiment of the present invention is schematically shown.

[0141] like Figure 5 As shown, the electronic device described in this embodiment includes: the electronic device 500 includes a processor 510 and a computer readable storage medium 520. The electronic device 500 can execute the above reference Figure 2 Describes the method to achieve detection of specific operations.

[0142] Specifically, the processor 510 may include, for example, a general-purpose microprocessor, an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 510 may also include an onboard memory for cache purposes. The processor 510 may be a processor for executing reference Figure 2 The described method flow according to the embodiment of the present invention may be a single processing unit or multiple processing units for different actions.

[0143] Computer readable storage medium 520, for example, can be any medium capable of containing, storing, conveying, propagating or transmitting instructions. For example, readable storage media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices, devices or propagation media. Specific examples of readable storage media include: magnetic storage devices, such as magnetic tapes or hard disks (HDDs); optical storage devices, such as compact disks (CD-ROMs); memory, such as random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0144] The computer readable storage medium 520 may include a computer program 521, which may include code / computer executable instructions, which when executed by the processor 510 causes the processor 510 to perform, for example, the above-mentioned combination Figure 2 The described method sequences and any variations thereof.

[0145] The computer program 521 may be configured to have, for example, computer program codes including computer program modules. For example, in an exemplary embodiment, the codes in the computer program 521 may include one or more program modules, for example, 521A, 521B, ... It should be noted that the division method and number of modules are not fixed, and those skilled in the art may use appropriate program modules or program module combinations according to actual conditions. When these program module combinations are executed by the processor 510, the processor 510 may execute, for example, the above combination of Figure 2 The described method sequences and any variations thereof.

[0146] According to an embodiment of the present invention, at least one of the acquisition module 401, the classification module 402, the extraction module 403, the fusion module 404, the segmentation module 405 and the generation module 406 can be implemented as a reference. Figure 5 The computer program modules described herein, when executed by the processor 510, can implement the corresponding operations described above.

[0147] The present invention also provides a computer-readable medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0148] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0149] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for image change detection based on key point graph structural features, characterized in that: include: Acquire the pre-phase SAR image and post-phase SAR image of the same location, and extract the initial difference image of the overlapping area; Classifying the initial difference image, identifying the changed area, and obtaining a rough changed image; Based on the rough change image, extract the key point set of the front-phase SAR image or the back-phase SAR image, and construct the front-phase graph structure and the back-phase graph structure corresponding to the front-phase SAR image and the back-phase SAR image respectively according to the key point set; the construction of the front-phase graph structure and the back-phase graph structure corresponding to the front-phase SAR image and the back-phase SAR image respectively according to the key point set includes: using a distance measurement method to calculate the distance between each key point in the key point set and other key points, and selecting the K key points with the closest distance as adjacent key points; according to the radiation characteristics of the front-phase SAR image and the back-phase SAR image, respectively calculate the connection edge weights of each key point and the adjacent key point to obtain the similarity between each key point and the adjacent key point; based on the similarity between each key point and the adjacent key point, construct the front-phase graph structure and the back-phase graph structure; The structure of the front phase image and the structure of the back phase image are cross-projected, and a forward difference image and a back phase difference image are calculated, and the forward difference image and the back phase difference image are fused and classified to obtain a point change image; specifically including: forward mapping the structure of the front phase image to the back phase SAR image to obtain a forward mapping image, and backward mapping the structure of the back phase image to the front phase SAR image to obtain a backward mapping image; measuring the structural consistency of the forward mapping image and the structure of the back phase image to obtain a forward difference image, and measuring the structural consistency of the backward mapping image and the structure of the front phase image to obtain a backward difference image; fusing the forward difference image and the back phase difference image to obtain a point difference image; classifying each key point in the key point set of the point difference image to obtain a point change image; Performing a joint segmentation operation on the front-phase SAR image and the back-phase SAR image to generate a spatiotemporally consistent superpixel mask; Based on the superpixel mask, the point change map is refined to generate a binary change map; specifically, the method includes: based on the point change map, all pixels in the key point detection window centered on the key point are expanded to generate an expanded point change map; using the superpixel mask, regional analysis is performed on the expanded point change map to determine the changed area and the non-changed area, and the binary change map corresponding to the changed area is obtained.

2. The image change detection method based on key point graph structural features according to claim 1, characterized in that: The classifying the initial difference image, identifying the changed area, and obtaining the rough changed image comprises: Based on the initial difference image, determining a segmentation threshold using a maximum inter-class variance; The initial difference image is classified using the segmentation threshold to identify changed pixels and non-changed pixels, thereby obtaining a rough changed image.

3. The image change detection method based on key point graph structural features according to claim 1, characterized in that: The extracting the key point set of the front-phase SAR image or the back-phase SAR image based on the rough change image comprises: Based on the rough change image, the front-phase SAR image or the back-phase SAR image is traversed through a preset key point detection window to extract the key points and obtain a corresponding key point set.

4. The image change detection method based on key point graph structural features according to claim 2, characterized in that: The step of traversing the front-phase SAR image or the back-phase SAR image through a preset key point detection window based on the rough change image to extract the key points and obtain the corresponding key point set further comprises: When pixels of the preceding phase SAR image or the succeeding phase SAR image are identified as changed pixels by the rough change image, the key point detection window is reduced, and detection is performed again to obtain the key point set.

5. An image change detection device based on key point graph structural features, characterized in that: include: An acquisition module is used to acquire the pre-phase SAR image and the post-phase SAR image of the same location and extract the initial difference image of the overlapping area; A classification module, used to classify the initial difference image, identify the change area, and obtain a rough change image; An extraction module is used to extract the key point set of the front-phase SAR image or the back-phase SAR image based on the rough change image, and construct the front-phase graph structure and the back-phase graph structure corresponding to the front-phase SAR image and the back-phase SAR image respectively according to the key point set; the construction of the front-phase graph structure and the back-phase graph structure corresponding to the front-phase SAR image and the back-phase SAR image respectively according to the key point set includes: using a distance measurement method to calculate the distance between each key point in the key point set and other key points, and selecting the K key points with the closest distance as adjacent key points; according to the radiation characteristics of the front-phase SAR image and the back-phase SAR image, respectively calculating the connection edge weights of each key point and the adjacent key point to obtain the similarity between each key point and the adjacent key point; based on the similarity between each key point and the adjacent key point, constructing the front-phase graph structure and the back-phase graph structure; A fusion module is used to cross-project the structure of the front phase image and the structure of the back phase image, calculate a forward difference image and a back phase difference image, and fuse and classify the forward difference image and the back phase difference image to obtain a point change image; specifically comprising: forward mapping the structure of the front phase image to the back phase SAR image to obtain a forward mapping image, and backward mapping the structure of the back phase image to the front phase SAR image to obtain a backward mapping image; measuring the structural consistency of the forward mapping image and the structure of the back phase image to obtain a forward difference image, and measuring the structural consistency of the backward mapping image and the structure of the front phase image to obtain a backward difference image; fusing the forward difference image and the back phase difference image to obtain a point difference image; classifying each key point in the key point set of the point difference image to obtain a point change image; A segmentation module, used for performing a joint segmentation operation on the front-phase SAR image and the back-phase SAR image to generate a spatiotemporally consistent superpixel mask; A generation module is used to refine the point change map based on the superpixel mask to generate a binary change map; specifically includes: based on the point change map, all pixels in the key point detection window centered on the key point are expanded to generate an expanded point change map; using the superpixel mask, performing regional analysis on the expanded point change map to determine the changed area and the non-changed area, and obtaining the binary change map corresponding to the changed area.

6. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, each step of the image change detection method based on key point graph structural features as described in any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the image change detection method based on key point graph structural features described in any one of claims 1 to 4 is implemented.

Images