A multi-temporal remote sensing image map spot space-time correlation analysis method and device

Abstract

The application discloses a multi-temporal remote sensing image map spot space-time correlation analysis method and device, the method comprises the following steps: acquiring multi-temporal remote sensing images for the same monitoring area, extracting target class change map spots from each time image to form a target class change map spot set arranged in time sequence; constructing a space-time correlation rule and a similarity measurement model, calculating the comprehensive correlation confidence of any two different time target class change map spots belonging to the same ground object; taking each time target class change map spot as a node and taking the correlation relationship constructed based on the comprehensive correlation confidence as an edge, constructing a space-time correlation graph; based on the space-time correlation graph, finding an optimal correlation path through a graph optimization algorithm, correlating and integrating multiple map spots belonging to the same ground object and distributed in different time phases to generate a space-time ground object object representing the complete life cycle of a single ground object, and outputting the space-time ground object object and its correlation information.

Description

Technical Field

[0001] This application relates to the field of remote sensing information technology, and more specifically, to a method and apparatus for spatiotemporal correlation analysis of multi-temporal remote sensing image patches. Background Technology

[0002] In remote sensing dynamic monitoring operations in fields such as water conservancy, land resources, and environmental protection, interpreting multi-temporal remote sensing images of the same area acquired at different times is a core technical means to detect and track surface changes (especially construction projects). Based on this, it is possible to determine whether a project is compliant, identify unauthorized construction, monitor its construction progress, and investigate and punish illegal activities such as encroachment on river channels / water areas. Current technology can extract newly added or changed surface patches from each image and identify their land cover types (such as construction land, water bodies, vegetation, etc.).

[0003] However, existing technologies primarily focus on identifying a single time phase or detecting changes in adjacent time phases, resulting in a set of discrete, independent patches along the time dimension. When faced with a long-term construction project spanning multiple monitoring periods (e.g., a water conservancy facility from foundation laying and construction to completion), this method interprets it as multiple changing patches appearing at different times, failing to automatically establish the inherent connections between these patches. Therefore, existing technologies suffer from the following main drawbacks:

[0004] First, it is impossible to reconstruct the complete life cycle of ground features. Because the map features belonging to the same construction project detected at different times are fragmented, it is impossible to intuitively and quickly obtain the start time, construction period, construction speed, and morphological evolution (such as expansion, splitting, or merging) of a specific project for a currently monitored change map feature.

[0005] Second, existing correlation methods suffer from low automation, high subjectivity, and low efficiency. Currently, associating change patches from different periods with the same ground object mainly relies on manual interpretation and experience of operational personnel. For large-scale, long-term monitoring areas, manual correlation methods are extremely labor-intensive, inefficient, and the accuracy and consistency of the correlation results are difficult to guarantee, severely limiting the in-depth application value of massive remote sensing monitoring data.

[0006] Third, limited automated association methods suffer from poor robustness. Some methods attempting automatic association typically rely on simple matching based solely on single geometric features such as the spatial overlap rate between patches. However, in real-world scenarios, construction projects evolve dynamically, and the shape, area, and spatial extent of their patches change over time (e.g., from foundation to main structure, and then to ancillary facilities). Simple location overlap methods cannot effectively handle complex situations such as patch splitting, merging, expansion, and morphological evolution, easily leading to association errors and insufficient reliability.

[0007] Therefore, there is an urgent need for an analytical method that can automatically and accurately associate image patches belonging to the same ground object in remote sensing images from different time phases, and thereby construct their continuous spatiotemporal evolution sequence. Summary of the Invention

[0008] To address at least one deficiency or improvement need in the existing technology, the present invention provides a method and apparatus for spatiotemporal correlation analysis of multi-temporal remote sensing image patches, which solves the technical problem that existing remote sensing dynamic monitoring technology cannot automatically and accurately correlate changing patches of different periods when processing long-term, multi-temporal construction projects, thus failing to restore the complete life cycle information of a single construction project.

[0009] To achieve the above objectives, according to a first aspect of the present invention, a method for spatiotemporal correlation analysis of multi-temporal remote sensing image patches is provided. This method includes: acquiring multi-temporal remote sensing images of the same monitoring area; extracting target-class change patches from each temporal image to form a set of target-class change patches arranged in chronological order; constructing spatiotemporal correlation rules and a similarity measurement model; calculating the comprehensive correlation confidence that any two target-class change patches from different temporal phases belong to the same land cover; constructing a spatiotemporal correlation graph using target-class change patches from each temporal phase as nodes and correlation relationships constructed based on the comprehensive correlation confidence as edges; and, based on the spatiotemporal correlation graph, finding the optimal correlation path using a graph optimization algorithm to integrate and correlate multiple patches belonging to the same land cover and distributed in different temporal phases, generating a spatiotemporal land cover object representing the complete lifecycle of a single land cover, and outputting the spatiotemporal land cover object and its correlation information.

[0010] In an exemplary embodiment, the step of acquiring multi-temporal remote sensing images of the same monitoring area and extracting target-type change patches from each temporal image to form a set of target-type change patches arranged in chronological order includes: performing change detection on images of consecutive temporal phases in the multi-temporal remote sensing images to generate a binary change mask; performing object-oriented segmentation on the binary change mask to obtain a set of change patches for each temporal phase; and using a classifier to filter out the target-type change patches from the set of change patches to form the set of target-type change patches.

[0011] In an exemplary embodiment, the calculation of the comprehensive association confidence that any two target class change patches from different time phases belong to the same land cover includes: the comprehensive association confidence is calculated based on the fusion of spatiotemporal proximity factors and morphological similarity factors; the spatiotemporal proximity factors include distance similarity calculated based on the distance between the geometric center points of the two patches and spatial overlap calculated based on the ratio of the intersection and union areas of the two patches; the morphological similarity factors include area similarity calculated based on the reasonableness of the area change trends of the two patches and shape similarity calculated based on the similarity of the shape features of the two patches.

[0012] In an exemplary embodiment, the comprehensive association confidence score is obtained by calculating a weighted geometric mean of the distance similarity, spatial overlap, area similarity, and shape similarity.

[0013] In an exemplary embodiment, after constructing the spatiotemporal association rules and similarity measurement model, the method further includes: applying spatiotemporal context constraint factors, wherein the spatiotemporal context constraint factors include at least one of the following rules: association time-series unidirectionality rule, life-cycle continuity assumption rule, and maximum spatial search distance rule.

[0014] In an exemplary embodiment, the step of constructing a spatiotemporal correlation graph using target class change patches of each time phase as nodes and correlation relationships constructed based on the comprehensive correlation confidence as edges includes: constructing a vertex set using the target class change patches of each time phase as vertices; for two vertices in the vertex set belonging to different time phases and whose temporal sequence satisfies a preset direction, if the comprehensive correlation confidence between the two corresponding patches is higher than a preset edge existence threshold, then constructing an edge between the two vertices; assigning a corresponding weight to each edge based on the comprehensive correlation confidence, wherein the weight is determined based on the comprehensive correlation confidence, and the weight of the edge is negatively correlated with the comprehensive correlation confidence of the edge; and forming the spatiotemporal correlation graph from the vertex set and the edge set composed of all the edges.

[0015] In an exemplary embodiment, generating a spatiotemporal feature object representing the complete life cycle of a single feature includes: on the spatiotemporal association graph, using a dynamic programming algorithm to solve for the optimal set of association paths covering all vertices with the objective of minimizing the total path weight; and generating a spatiotemporal feature object based on each of the optimal association paths.

[0016] According to a second aspect of the present invention, a spatiotemporal correlation analysis device for multi-temporal remote sensing image patches is also provided, which performs the above-described spatiotemporal correlation analysis method for multi-temporal remote sensing image patches, comprising: an acquisition unit, configured to acquire multi-temporal remote sensing images of the same monitoring area, extract target-type change patches from each temporal image, and form a set of target-type change patches arranged in chronological order; a calculation unit, configured to construct spatiotemporal correlation rules and a similarity measurement model, and calculate the comprehensive correlation confidence that any two target-type change patches from different temporal phases belong to the same land feature; a construction unit, configured to construct a spatiotemporal correlation graph using target-type change patches from each temporal phase as nodes and correlation relationships constructed based on the comprehensive correlation confidence as edges; and an output unit, configured to find the optimal correlation path based on the spatiotemporal correlation graph using a graph optimization algorithm, integrate multiple patches belonging to the same land feature and distributed in different temporal phases, generate a spatiotemporal land feature object representing the complete life cycle of a single land feature, and output the spatiotemporal land feature object and its correlation information.

[0017] According to a third aspect of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, wherein the computer program is configured to execute the above-described method for spatiotemporal correlation analysis of multi-temporal remote sensing image patches at runtime.

[0018] According to a fourth aspect of the present invention, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the above-described multi-temporal remote sensing image patch spatiotemporal correlation analysis method through the computer program.

[0019] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects:

[0020] (1) This invention provides a spatiotemporal correlation analysis method for multi-temporal remote sensing image patches. By constructing a spatiotemporal correlation model, it automatically connects isolated patches from different temporal phases into continuous spatiotemporal ground objects, fully reproducing the entire life cycle of each construction project from commencement to construction and final form. This improves the automation and intelligence level of remote sensing dynamic monitoring. A comprehensive measurement model integrating spatiotemporal proximity, morphological similarity, and spatiotemporal context constraints is constructed. This model does not rely solely on a single feature but makes multi-feature decisions through weighted geometric average. At the same time, the optimal path search algorithm based on graph theory and dynamic programming can find the most reasonable correlation scheme from a global perspective, thus ensuring the accuracy and logical rationality of the correlation results even in complex scenarios.

[0021] (2) Based on spatiotemporal objects, this invention can directly extract key business indicators such as the start time, construction period, land area change curve, and average construction speed. More importantly, it can build precise business rules based on these (such as new projects that appear after the regulations take effect within the scope of river management), realize the automatic screening and push of suspected illegal projects, and improve the efficiency of supervision.

[0022] (3) Each spatiotemporal object generated by this invention contains spatial snapshots of it across all time periods and their interrelationships, forming a complete, continuous, and traceable spatiotemporal evidence chain. This data objectively records the entire historical evolution of the construction project and can serve as strong technical evidence in law enforcement processes such as identifying illegal projects and tracing responsibility, effectively avoiding the problem of difficulty in obtaining evidence due to on-site damage. At the same time, the method framework described in this invention has universality. The association rule weights and threshold parameters can be flexibly adjusted and customized according to the characteristics of land features in different regions and different business focuses (such as river encroachment in the water conservancy field and ecological damage in the environmental protection field), that is, it has high scalability and adaptability. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart illustrating an optional multi-temporal remote sensing image patch spatiotemporal correlation analysis method provided in this application embodiment;

[0025] Figure 2 A flowchart illustrating another optional multi-temporal remote sensing image patch spatiotemporal correlation analysis method provided in this application embodiment;

[0026] Figure 3 A schematic diagram of a patch sequence for an optional multi-temporal remote sensing image patch spatiotemporal correlation analysis method provided in this application embodiment;

[0027] Figure 4 A schematic diagram of a patch sequence for another optional multi-temporal remote sensing image patch spatiotemporal correlation analysis method provided in this application embodiment;

[0028] Figure 5 A schematic diagram of a patch sequence for another optional multi-temporal remote sensing image patch spatiotemporal correlation analysis method provided in this application embodiment;

[0029] Figure 6 A schematic diagram of the mosaic evolution of an optional multi-temporal remote sensing image mosaic spatiotemporal correlation analysis method provided in this application embodiment;

[0030] Figure 7 This is a schematic diagram of an optional electronic device provided in an embodiment of this application. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0032] The terms "first," "second," "third," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0033] According to one aspect of the embodiments of this application, a method for spatiotemporal correlation analysis of multi-temporal remote sensing image patches is provided. The following is in conjunction with... Figure 1 This application describes a method for spatiotemporal correlation analysis of multi-temporal remote sensing image patches provided in its embodiments.

[0034] Figure 1 This is a flowchart illustrating an optional multi-temporal remote sensing image patch spatiotemporal correlation analysis method provided in an embodiment of this application, as shown below. Figure 1 As shown, the process of this method may include the following steps:

[0035] S102, acquire multi-temporal remote sensing images of the same monitoring area, extract target-type change patches from each temporal image, and form a set of target-type change patches arranged in chronological order;

[0036] S104, Construct a spatiotemporal association rule and similarity measurement model to calculate the comprehensive association confidence that any two target class change patches from different time phases belong to the same land cover;

[0037] S106, using the target class change patches of each time phase as nodes and the association relationships constructed based on the comprehensive association confidence as edges, construct a spatiotemporal association graph;

[0038] S108. Based on the spatiotemporal association map, the optimal association path is found through graph optimization algorithm. Multiple map patches belonging to the same land feature and distributed in different time phases are associated and integrated to generate a spatiotemporal land feature object representing the complete life cycle of a single land feature. The spatiotemporal land feature object and its association information are output.

[0039] This application provides a method for spatiotemporal correlation analysis of multi-temporal remote sensing image patches, which can be applied to scenarios that require process-oriented analysis of land surface changes, such as natural resources, agriculture, forestry, transportation, and other fields. This embodiment takes the identification, tracking, and spatiotemporal correlation analysis of land surface changes as an example when the target type is a construction project.

[0040] Combination Figure 1 and Figure 2As shown, firstly, a multi-temporal change map dataset is constructed to prepare high-quality input data for subsequent spatiotemporal correlation analysis, that is, to obtain a set of construction-related change maps of the same monitoring area at different time points.

[0041] S0, Data Preparation: Acquire multiple high-resolution remote sensing images covering the same monitoring area, arranged in a time series. Simultaneously, acquire a vector map of the area.

[0042] S1, Change Patch Extraction: For consecutive temporal image pairs, a deep learning-based change detection model is used to calculate the probability of surface change and generate a binary change mask through threshold segmentation. This mask identifies the areas where changes have occurred. For each binary change mask, a multi-resolution segmentation algorithm is applied to divide the changed areas into independent, homogeneous objects, which are then vectorized to obtain a set of change patches.

[0043] Not all changed patches represent construction activities (e.g., water level fluctuations or vegetation harvesting). To focus on construction projects, this embodiment utilizes a pre-trained classifier (such as a random forest) to calculate the change for each patch. Based on the patch's spectral, shape, and texture features, the classifier outputs a confidence score indicating whether it belongs to the construction category. Patches with scores higher than a preset threshold are selected to form a candidate set of construction patches. Finally, a series of patches arranged chronologically is obtained.

[0044] S2, Constructing a spatiotemporal association rule and similarity measurement model: Calculating the probability that any two map patches from different periods belong to the same construction project, i.e., the comprehensive association confidence score. This model integrates multiple measurement factors and follows strict spatiotemporal constraints.

[0045] For example, the various measurement factors include spatiotemporal proximity factors and morphological similarity factors. Spatiotemporal proximity factors include center point distance similarity and spatial overlap; morphological similarity factors include area similarity and shape similarity. A weighted geometric mean is used to fuse the above four factors to calculate the comprehensive association confidence score.

[0046] Optionally, before performing the above calculations, spatiotemporal context constraints are applied, and the following hard rules are used for filtering: temporal unidirectionality rule (constructing directed associations), lifecycle continuity rule (associated patches should be roughly continuous in time to avoid incorrectly associating newly created patches with already stable old patches), and maximum search distance rule (setting an absolute distance threshold, patches exceeding this distance are not considered for association).

[0047] S3, Spatiotemporal Correlation Analysis: Constructing a spatiotemporal correlation graph organizes abstract correlation relationships into a graph structure that can be processed by the algorithm. All construction-type change features across all time phases are considered vertices, forming a vertex set. For any two features that satisfy certain conditions, a directed edge is established between their corresponding vertices, forming an edge set. The correlation confidence is converted into the cost of path search. The smaller the cost, the greater the correlation probability; edge weights are defined. Finally, a directed weighted graph, i.e., a spatiotemporal correlation graph, is formed, where the graph's topology (edges) reflects potential correlation relationships, and the edge weights reflect the correlation confidence.

[0048] To find the optimal path, a set of non-intersecting paths in the spatiotemporal association graph is sought, minimizing the total cost of all paths (i.e., maximizing the total association confidence). Optionally, this embodiment employs dynamic programming for efficient solution. The algorithm iterates backward from the latest time phase to the earliest time phase, progressively calculating the optimal successor and minimum cumulative cost for each vertex through state transition equations.

[0049] Furthermore, starting from each vertex that appears in the earliest time phase without a predecessor (which can be regarded as the origin of the project), tracing backwards yields a vertex sequence (path) from the earliest to the latest time phase. Each such path represents a unique spatiotemporal feature object spanning multiple time phases. This object fully records a construction project from its first detection, through its morphological and area evolution, to its latest state—a complete spatiotemporal snapshot.

[0050] S4, the main applications of correlation result output include:

[0051] Spatiotemporal object layer: Each object is represented by its latest phase patch and assigned a unique ID, which is the main body of spatial query.

[0052] Spatiotemporal correlation line layer: Visually displays the lines connecting the center points of adjacent temporal patches within the same object, intuitively presenting the evolution path. For example... Figure 6 The evolution of the pattern is illustrated in the figure.

[0053] Association attribute table: records the ID of each object, the IDs of all contained polygons, the area sequence, and the key attributes of initial equality.

[0054] Visualization and Analysis: On a Geographic Information System (GIS) platform, render patches of the same object at different times using colors of the same color family but different brightness levels to achieve time-series animation playback. For example... Figure 3 , Figure 4 , Figure 5As shown, the map displays the same construction project at three different time points. Furthermore, the resulting layer is overlaid with operational layers such as river management scope layers. The system can automatically execute rules, providing precise targeting for on-site verification and greatly improving regulatory efficiency.

[0055] Through the above steps, multi-temporal remote sensing images of the same monitoring area are acquired, and target-type change patches are extracted from each temporal image to form a set of target-type change patches arranged in chronological order. A spatiotemporal association rule and similarity measurement model are constructed to calculate the comprehensive association confidence that any two target-type change patches from different temporal phases belong to the same land feature. Using the target-type change patches from each temporal phase as nodes and the association relationships constructed based on the comprehensive association confidence as edges, a spatiotemporal association graph is constructed. Based on the spatiotemporal association graph, a graph optimization algorithm is used to find the optimal association path, integrating multiple patches belonging to the same land feature and distributed across different temporal phases to generate a spatiotemporal land feature object representing the complete lifecycle of a single land feature. The spatiotemporal land feature object and its association information are then output. This process automatically connects isolated patches from different temporal phases into a continuous spatiotemporal land feature object, fully reproducing the entire lifecycle of each construction project from commencement and construction to its final form, thereby improving the automation and intelligence level of remote sensing dynamic monitoring.

[0056] In an exemplary embodiment, the step of acquiring multi-temporal remote sensing images of the same monitoring area, extracting target-type change patches from each temporal image, and forming a set of target-type change patches arranged in chronological order includes:

[0057] Change detection is performed on images of consecutive time phases in the multi-temporal remote sensing images to generate a binary change mask;

[0058] The binary change mask is segmented using an object-oriented approach to obtain change pattern sets for each time phase;

[0059] The target class of change patches is selected from the set of change patches using a classifier to form the target class of change patch set.

[0060] In this embodiment, the construction of a multi-temporal change patch dataset aims to accurately extract change patches belonging to construction projects in each temporal phase from the original remote sensing image sequence and assign them initial attributes for subsequent correlation analysis.

[0061] Optionally, acquire multiple high-resolution remote sensing images covering the same area, arranged in a time series. ,in Let T represent the image at time t, where T is the total number of time phases. To identify geographic areas that have changed significantly between two consecutive time phases, deep learning-based change detection models, such as the Bitemporal Image Transformer (BIT), the Transformer-based Change Detection Network (ChangeFormer), and the Siamese Nested U-Net (SNUNet), are used to better capture complex features such as texture and context, outperforming traditional pixel-level interpolation methods.

[0062] Change detection is performed on continuous temporal images to extract all changed surface patches. A classifier (such as support vector machine, random forest, or deep learning model) is then used to distinguish changed patches into construction-related categories and other categories (such as vegetation destruction, water body changes, etc.). This embodiment will illustrate the processing of construction-related patches as an example.

[0063] For two consecutive time phases after registration and The image, for the change detection function Extracting changed features:

[0064]

[0065] in It is a binary transformation mask.

[0066] right Apply segmentation algorithms (such as multi-resolution segmentation) to generate a set of image patches:

[0067]

[0068] in Indicates phase The A polygon vector of a map feature. This represents the total number of phase patches at that time.

[0069] Furthermore, using a classifier (For example, based on random forest or deep network) Calculate the probability that each patch belongs to the construction class, and the classifier selection expression is:

[0070]

[0071] in , , These are the spectral, shape, and texture feature extraction functions, respectively. Filtering threshold. These are the classifier parameters (such as the set of decision trees in a random forest).

[0072] Candidate graph patch set for construction at time t: .

[0073] In an exemplary embodiment, calculating the comprehensive association confidence that any two target class change patches from different time phases belong to the same land cover includes:

[0074] The comprehensive association confidence score is calculated based on the fusion of spatiotemporal proximity factor and morphological similarity factor;

[0075] The spatiotemporal proximity factor includes distance similarity calculated based on the distance between the geometric center points of two patches and spatial overlap calculated based on the ratio of the intersection to the union area of ​​two patches;

[0076] The morphological similarity factor includes area similarity calculated based on the reasonableness of the area change trend of the two patches and shape similarity calculated based on the similarity of the shape features of the two patches.

[0077] In this embodiment, the similarity measurement model integrates measurement factors such as spatiotemporal proximity, morphological similarity, and spatiotemporal context constraints.

[0078] For example, in a multi-factor fusion model, let... and Phases and ( < The model uses two construction class vector polygons to form a comprehensive association confidence level. It is determined by a combination of metrics such as spatiotemporal proximity, morphological similarity, and spatiotemporal context constraints. The higher the value, the better. and The greater the probability, the more likely it is that the same construction project exists at different stages of construction.

[0079] The same land feature exhibits high continuity and stability in spatial location. Even with morphological changes, its geographical location should highly overlap or be adjacent. The spatiotemporal proximity factor is obtained by calculating the similarity of the center point distance and the spatial overlap.

[0080] To calculate the similarity of the distance between the center points First, calculate the Euclidean distance between the geometric centers of the two patches:

[0081] For the image Geometric center point For the image Geometric center point.

[0082] Centroid distance similarity is related to distance; the closer the distance, the higher the probability of association. The formula for centroid distance similarity is:

[0083]

[0084] in, It is the Euclidean distance between the center points; This is the distance bandwidth parameter, which controls the rate at which similarity decays with distance. For example, setting... =30 meters, meaning that when the distance is 30 meters, the similarity drops to ≈0.6.

[0085] To calculate spatial overlap The intersection area of ​​the two polygons needs to be calculated. ) and the area of ​​union ( The ratio of spatial overlap. Even with changes in the shape of map features such as building expansion, their core parts usually overlap. Specifically, it is expressed as follows:

[0086]

[0087] The range of values ​​is [0,1], where 1 represents complete overlap and 0 represents no overlap.

[0088] Furthermore, construction projects typically progress from nothing to something, from small to large, and remain stable after construction. Therefore, the morphological evolution of map features at different stages of the same construction project should exhibit a reasonable pattern, rather than random mutations. Based on this, the rationality of area change trends can be analyzed. and shape similarity .

[0089] Calculate the map area( ) and pattern area( area ratio :

[0090]

[0091] A reasonable increase or small fluctuation (considering image segmentation error) assigns a high confidence level; while a sharp and irregular change in area reduces the confidence level. A truncated Gaussian function is used to simulate this reasonableness, with a minimum threshold set. This is to filter out unreasonable area shrinkage (such as a sharp reduction in area due to image segmentation errors).

[0092] Reasonableness of area change trend:

[0093]

[0094] in: This is the expected rate of change in area. It is usually set as... (e.g., 1.2) indicates that the expected project is growing, but for a stable situation ( We also maintain a certain degree of tolerance. Control the tolerance for the range of area fluctuations. The larger the area, the greater the tolerance for changes in area. This is the lower limit threshold for area shrinkage (e.g., 0.5). If the area is below this value, it is considered not to belong to the same land feature.

[0095] Although buildings may expand, their main outlines (such as rectangles, L-shapes, and T-shapes) typically retain certain shape characteristics. The difference in shape indices (such as compactness) between two patches is calculated. Optionally, compactness is used as a shape metric, measuring the tightness of the shape (1 for circles, close to 0 for irregular shapes), mathematically expressed as:

[0096]

[0097] in, For shape The roundness measure (compactness). For shape area, For shape The perimeter.

[0098] The shape similarity between the two patches is:

[0099]

[0100] in, It is the tolerance parameter for shape differences. The larger the value, the greater the range of shape variations allowed.

[0101] In an exemplary embodiment, the comprehensive association confidence score is obtained by calculating a weighted geometric mean of the distance similarity, spatial overlap, area similarity, and shape similarity.

[0102] The similarity scores from the multiple dimensions are combined into a single overall confidence score. We use a weighted geometric mean because it is highly sensitive to low scores (i.e., a veto effect), which aligns with our business logic—low similarity scores in any one dimension will significantly lower the overall confidence score. Its mathematical expression is:

[0103]

[0104] Specifically:

[0105]

[0106] in, , , , These are the weights corresponding to each factor, satisfying... Based on business experience, spatial overlap is the most important factor, as it is the most direct reflection of spatial consistency and can be set. .

[0107] In one exemplary embodiment, after constructing the spatiotemporal association rules and similarity measurement model, the method further includes:

[0108] The spatiotemporal context constraint factor is applied, which includes at least one of the following rules: association time series unidirectionality rule, life cycle continuity assumption rule, and maximum spatial search distance rule.

[0109] Preferably, a spatiotemporal context constraint factor (business rule injection) is used. The following rules do not directly contribute to the confidence score, but are used as filters before or after calculation to directly determine whether two patches are eligible for association. Specifically, these include:

[0110] (1) Temporal unidirectionality: The association only allows the mapping from the early phase to the late phase, thus constructing a directed network.

[0111] (2) Lifecycle continuity: Once a feature first appears, it should continue to exist in subsequent time phases (unless it is removed, in which case the association terminates). Avoid incorrectly associating newly created features with existing stable features.

[0112] (3) Maximum search distance rule: Set an absolute distance threshold, and no further calculation is required if the distance exceeds the threshold.

[0113] In summary, for all pairs of polygons that satisfy the spatiotemporal context constraints... A comprehensive association confidence level is calculated. This value will be used to construct a spatiotemporal correlation graph in order to find the optimal correlation path.

[0114] In an exemplary embodiment, constructing a spatiotemporal association graph using target class change patches of each time phase as nodes and association relationships built based on the comprehensive association confidence as edges includes:

[0115] A vertex set is constructed using the target class change patches of each time phase as vertices;

[0116] For two vertices in the vertex set that belong to different time phases and whose time sequence satisfies a preset direction, if the comprehensive correlation confidence between the two corresponding patches is higher than a preset edge existence threshold, then an edge is constructed between the two vertices.

[0117] Each edge is assigned a corresponding weight based on the comprehensive association confidence score, wherein the weight is determined based on the comprehensive association confidence score, and the weight of an edge is negatively correlated with the comprehensive association confidence score of that edge.

[0118] The spatiotemporal relation graph is formed by the set of vertices and the set of edges consisting of all the edges.

[0119] In this embodiment, a spatiotemporal correlation graph G(V,E) is constructed, where all construction-type patches of all time phases are regarded as nodes in the graph, and edges are patches connecting different time phases (only connections from earlier time phases to later time phases are allowed). Using the aforementioned constructed metric model, the weights of the edges between nodes in different time periods are calculated (the higher the confidence level, the smaller the weight).

[0120] Spatiotemporal Relationship Graph Vertex Set The expression for all construction class patches encompassing all time phases is:

[0121]

[0122] in, It represents the total number of polygons, per vertex. Corresponding to a specific image patch .

[0123] Edge set The expression for connecting patch pairs from different time phases with sufficiently high association confidence is:

[0124]

[0125] In the formula A threshold is set for edges to filter out associations with low confidence.

[0126] Border rights The association confidence is converted into a cost, which facilitates the use of a path minimization algorithm. The expression is:

[0127]

[0128] Thus, high-confidence associations correspond to low weights, and low-confidence associations correspond to high weights.

[0129] In one exemplary embodiment, generating a spatiotemporal land cover object representing the complete lifecycle of a single land cover includes:

[0130] On the spatiotemporal association graph, with the goal of minimizing the total path weight, a dynamic programming algorithm is used to solve for the optimal set of association paths covering all vertices;

[0131] A spatiotemporal feature object is generated for each of the optimal association paths.

[0132] In this embodiment, an optimal path search is performed to generate spatiotemporal feature objects. Optionally, a path from the earliest to the latest time phase is found such that the sum of the weights of all edges on the path is minimized (i.e., the overall association confidence is maximized). However, a feature may have multiple candidate patches in different time phases, so an optimal continuous path needs to be found for each feature.

[0133] For example, the problem is modeled as a minimum-cost path problem. Due to the graph's structure (plots within each time phase are not connected, only plots from different time phases are connected), a graph-theoretic algorithm, dynamic programming, is used to solve it. For each plot terminating in the latest time phase, an optimal association path is found that traces back from it to its origin time phase. All plots on this path are considered to be the forms of the same construction type feature at different times.

[0134] Define the state of dynamic programming as , indicating from The cumulative cost of the best subsequent path from the start to the latest phase.

[0135] Optimize the objective function to find a set of non-intersecting paths that cover all vertices and minimize the total path weight, i.e. maximize the total association confidence. Given a set of paths in the graph where the vertices do not intersect (i.e., each path has distinct vertices), covering all vertices. The optimization objective is to find the set of paths that minimizes the total association cost (i.e., maximizes the total association confidence).

[0136]

[0137] The constraints are:

[0138] The above problem is a nondeterministic polynomial time (NP) problem. An efficient approximation algorithm based on dynamic programming is used to process the problem in reverse order.

[0139] For the current image patch Examine all possible successor patches The choice makes from arrive The cost plus from The one with the minimum total cost to reach the destination. As the optimal successor, the expression is:

[0140]

[0141] Among them, candidate set Dynamic programming state transition equation .

[0142] Starting from the vertex of each phase 1, along... By backtracking the pointer, the optimal path can be obtained, and the output is a collection containing all reconstructed spatiotemporal feature objects. Each successfully traced path generates a unique spatiotemporal feature object. Each object fully describes the entire spatiotemporal evolution of a construction project from its inception to its latest state, including all map instances, temporal information, and evolution attributes (such as cumulative area change curves, construction speed, etc.) throughout its entire lifecycle.

[0143] This step integrates discrete remote sensing change patches from different periods into ground features with a continuous life cycle.

[0144] When outputting correlation results, all results of spatiotemporal correlation analysis are stored in a standardized, machine-readable format, facilitating access and querying by GIS platforms, databases, and business systems.

[0145] The spatiotemporal correlation map results are output in the form of GIS layers and correlation tables. The GIS layers include a spatiotemporal feature object layer and a spatiotemporal correlation line layer. The spatiotemporal feature object layer contains the latest phase of each feature object's polygon, representing the latest state of each construction project and serving as the primary carrier for spatial queries and statistical analysis. The spatiotemporal correlation line layer contains lines connecting the center points of adjacent phase polygons within the same feature object, visually demonstrating the evolution path of features in space and time. The correlation table records the ID of each spatiotemporal feature object and the IDs of all phase polygons it contains.

[0146] The system provides visualization capabilities, performing time-series rendering through a GIS platform. It renders selected geographic features at different time phases using colors of the same color family but different brightness or saturation, visually showcasing their spatiotemporal evolution. A layer of correlation lines is overlaid on the map, with line thickness or dashed lines indicating the level of correlation confidence, forming a spatiotemporal correlation map.

[0147] The system provides statistical analysis functions, extracting quantitative indicators from spatiotemporal features to assess and monitor the overall status of regional construction activities. It analyzes the total land area, commencement time (first appearance), construction period, and latest status of any construction project. It also statistically analyzes the number of newly emerging projects (suspected starts) within a specified time period and the average area expansion rate of all projects that continuously expand during the monitoring period. The system quickly filters out features exhibiting specific patterns, such as continuous expansion or sudden new appearances during the monitoring period. By combining spatiotemporal analysis results with business rules, it automatically screens for suspected violations, such as identifying and judging suspected unauthorized construction of new projects within river management areas and identifying and judging suspected discrepancies between approved and actual construction of continuously and rapidly expanding projects.

[0148] In another alternative embodiment, a typical river and lake management business scenario—dynamic monitoring of illegal construction activities within the river management area—is used for illustration. This embodiment demonstrates the entire process from data preparation to business application.

[0149] I. Data Preparation

[0150] Select high-resolution visible light satellite imagery from 2019 onwards, vector maps of river and lake management boundary delineation results, and ledgers of compliant river-related construction projects (for model verification, sample production, and compliance assessment after the introduction of business rules).

[0151] II. Extraction of Construction-Related Change Map Patches

[0152] Samples were pre-trained for typical construction project types. A deep learning change detection model based on Siamese neural networks was used to binarize the change probability map for continuous temporal image pairs since 2019, generating initial change masks. A multi-resolution segmentation algorithm was used to segment each binary change mask, and the results were converted into vector patches. Feature extraction was performed on each vector patch to form a candidate patch set for each construction type in each temporal phase.

[0153] III. Spatiotemporal Correlation Analysis

[0154] Initialize the system and import the construction class map vector data for each time phase.

[0155] Set association rule parameters, including: center point distance threshold set to 50 meters, spatial overlap weight set to 0.4, area change trend weight set to 0.3, and morphological similarity weight set to 0.3, etc.

[0156] Initiate correlation analysis. The system reads all temporal patches, using each patch as a vertex, and then... Create edges, edge weights .

[0157] Iterate backward from the latest temporal imagery to 2019, calculating the optimal successor and minimum cumulative cost for each patch. Starting with patches from 2019, extract the complete association path based on the Next pointer.

[0158] Short paths are filtered out, and isolated polygons with path length L < 2 are ignored, as they are considered unable to form a valid lifecycle. For one-to-many split cases (such as a large construction site divided into two independent buildings), the two edges with the highest confidence are retained, and two associated sub-paths are generated.

[0159] Output a collection of spatiotemporal objects, containing 125 valid construction project objects.

[0160] IV. Generation Results

[0161] Group the spatiotemporal feature object layers by object ID, and render different periods of the same object using the same color scheme. Overlay a spatiotemporal correlation line layer, with line thickness proportional to the correlation confidence. Place the river management area line at the top layer.

[0162] The resulting maps and attribute data will be published as GIS map services and integrated into the existing water conservancy map for online query by relevant personnel.

[0163] By integrating analysis with business rules, the system automatically identified three suspected projects that had commenced construction without prior approval, generating a list of potential issues. Further verification was then conducted through on-site inspections and drone patrols.

[0164] This embodiment enables precise spatiotemporal correlation of construction-related change patches extracted from remote sensing images at different times, constructing a spatiotemporal correlation network for the same construction project. This provides strong technical support for project supervision, law enforcement evidence collection, and trend analysis, and improves the automation and intelligence level of remote sensing dynamic monitoring.

[0165] According to another aspect of the embodiments of this application, a multi-temporal remote sensing image patch spatiotemporal correlation analysis apparatus for implementing the above-described multi-temporal remote sensing image patch spatiotemporal correlation analysis method is also provided. The apparatus may include:

[0166] The acquisition unit is used to acquire multi-temporal remote sensing images of the same monitoring area, extract target-type change patches from each temporal image, and form a set of target-type change patches arranged in chronological order.

[0167] The computing unit is used to construct spatiotemporal association rules and similarity measurement models, and to calculate the comprehensive association confidence that any two target class change patches from different time phases belong to the same land cover.

[0168] The construction unit is used to construct a spatiotemporal association graph using the target class change patches of each time phase as nodes and the association relationships constructed based on the comprehensive association confidence as edges.

[0169] The output unit is used to find the optimal association path based on the spatiotemporal association map using a graph optimization algorithm, to associate and integrate multiple map patches belonging to the same land feature but distributed in different time phases, to generate a spatiotemporal land feature object representing the complete life cycle of a single land feature, and to output the spatiotemporal land feature object and its association information.

[0170] According to another aspect of the embodiments of this application, a storage medium is also provided. Optionally, in this embodiment, the storage medium can be used to execute the program code of any of the multi-temporal remote sensing image patch spatiotemporal correlation analysis methods described in the embodiments of this application.

[0171] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps:

[0172] S1. Acquire multi-temporal remote sensing images of the same monitoring area, extract target-type change patches from each temporal image, and form a set of target-type change patches arranged in chronological order.

[0173] S2, construct spatiotemporal association rules and similarity measurement models, and calculate the comprehensive association confidence that any two target class change patches from different time phases belong to the same land cover;

[0174] S3, using the target class change patches of each time phase as nodes and the association relationships constructed based on the comprehensive association confidence as edges, construct a spatiotemporal association graph;

[0175] S4. Based on the spatiotemporal association map, the optimal association path is found through graph optimization algorithm. Multiple map patches belonging to the same land feature but distributed in different time phases are associated and integrated to generate a spatiotemporal land feature object representing the complete life cycle of a single land feature. The spatiotemporal land feature object and its association information are output.

[0176] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated in this embodiment.

[0177] The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0178] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described multi-temporal remote sensing image patch spatiotemporal correlation analysis method is also provided. The electronic device may be a server, a terminal, or a combination thereof.

[0179] Figure 7 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application, such as... Figure 7 As shown, it includes a processor 702, a communication interface 704, a memory 706, and a communication bus 708. The processor 702, communication interface 704, and memory 706 communicate with each other via the communication bus 708.

[0180] Memory 706 is used to store computer programs;

[0181] When processor 702 executes a computer program stored in memory 706, it performs the following steps:

[0182] S1. Acquire multi-temporal remote sensing images of the same monitoring area, extract target-type change patches from each temporal image, and form a set of target-type change patches arranged in chronological order.

[0183] S2, construct spatiotemporal association rules and similarity measurement models, and calculate the comprehensive association confidence that any two target class change patches from different time phases belong to the same land cover;

[0184] S3, using the target class change patches of each time phase as nodes and the association relationships constructed based on the comprehensive association confidence as edges, construct a spatiotemporal association graph;

[0185] S4. Based on the spatiotemporal association map, the optimal association path is found through graph optimization algorithm. Multiple map patches belonging to the same land feature but distributed in different time phases are associated and integrated to generate a spatiotemporal land feature object representing the complete life cycle of a single land feature. The spatiotemporal land feature object and its association information are output.

[0186] Optionally, the communication bus can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 The symbol is represented by a single thick line, but this does not indicate that there is only one bus or one type of bus. The communication interface is used for communication between the aforementioned electronic device and other devices.

[0187] The memory may include RAM, or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0188] The processors mentioned above can be general-purpose processors, including but not limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; they can also be DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0189] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0190] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0191] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0192] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0193] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of embodiments of this disclosure upon considering the specification and practicing the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described herein. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

[0194] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0195] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for spatiotemporal correlation analysis of multi-temporal remote sensing image patches, characterized in that, include: Acquire multi-temporal remote sensing images of the same monitoring area, extract target-type change patches from each temporal image, and form a set of target-type change patches arranged in chronological order; Construct spatiotemporal association rules and similarity measurement models to calculate the comprehensive association confidence that any two target class change patches from different time phases belong to the same land cover. Using the target class change patches of each time phase as nodes and the association relationships constructed based on the comprehensive association confidence as edges, a spatiotemporal association graph is constructed. Based on the spatiotemporal association map, the optimal association path is found through graph optimization algorithm. Multiple map patches belonging to the same land feature but distributed in different time phases are associated and integrated to generate a spatiotemporal land feature object representing the complete life cycle of a single land feature. The spatiotemporal land feature object and its association information are then output.

2. The spatiotemporal correlation analysis method for multi-temporal remote sensing image patches as described in claim 1, characterized in that, The process of acquiring multi-temporal remote sensing images of the same monitoring area, extracting target-type change patches from each temporal image, and forming a set of target-type change patches arranged in chronological order includes: Change detection is performed on images of consecutive time phases in the multi-temporal remote sensing images to generate a binary change mask; The binary change mask is segmented using an object-oriented approach to obtain change pattern sets for each time phase; The target class of change maps is selected from the set of change maps using a classifier to form the set of target class change maps.

3. The spatiotemporal correlation analysis method for multi-temporal remote sensing image patches as described in claim 1, characterized in that, The calculation of the comprehensive association confidence that any two target class change patches from different time phases belong to the same land cover includes: The comprehensive association confidence score is calculated based on the fusion of spatiotemporal proximity factor and morphological similarity factor; The spatiotemporal proximity factor includes distance similarity calculated based on the distance between the geometric center points of two patches and spatial overlap calculated based on the ratio of the intersection to the union area of ​​two patches; The morphological similarity factor includes area similarity calculated based on the reasonableness of the area change trend of the two patches and shape similarity calculated based on the similarity of the shape features of the two patches.

4. The spatiotemporal correlation analysis method for multi-temporal remote sensing image patches as described in claim 3, characterized in that, The overall association confidence score is calculated by weighted geometric mean of the distance similarity, spatial overlap, area similarity, and shape similarity.

5. The spatiotemporal correlation analysis method for multi-temporal remote sensing image patches as described in claim 1, characterized in that, After constructing the spatiotemporal association rules and similarity measurement model, the method further includes: The spatiotemporal context constraint factor is applied, which includes at least one of the following rules: association time series unidirectionality rule, life cycle continuity assumption rule, and maximum spatial search distance rule.

6. The spatiotemporal correlation analysis method for multi-temporal remote sensing image patches as described in claim 1, characterized in that, The construction of a spatiotemporal correlation graph, using target class change patches from each time phase as nodes and correlation relationships constructed based on the comprehensive correlation confidence as edges, includes: A vertex set is constructed using the target class change patches of each time phase as vertices; For two vertices in the vertex set that belong to different time phases and whose time sequence satisfies a preset direction, if the comprehensive correlation confidence between the two corresponding patches is higher than a preset edge existence threshold, then an edge is constructed between the two vertices. Each edge is assigned a corresponding weight based on the comprehensive association confidence score, wherein the weight is determined based on the comprehensive association confidence score, and the weight of an edge is negatively correlated with the comprehensive association confidence score of that edge. The spatiotemporal relation graph is formed by the set of vertices and the set of edges consisting of all the edges.

7. The spatiotemporal correlation analysis method for multi-temporal remote sensing image patches as described in claim 6, characterized in that, The generated spatiotemporal ground feature objects representing the complete lifecycle of a single ground feature include: On the spatiotemporal association graph, with the goal of minimizing the total path weight, a dynamic programming algorithm is used to solve for the optimal set of association paths covering all vertices; A spatiotemporal feature object is generated for each of the optimal association paths.

8. A device for spatiotemporal correlation analysis of multi-temporal remote sensing image patches, performing the spatiotemporal correlation analysis method for multi-temporal remote sensing image patches as described in any one of claims 1-7, characterized in that, include: The acquisition unit is used to acquire multi-temporal remote sensing images of the same monitoring area, extract target-type change patches from each temporal image, and form a set of target-type change patches arranged in chronological order. The computing unit is used to construct spatiotemporal association rules and similarity measurement models, and to calculate the comprehensive association confidence that any two target class change patches from different time phases belong to the same land cover. The construction unit is used to construct a spatiotemporal association graph using the target class change patches of each time phase as nodes and the association relationships constructed based on the comprehensive association confidence as edges. The output unit is used to find the optimal association path based on the spatiotemporal association map using a graph optimization algorithm, to associate and integrate multiple map patches belonging to the same land feature but distributed in different time phases, to generate a spatiotemporal land feature object representing the complete life cycle of a single land feature, and to output the spatiotemporal land feature object and its association information.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs a spatiotemporal correlation analysis method for multi-temporal remote sensing image patches according to any one of claims 1 to 7.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute, through the computer program, a method for spatiotemporal correlation analysis of multi-temporal remote sensing image patches according to any one of claims 1 to 7.

Images