Urban illegal building expansion remote sensing change monitoring method based on three-flow graph neural network

Abstract

This invention discloses a remote sensing method for monitoring urban illegal construction expansion based on a three-flow graph neural network, belonging to the field of urban management technology. The method includes: acquiring multi-source image data and preprocessing it to obtain a standard image dataset; constructing a heterogeneous graph structure and inputting it into a neural network branch to obtain multi-flow feature representations; fusing these multi-flow feature representations through cross-modal graph dynamic fusion operations to generate cross-modal fused graph feature representations; constructing structural anchoring relationships using pre-acquired historical image sequences and aligning the graph node sets in the cross-modal fused graph feature representations to obtain a time-consistent graph structure; constructing and training a neural network model to generate a node-level probability graph of illegal construction changes, acquiring illegal construction change information, and generating urban illegal construction alarm records. This invention, through the heterogeneous graph structure, fundamentally improves the multi-dimensional information characterization capability of the original image data.

Description

Technical Field

[0001] This invention relates to the field of urban management technology, and more specifically, to a remote sensing method for monitoring changes in the expansion of illegal urban buildings based on a three-flow graph neural network. Background Technology

[0002] With the improvement of remote sensing image resolution and the development of graph neural networks, the identification and supervision of illegal construction in urban construction is gradually moving towards intelligence and automation. In the practical application of urban management, monitoring the expansion of illegal construction is an important part of urban and rural planning law enforcement. Its core objective is to efficiently and accurately identify changes in buildings that are newly added, demolished, or remain unchanged in urban space. However, most of the current mainstream remote sensing monitoring methods for illegal construction in cities are based on single-modal image processing technology or static graph structure analysis, which have obvious technical limitations.

[0003] Most existing remote sensing analysis methods extract features independently from optical images or synthetic aperture radar images, lacking the ability to complement the structural semantics between modes. This leads to information fragmentation problems in the process of multi-source data fusion. For example, traditional methods are mostly based on pixel layer change detection algorithms or simple deep convolutional networks for time series change recognition. Although they can play a role in some scenarios, they cannot simultaneously take into account the dynamic relationship between building material, geometry and microwave scattering, which is prone to false detection or false negative.

[0004] Furthermore, existing graph neural networks generally rely on static nodes and fixed edge weights when constructing remote sensing map structures, making it difficult to adapt to the rapid changes in urban illegal construction areas over time. Especially when processing multi-temporal remote sensing image alignment and change evolution analysis, they often ignore the dynamic consistency of historical states between graph nodes. More importantly, most methods fail to effectively integrate prior knowledge of height restrictions, floor area ratio, and land use attributes in urban and rural planning regulations, resulting in a lack of compliance guidance in actual judgment and making it difficult to support law enforcement agencies in judging the legality of illegal construction.

[0005] Therefore, there is an urgent need for a comprehensive graph learning method that incorporates multi-source remote sensing, multi-scale features, multi-temporal consistency, and regulatory guidance to improve the accuracy and practicality of remote sensing monitoring of urban illegal construction expansion.

[0006] No effective solutions have yet been proposed to address the problems in the relevant technologies. Summary of the Invention

[0007] To address the problems in related technologies, this invention proposes a remote sensing method for monitoring changes in urban illegal construction expansion based on a three-flow graph neural network, in order to overcome the aforementioned technical problems existing in existing related technologies.

[0008] Therefore, the specific technical solution adopted by the present invention is as follows:

[0009] A remote sensing method for monitoring changes in urban illegal construction expansion based on a three-flow graph neural network, the method comprising the following steps:

[0010] S1. Acquire multi-source image data and preprocess the multi-source image data to obtain a standard image dataset;

[0011] S2. Construct a heterogeneous graph structure based on a standard image dataset, and input the heterogeneous graph structure into a neural network branch to obtain multi-stream feature representations. Then, fuse the multi-stream feature representations through a cross-modal graph dynamic fusion operation to generate a cross-modal fused graph feature representation.

[0012] S3. Construct structural anchoring relationships using pre-acquired historical image sequences, and align the graph node sets in the cross-modal fusion graph feature representation according to the structural anchoring relationships to obtain a temporally consistent graph structure;

[0013] S4. Construct and train a neural network model through a time-consistent graph structure, use the trained neural network model to generate a node-level illegal construction change probability map, obtain illegal construction change information based on the node-level illegal construction change probability map, and generate urban illegal construction alarm records based on the illegal construction change information.

[0014] Furthermore, in order to obtain a standard image dataset, preprocessing steps including geometric registration, radiometric normalization, and common coordinate system resampling are performed to standardize multi-source image data, obtain multi-source image data, and preprocess the multi-source image data to obtain a standard image dataset including:

[0015] S11. Acquire multi-source image data and perform geometric registration on the multi-source image data to obtain aligned multi-source image data;

[0016] S12. Radiometric difference elimination is performed on the aligned multi-source image data using radiometric normalization to obtain multi-source image data after difference elimination.

[0017] S13. After eliminating the differences, the multi-source image data are uniformly resampled to a common coordinate system to obtain a standard image dataset.

[0018] Furthermore, to improve the accuracy of remote sensing monitoring of urban illegal construction expansion, and to enhance the ability of multimodal structural semantic complementarity through the generation of cross-modal fusion map feature representations, while avoiding information fragmentation during multi-source data fusion, a heterogeneous map structure is constructed based on a standard image dataset. This heterogeneous map structure is then input into a neural network branch to obtain multi-stream feature representations. The multi-stream feature representations are then fused through a dynamic cross-modal map fusion operation to generate a cross-modal fusion map feature representation, including:

[0019] S21. Construct heterogeneous graph structures based on standard image datasets to obtain spectral flow graph structures, spatial flow graph structures, and radar flow graph structures;

[0020] S22. Input the spectral flow graph structure, spatial flow graph structure and radar flow graph structure into the three-flow graph neural network branch respectively to obtain the spectral flow feature representation, spatial flow feature representation and radar flow feature representation;

[0021] S23. By performing a cross-modal graph dynamic fusion operation inside the three-flow graph neural network, the spectral flow feature representation, spatial flow feature representation, and radar flow feature representation are fused to generate a cross-modal fused graph feature representation.

[0022] Furthermore, to enhance the ability to characterize multidimensional information and improve the ability to analyze the evolution of multimodal graph structures by connecting edges to the graph node set, thereby improving the monitoring capability of urban illegal construction areas, a heterogeneous graph structure is constructed based on a standard image dataset, resulting in spectral flow graph structure, spatial flow graph structure, and radar flow graph structure, including:

[0023] S211. Based on the image modal types of the standard image dataset, the graph node set is divided to obtain the spectral flow graph node set, the spatial flow graph node set, and the radar flow graph node set;

[0024] S212. Calculate the spectral angle similarity between spectral feature vectors based on the spectral flow graph node set, and use the spectral angle similarity combined with the preset spectral edge connection threshold to construct spectral edge connection relationships. Generate the spectral flow graph structure based on the spectral edge connection relationships and the spectral flow graph node set.

[0025] S213. Obtain the Euclidean distance between the geometric centroids of the corresponding building outlines using the spatial flow graph node set, and construct the spatial flow graph edge connection relationship based on the Euclidean distance and the preset adjacency threshold. Generate the spatial flow graph structure based on the spatial flow graph edge connection relationship and the spatial flow graph node set.

[0026] S214. Calculate the radar backscattering similarity based on the average backscattering coefficient of the radar patch region in the radar flow graph node set, and use the radar backscattering similarity to construct the radar flow graph edge connection relationship. Generate the radar flow graph structure based on the radar flow graph edge connection relationship and the radar flow graph node set.

[0027] Furthermore, to extract multi-scale features, and with the acquisition of spectral flow feature representation, spatial flow feature representation, and radar flow feature representation, the influence of neighboring nodes on target nodes can be dynamically adjusted, and fine-grained changes in building morphology and structure during illegal construction expansion can be captured. Simultaneously, the sensitivity to changes in radar scattering characteristics during urban illegal construction expansion is optimized. The spectral flow graph structure, spatial flow graph structure, and radar flow graph structure are respectively input into the three-flow graph neural network branches to obtain the spectral flow feature representation, spatial flow feature representation, and radar flow feature representation, including:

[0028] S221. Input the spectral flow graph structure into the spectral branch of the three-flow graph neural network, and introduce a spectral attention graph convolutional unit with non-constructed spectral drift to update the features of the nodes in the spectral flow graph structure to obtain the spectral flow feature representation.

[0029] S222. Input the spatial flow graph structure into the spatial flow graph branch of the three-flow graph neural network, and introduce a spatial gated graph convolutional unit enhanced by illegal geometric topology changes to update the features of the nodes of the spatial flow graph structure, thereby obtaining the spatial flow feature representation;

[0030] S223. Input the radar flow graph structure into the radar flow graph branch of the three-flow graph neural network, and introduce the radar flow residual graph convolution unit with dynamic adjustment of the backscattering structure of the illegal construction area to update the features of the nodes of the radar flow graph structure, so as to obtain the radar flow feature representation.

[0031] Furthermore, to ensure the monitoring method exhibits multi-temporal consistency and enables structural registration and feature alignment of graph nodes under the influence of structural anchoring relationships, significantly improving the adaptability and robustness of time-varying graph structures in cross-temporal change detection tasks, structural anchoring relationships are constructed using pre-acquired historical image sequences. Based on these relationships, the graph node sets in the cross-modal fusion graph feature representation are aligned, resulting in a time-consistent graph structure including:

[0032] S31. By utilizing the consistency of the geographical location of the same spatial entity at different time points in a pre-acquired historical image sequence, construct a structural anchoring relationship between time points;

[0033] S32. Align the graph node set in the cross-modal fusion graph feature representation according to the structural anchoring relationship to obtain the aligned fusion node set and cross-time edge connections;

[0034] S33. Utilize the aligned fused node set and cross-time edge connections to construct the graph structure and obtain a time-consistent graph structure.

[0035] Furthermore, to achieve the legality assessment of illegal constructions, and to enhance the monitoring methods' ability to determine the legality of illegal constructions through the generation of urban illegal construction alarm records, a neural network model is constructed and trained using a time-series consistent graph structure. This trained neural network model generates a node-level illegal construction change probability map, and illegal construction change information is obtained based on this map. The generation of urban illegal construction alarm records based on this change information includes:

[0036] S41. Construct a contrastive learning framework based on the temporally consistent graph structure, and use the contrastive learning framework to construct and train a three-flow graph neural network model.

[0037] S42. Use the trained three-flow graph neural network model to generate cross-modal fusion graph feature representations embedded with regulatory knowledge, and infer the node-level illegal construction change probability map from the cross-modal fusion graph feature representations.

[0038] S43. Perform conditional random field post-processing on the node-level illegal construction change probability map to obtain three types of illegal construction change information, and generate urban illegal construction alarm records based on the illegal construction change information.

[0039] Furthermore, to train the three-flow graph neural network model and improve the accuracy of obtaining the probability map of illegal construction changes under the influence of the joint loss function on the parameter updates of the three-flow graph neural network model, a contrastive learning framework is constructed based on the temporally consistent graph structure. The three-flow graph neural network model is then constructed and trained using this contrastive learning framework, including:

[0040] S411. Based on the temporally consistent graph structure, a contrastive learning framework is introduced. The contrastive learning framework is used to construct positive and negative sample pairs and model similarity constraints on the node feature representation of the temporally consistent graph structure, and the contrastive learning loss function is obtained.

[0041] S412. Using the pre-acquired unlabeled image data, set a pseudo-label generation function in the temporal consistent graph structure, generate pseudo-labels according to the pseudo-label generation function, and construct a pseudo-label supervision loss function through the pseudo-labels.

[0042] S413. Combine the pseudo-label supervision loss function and the contrastive learning loss function to obtain the joint loss function. Optimize the parameters of the three-flow graph neural network model using the joint loss function to obtain the optimized three-flow graph neural network model.

[0043] S414. Train the optimized three-flow graph neural network model, embed urban and rural planning regulations into the three-flow graph edge weight calculation process, and perform regulatory correction on the edge connection weights in the time-series consistent graph structure to obtain the trained three-flow graph neural network model.

[0044] Furthermore, in order to obtain the node-level illegal construction change probability map, the graph feature representation can be mapped to the illegal construction change category space under the mapping effect of the node-level category discrimination model, thereby obtaining the node-level illegal construction change category probability set. A trained three-flow graph neural network model is used to generate a cross-modal fusion graph feature representation embedding legal knowledge, and reasoning is performed on the cross-modal fusion graph feature representation to obtain the node-level illegal construction change probability map, including:

[0045] S421. Generate a cross-modal fusion graph feature representation with embedded regulatory knowledge based on the trained three-flow graph neural network model, and input the cross-modal fusion graph feature representation into the node-level category discrimination model;

[0046] S422. By using the learnable classification weight matrix in the node-level category discrimination model, the cross-modal fusion graph feature representation is mapped to the illegal construction change category space to obtain the mapping result;

[0047] S423. Normalize the mapping result using a normalization function to obtain a set of probabilities for node-level illegal construction change categories, and generate a node-level illegal construction change probability map based on the set of probabilities for node-level illegal construction change categories.

[0048] Furthermore, in order to generate urban illegal construction alarm records, three types of illegal construction change information can be obtained through differential operations, thereby identifying the category of illegal construction change. Conditional random field post-processing is performed on the node-level illegal construction change probability map to obtain the three types of illegal construction change information. Based on this information, urban illegal construction alarm records are generated, including:

[0049] S431. Perform conditional random field post-processing on the node-level illegal construction change probability map to generate a pixel-level urban illegal construction change mask map.

[0050] S432. By performing a difference operation between the pixel-level urban illegal construction change mask map and the pre-acquired historical urban illegal construction change mask map, three types of illegal construction change information are obtained.

[0051] S433. Write the three types of illegal construction change information into the city geographic information system to generate urban illegal construction alarm records containing spatial coordinates, change type and violation clause labels.

[0052] The beneficial effects of this invention are as follows:

[0053] 1. This invention constructs three heterogeneous graph structures simultaneously: spectral flow graph, spatial flow graph, and radar flow graph. These graphs are built from three dimensions: texture spectral features, urban spatial structure relationships, and radar microwave scattering information. Through a dedicated graph construction strategy, physical edge weight functions are defined in each graph, fundamentally improving the ability to characterize multidimensional information from the original image data.

[0054] 2. This invention enables dynamic fusion operation across modal graphs to dynamically adjust the fusion operation based on the stability, autocorrelation, and trend of nodes in historical image sequences when fusing spectral flow, spatial flow, and radar flow graph features. At the same time, combined with temporal consistency anchoring mapping, structural registration and feature alignment of graph nodes are achieved between different time nodes, significantly improving the adaptability and robustness of time-varying graph structures in cross-temporal change detection tasks.

[0055] 3. By embedding knowledge of urban and rural planning regulations into the edge weight calculation process of the three-flow graph, this invention can adjust the edge weights in the fused graph according to regulations by constructing building height adjustment factors, plot ratio factors, and land use attribute factors in practical applications. This enables the network to automatically suppress node associations that do not conform to urban and rural planning standards during training and inference, thereby enhancing the monitoring method's ability to determine the legality of illegal constructions. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a flowchart of a remote sensing change monitoring method for urban illegal construction expansion based on a three-flow graph neural network according to an embodiment of the present invention. Detailed Implementation

[0058] To further illustrate the various embodiments, the present invention provides accompanying drawings, which are part of the disclosure of the present invention. These drawings are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these drawings, those skilled in the art should be able to understand other possible implementation methods and the advantages of the present invention.

[0059] According to an embodiment of the present invention, a remote sensing method for monitoring changes in the expansion of illegal urban buildings based on a three-flow graph neural network is provided.

[0060] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments, such as... Figure 1 As shown, the method for monitoring urban illegal construction expansion based on a three-flow graph neural network according to an embodiment of the present invention includes the following steps:

[0061] S1. Acquire multi-source image data and preprocess the multi-source image data to obtain a standard image dataset.

[0062] Specifically, multi-source image data is acquired and preprocessed to obtain a standard image dataset, including:

[0063] S11. Acquire multi-source image data and perform geometric registration on the multi-source image data to obtain aligned multi-source image data;

[0064] S12. Radiometric difference elimination is performed on the aligned multi-source image data using radiometric normalization to obtain multi-source image data after difference elimination.

[0065] S13. After eliminating differences, the multi-source image data is uniformly resampled to a common coordinate system to obtain a standard image dataset. This enables the standardization of multi-source image data through preprocessing steps such as geometric registration, radiometric normalization, and resampling in the common coordinate system.

[0066] Specifically, multi-source remote sensing image data (i.e., multi-source image data) is collected and uniform preprocessing (i.e., geometric registration, radiometric normalization, noise suppression, and common coordinate system resampling) is performed. The multi-source remote sensing image data includes optical remote sensing images covering the main urban area, synthetic aperture radar remote sensing images, and historical remote sensing image sequences to obtain a standard reference remote sensing image dataset.

[0067] S2. Construct a heterogeneous graph structure based on a standard image dataset, and input the heterogeneous graph structure into a neural network branch to obtain multi-stream feature representations. Then, fuse the multi-stream feature representations through a cross-modal graph dynamic fusion operation to generate a cross-modal fused graph feature representation.

[0068] Specifically, a heterogeneous graph structure is constructed based on a standard image dataset, and this structure is input into a neural network branch to obtain multi-stream feature representations. Cross-modal graph dynamic fusion operations are then used to fuse these multi-stream feature representations to generate cross-modal fused graph feature representations, including:

[0069] S21. Construct heterogeneous graph structures based on standard image datasets to obtain spectral flow graph structures, spatial flow graph structures, and radar flow graph structures.

[0070] Specifically, heterogeneous graph structures are constructed based on standard image datasets, resulting in spectral flow graph structures, spatial flow graph structures, and radar flow graph structures, including:

[0071] S211. Based on the image modal types of the standard image dataset, the graph node set is divided to obtain the spectral flow graph node set, the spatial flow graph node set, and the radar flow graph node set;

[0072] S212. Calculate the spectral angle similarity between spectral feature vectors based on the spectral flow graph node set, and use the spectral angle similarity combined with the preset spectral edge connection threshold to construct spectral edge connection relationships. Generate the spectral flow graph structure based on the spectral edge connection relationships and the spectral flow graph node set.

[0073] S213. Obtain the Euclidean distance between the geometric centroids of the corresponding building outlines using the spatial flow graph node set, and construct the spatial flow graph edge connection relationship based on the Euclidean distance and the preset adjacency threshold. Generate the spatial flow graph structure based on the spatial flow graph edge connection relationship and the spatial flow graph node set.

[0074] S214. Calculate the radar backscattering similarity based on the average backscattering coefficient of the radar patch region in the radar flow graph node set, and construct the radar flow graph edge connection relationship using the radar backscattering similarity. Generate the radar flow graph structure based on the radar flow graph edge connection relationship and the radar flow graph node set. This can improve the ability to analyze the changes and evolution of multimodal graph structures by connecting the graph node set with edges, thereby improving the monitoring ability of illegal construction areas in cities.

[0075] S22. Input the spectral flow graph structure, spatial flow graph structure, and radar flow graph structure into the three-flow graph neural network branch respectively to obtain the spectral flow feature representation, spatial flow feature representation, and radar flow feature representation.

[0076] Specifically, the spectral flow graph structure, spatial flow graph structure, and radar flow graph structure are respectively input into the three-flow graph neural network branch to obtain spectral flow feature representation, spatial flow feature representation, and radar flow feature representation, including:

[0077] S221. Input the spectral flow graph structure into the spectral branch of the three-flow graph neural network, and introduce a spectral attention graph convolutional unit with non-constructed spectral drift to update the features of the nodes in the spectral flow graph structure to obtain the spectral flow feature representation.

[0078] S222. Input the spatial flow graph structure into the spatial flow graph branch of the three-flow graph neural network, and introduce a spatial gated graph convolutional unit enhanced by illegal geometric topology changes to update the features of the nodes of the spatial flow graph structure, thereby obtaining the spatial flow feature representation;

[0079] S223. The radar flow graph structure is input into the radar flow graph branch of the three-flow graph neural network, and a radar flow residual graph convolution unit for dynamically adjusting the backscattering structure of the illegal construction area is introduced to update the features of the nodes of the radar flow graph structure, thereby obtaining the radar flow feature representation. Under the acquisition of spectral flow feature representation, spatial flow feature representation and radar flow feature representation, it can help to dynamically adjust the degree of influence of neighboring nodes on target nodes, capture the fine-grained changes in building form and structure during the expansion of illegal construction, and optimize the sensitivity of radar scattering feature changes during the expansion of urban illegal construction.

[0080] S23. By performing cross-modal graph dynamic fusion operations within the three-flow graph neural network, the spectral flow feature representation, spatial flow feature representation, and radar flow feature representation are fused to generate a cross-modal fused graph feature representation. This can improve the ability of multimodal structural semantic complementarity under the generation of the cross-modal fused graph feature representation and avoid information fragmentation problems in the process of multi-source data fusion.

[0081] Specifically, the spectral flow graph structure uses spectral texture feature blocks as nodes and establishes edges based on spectral angle differences; the spatial flow graph structure uses building vector outlines and urban plot grids as nodes and establishes edges based on geographic topological adjacency relationships; and the radar flow graph structure uses synthetic aperture radar scattering units as nodes and establishes edges based on backscattering coefficient similarity.

[0082] Specifically, the standard reference remote sensing image dataset (i.e., the standard image dataset) is divided into three types of graph node sets according to image modality. The first type is the spectral flow graph node set V. (s) The second category is the set of spatial flow graph nodes V. (p) The third category is the radar flow graph node set V. (r) .

[0083] When constructing edge connections in a spectral flow graph structure, for any two spectral flow graph nodes v in the spectral flow graph node set... i (s) and v j (s) The spectral eigenvector is f i (s) with f j (s) The spectral angle similarity between spectral feature vectors is calculated. Based on the spectral angle similarity, it is determined whether to establish an edge connection relationship between two spectral flow graph nodes. If the spectral angle similarity is less than or equal to the spectral edge connection threshold, an edge connection relationship is established between the two spectral flow graph nodes. The edge connection weight value in the edge connection relationship is calculated based on the inverse cosine function of the angle cosine. The result is used to represent the strength of material similarity. The smaller the spectral angle similarity, the more similar the materials are, and the larger the edge connection weight is.

[0084] When constructing edge connections in a spatial flow graph structure, for any two spatial flow graph nodes v in the set of spatial flow graph nodes... i (p) and v j (p) The algorithm determines whether any two spatial flow map nodes have a topological adjacency relationship in geospatial space. If any two spatial flow map nodes have a common boundary or the geometric centroid Euclidean distance between their corresponding building outlines is less than a preset adjacency threshold, then an edge connection relationship is established between the two spatial flow map nodes. The edge connection weight is defined as the reciprocal function of the geometric centroid Euclidean distance. The edge connection weight is used to represent the adjacency strength of the urban spatial structure. The smaller the geometric distance, the larger the edge connection weight, indicating that the buildings are structurally closer.

[0085] When constructing the edge connections in the radar flow graph structure, for any two radar flow graph nodes v in the radar flow graph node set... i (r)and v j (r) The radar backscattering similarity between two radar flow maps is calculated using the average backscattering coefficient of the radar patch region where any two radar flow maps are located. The radar backscattering similarity is used as input, and the edge connection weight is calculated in the form of an exponential function with a Gaussian kernel function. The edge connection weight is controlled by the Gaussian kernel bandwidth parameter. The smaller the difference in the average backscattering coefficient, the more similar the radar structural characteristics of the two radar flow map nodes are, and the larger the edge connection weight value is. The edge connection weight is used to characterize the strength of the structural similarity of buildings at the radar scattering patch level.

[0086] The set of spectral flow graph nodes V (s) Spatial flow graph node set V (p) With radar flow graph node set V (r) Connect each edge to the corresponding set E. (s) E (p) E (r) Combine them to construct spectral flow graph structures G respectively. (s) =(V (s) E (s) Spatial flow graph structure G (p) =(V (p) E (p) Radar flow graph structure G (r) =(V (r) E (r) ).

[0087] The first category is the spectral flow graph node set V. (s) Each spectral flow graph node v in the set of spectral flow graph nodes i (s) The first category represents a spectral texture feature block of size 256×256 obtained from an optical remote sensing image; the second category is the spatial flow graph node set V. (p) Each spatial flow graph node v in the set of spatial flow graph nodes j ( p) The third type represents a vector outline of a building or a grid cell of an urban plot; the third type is the radar flow graph node set V( r) Each radar flow graph node v in the radar flow graph node set k (r) This represents a backscattered patch area in a synthetic aperture radar remote sensing image that has undergone focusing processing.

[0088] Specifically, the spectral flow graph structure, spatial flow graph structure, and radar flow graph structure are respectively input into the three-flow graph neural network branches. Each branch stacks multiple layers of graph convolutional units to extract deep graph features and outputs spectral flow feature representation, spatial flow feature representation, and radar flow feature representation.

[0089] Spectral flow graph structure G(s) Spatial flow graph structure G (p) and radar flow graph structure G (r) Input the corresponding three-flow graph neural network branches respectively. The three-flow graph neural network branches correspond to the spectral flow graph branch, the spatial flow graph branch, and the radar flow graph branch.

[0090] An attention graph convolutional unit targeting the spectral drift of illegal urban construction is introduced into the spectral flow graph branch. An attention weighting mechanism dynamically adjusts the influence of neighboring nodes on the target node. In the first-layer spectral flow graph attention graph convolutional unit, the influence of the spectral flow graph node v is adjusted. i (s) Perform feature updates:

[0091]

[0092] In the formula, h i (s，l) Represents the spectral flow graph node v in layer l. i (s) The graph feature representation is used to represent the material texture information of this area in the city under optical imagery, h j (s，l-1) Represents the adjacent nodes v of the spectral flow graph in layer l-1. j (s) The graph feature representation, W (s，l-1) N(v) represents the trainable transformation weight matrix of the (l-1)th layer spectral flow graph convolutional unit. i (s) ) represents the spectral flow graph node v i (s) The set of adjacent nodes, σ(·) represents the nonlinear activation function, α ij (s，l-1) Represents the branch node v of the (l-1)th layer spectral flow graph j (s) For node v i (s) Attention coefficient.

[0093] The attention coefficient is dynamically adjusted by calculating the degree of difference in spectral drift between adjacent nodes in historical remote sensing image sequences. If the historical changes in the area corresponding to a node exceed a threshold, the attention coefficient is enhanced, focusing on the spectral change area of ​​suspected illegal construction.

[0094] The spectral flow attention graph convolutional unit is used to process each spectral flow graph node in the spectral flow graph structure. Its significance lies in introducing an attention mechanism to achieve dynamic adjustment of the influence of neighboring nodes. The attention mechanism automatically assigns different attention weight values ​​by calculating the spectral change differences between a graph node and its neighboring nodes. The attention weight values ​​reflect the degree of spectral drift in the historical remote sensing image sequence. If the spectral change characteristics of a certain neighboring node are obvious, the system will give it a higher attention response to emphasize its importance in the propagation of spectral information.

[0095] In the scenario of remote sensing monitoring of urban illegal construction expansion, building additions and renovations can cause local material changes, resulting in significant spectral shifts. Traditional graph convolution methods cannot distinguish between stable and abnormal regions, potentially leading to feature averaging. However, by introducing an attention mechanism, spectral flow graph nodes can adaptively focus on suspected illegal construction areas, effectively improving the representation ability of spectrally change-sensitive regions and thus significantly enhancing the detection accuracy of sudden spectral changes in newly added illegal constructions.

[0096] In the spatial flow graph branch, a gated graph convolutional unit based on enhanced geometric topological changes of urban illegal constructions is introduced to capture fine-grained changes in building morphology and structure during the expansion of illegal constructions. The spatial flow graph node v in the l-th layer gated graph convolutional unit... i (p) The update is as follows:

[0097]

[0098] In the formula, h i (p，l) Represents the spatial flow graph node v in the l-th layer. i (p) The graphical feature representation is used to characterize the structural form of the area at the level of urban plots or building outlines, h i (p，l-1) Represents the spatial flow graph node v in layer l-1. i (p) The previous layer graph feature table, Represents the spatial flow graph node v in the l-th layer. i (p) The graph convolution updates the value, z. i (p,l-1) This represents updating the gated variable, with a value range of [0, 1], used for adaptive adjustment of spatial flow graph node v. i (p) The update intensity is determined based on the spatial flow graph node v. i (p) The degree of contour change of the corresponding building in multi-temporal remote sensing images is adaptively calculated. If there are additions or reconstructions of structures in the area, the gate variable z is updated. i (p,l-1)Approaching 1, ⊙ represents the Hadamard element-wise multiplication symbol, N(v i (p) ) represents a spatial flow graph node v i (p) The set of adjacent nodes represents the connection relationship of the building outline in the spatial topology graph.

[0099] The spatial flow gated map convolutional unit is used to process the spatial flow map structure composed of urban plots and building outlines. Its core construction lies in the introduction of a gating mechanism, which achieves weighted fusion between features of new and old maps through gating variables. The gating variables adaptively adjust their values ​​according to the geometric changes of building outlines in multi-temporal remote sensing images. If the outline undergoes significant deformation or area expansion, the proportion of new feature updates is increased; if the outline remains stable, the ability to preserve historical features is enhanced.

[0100] This method is of significant value in monitoring geometric and topological changes during the expansion of illegal urban structures. Especially in cases involving fine-grained targets such as mezzanine additions and lateral expansions, where the contour deformation is small and the location of changes is not fixed, traditional graph neural networks often struggle to capture key boundary changes during feature propagation. The gating mechanism enables spatial flow graph nodes to update structural features on demand, improving the model's generalization ability while retaining sensitivity to local topological anomalies, effectively solving the problem of failing to identify structural changes when averaged or fine-tuned.

[0101] An adaptive residual graph convolutional unit based on dynamic adjustment of the backscattering structure of illegal construction areas is introduced into the radar flow graph branch to optimize the sensitivity of radar scattering characteristics changes during the expansion of illegal construction in the city. The node v of the first-layer radar flow residual graph convolutional unit... i (r) Updated to:

[0102]

[0103] In the formula, h i (r,l) Represents the radar flow graph node v in layer l. i (r) The graph feature representation is used to characterize the microwave structure of the radar scattering pattern corresponding to the node, h i (r,l-1) Represents the radar flow graph node v in layer l-1 i (r) The previous layer of graph features, w ij (r，l-1) This represents the radar flow graph node v at layer l-1 in the radar diagram. i (r) Its adjacent radar flow graph node v j (r) The edge connection weights between them are calculated from the backscatter similarity, W. (r，l-1)N(v) represents the learnable weight matrix of the (l-1)th layer radar image convolution. i (r) ) represents radar flow graph node v i ( (r) The set of adjacent nodes of ) represents the structural relationship of radar patches in the map domain, β i ( (r,l-1) () is a dynamic adjustment coefficient.

[0104] According to radar flow graph node v i (r) The significance of changes in the historical radar scattering structure characteristics of the corresponding area is determined. Due to the large differences in radar characteristics in the illegal construction area, the adjustment coefficient is set to a high value, thereby enhancing the sensitivity of the node to changes in radar characteristics in the illegal construction area.

[0105] The radar flow adaptive residual map convolutional unit is specifically designed for processing radar flow map structures constructed from synthetic aperture radar (SAR) patches. Its significance lies in enhancing features and amplifying anomalous scattering responses through the introduction of residual structures and dynamic adjustment coefficients. The residual structure allows nodes to retain feature information from the previous layer during each feature update process, preventing radar feature degradation during multi-layer propagation. The dynamic adjustment coefficients are determined based on the significance of scattering changes in historical remote sensing sequences, reflecting the sensitivity of illegal construction areas to microwave scattering.

[0106] Radar imagery offers significant advantages in monitoring variations in scattering structures within areas of illegal construction. In real-world scenarios, radar images can penetrate clouds and capture nighttime information, often supplementing the limitations of optical remote sensing. When changes occur in the wall structure or roof materials of buildings at the radar patch level, their scattering characteristics change accordingly. The combination of residual mechanisms and dynamic weighting enhances the model's nodal response capabilities while strengthening the representation of scattering anomalies in key areas, thereby improving the robust detection capability for changes in illegal construction. This makes it suitable for monitoring changes in high-density urban areas or under low visibility conditions.

[0107] After the spectral flow graph branch, spatial flow graph branch, and radar flow graph branch are calculated by multi-layer graph convolutional units, the corresponding deep graph feature representations are output, which are the spectral flow feature representations H. (s) =h i (s，L) Spatial flow characteristics representation H (p) =h i (p ，L) Radar flow characteristic representation H (r) =h i (r,L) , where L represents the total number of stacked layers of each flow graph convolutional unit.

[0108] The cross-modal graph dynamic fusion operation is performed inside the three-flow graph neural network. The spectral flow feature representation, spatial flow feature representation and radar flow feature representation are weighted and fused through the three-flow mutual information weight adjustment mechanism to generate cross-modal fused graph feature representation.

[0109] Define the mutual information weights of each modal feature class to the fusion result during the three-stream feature fusion process, and perform weighted fusion of the spectral flow feature representation, spatial flow feature representation, and radar flow feature representation to obtain the cross-modal feature representation h of the fused graph node i. i (f) :

[0110]

[0111] Where, λ i (s) , λ i (p) , λ i (r) λ represents the mutual information weight coefficients of the spectral flow feature representation, spatial flow feature representation, and radar flow feature representation corresponding to node i in the graph. These mutual information weight coefficients are adaptively adjusted based on the temporal stability and differences of each modal feature and the current node in historical remote sensing images. Specifically, if node i exhibits significant spectral drift but stable structural boundaries during the expansion of illegal urban construction, then λ... i (p) With λ i (r) The weight of will be relatively increased.

[0112] S3. Construct structural anchoring relationships using pre-acquired historical image sequences, and align the graph node sets in the cross-modal fusion graph feature representation according to the structural anchoring relationships to obtain a temporally consistent graph structure.

[0113] Specifically, structural anchoring relationships are constructed using pre-acquired historical image sequences, and the graph node sets in the cross-modal fusion graph feature representation are aligned according to these relationships to obtain a temporally consistent graph structure, including:

[0114] S31. By utilizing the consistency of the geographical location of the same spatial entity at different time points in a pre-acquired historical image sequence, construct a structural anchoring relationship between time points;

[0115] S32. Align the graph node set in the cross-modal fusion graph feature representation according to the structural anchoring relationship to obtain the aligned fusion node set and cross-time edge connections;

[0116] S33. By using the aligned fused node set and cross-time edge connections to construct the graph structure, a time-consistent graph structure is obtained. Under the effect of structural anchoring relationships, the structural registration and feature alignment of graph nodes can be achieved, which significantly improves the adaptability and robustness of time-varying graph structures in cross-time change detection tasks.

[0117] Specifically, a time-series self-calibration is performed on the multi-temporal graph structure to align entities corresponding to different time nodes in the cross-modal fusion graph feature representation, thereby obtaining a time-consistent graph structure.

[0118] Based on historical remote sensing image sequences, structural anchoring relationships are constructed, and temporal self-calibration is performed on the multi-temporal map structure. Entities corresponding to different time nodes are aligned in the cross-modal fusion map feature representation to obtain a temporally consistent map structure.

[0119] Specifically, by leveraging the consistency of the geographical location of the same spatial entity in historical remote sensing image sequences at different time points, a structural anchoring mapping relationship is constructed between time points, and a set of map nodes V is set at time t. t (f) Then the set of nodes in the graph at time t+Δt Then for any fused graph node v i (f) ∈V t (f) In the graph node set Find matching node v in j (f) This minimizes the differences in fused features; the calculation expression is:

[0120]

[0121] Where |·|² is the L2 norm, used to measure the feature distance of the same spatial entity at different times, h i (f) Represents the fusion graph node v i (f) The cross-modal fusion map feature vector at time t, which integrates spectral, structural, and radar information, is used to represent a multi-dimensional remote sensing representation of the built area. v (f) This represents the cross-modal fusion graph feature vector of any fusion graph node at time node t+Δt.

[0122] The graph node set V at each time point t (f) Alignment is achieved based on the structural anchoring mapping relationship in step S43, and the fused time-consistent graph structure Galigned is constructed. (f) =(Valigned) (f) Ealigned (f)), where Valigned (f) Ealigned represents the fused node set after node alignment at all time points. (f) This indicates the cross-time edge connections between aligned nodes that are preserved in the graph structure.

[0123] S4. Construct and train a neural network model through a time-consistent graph structure, use the trained neural network model to generate a node-level illegal construction change probability map, obtain illegal construction change information based on the node-level illegal construction change probability map, and generate urban illegal construction alarm records based on the illegal construction change information.

[0124] Specifically, a neural network model is constructed and trained using a time-consistent graph structure. This trained model is then used to generate a node-level probability map of illegal construction changes. Based on this probability map, information on illegal construction changes is obtained, and urban illegal construction alarm records are generated, including:

[0125] S41. Construct a contrastive learning framework based on the temporally consistent graph structure, and use the contrastive learning framework to construct and train a three-flow graph neural network model.

[0126] Specifically, the process involves constructing a contrastive learning framework based on the temporally consistent graph structure, and then using this framework to build and train a three-flow graph neural network model, including:

[0127] S411. Based on the temporally consistent graph structure, a contrastive learning framework is introduced. The contrastive learning framework is used to construct positive and negative sample pairs and model similarity constraints on the node feature representation of the temporally consistent graph structure, and the contrastive learning loss function is obtained.

[0128] S412. Using the pre-acquired unlabeled image data, set a pseudo-label generation function in the temporal consistent graph structure, generate pseudo-labels according to the pseudo-label generation function, and construct a pseudo-label supervision loss function through the pseudo-labels.

[0129] S413. Combine the pseudo-label supervision loss function and the contrastive learning loss function to obtain the joint loss function. Optimize the parameters of the three-flow graph neural network model using the joint loss function to obtain the optimized three-flow graph neural network model.

[0130] S414. Train the optimized three-flow graph neural network model and embed urban and rural planning regulations into the three-flow graph edge weight calculation process. Correct the edge connection weights in the time-series consistent graph structure according to regulations to obtain the trained three-flow graph neural network model. Under the effect of the joint loss function on the parameter update of the three-flow graph neural network model, the accuracy of obtaining the probability map of illegal construction changes can be improved.

[0131] S42. Use the trained three-flow graph neural network model to generate cross-modal fusion graph feature representations that embed legal knowledge, and infer the cross-modal fusion graph feature representations to obtain node-level illegal construction change probability graphs.

[0132] Specifically, a trained three-flow graph neural network model is used to generate cross-modal fusion graph feature representations embedded with regulatory knowledge, and inference is performed on these cross-modal fusion graph feature representations to obtain node-level probability graphs of illegal construction changes, including:

[0133] S421. Generate a cross-modal fusion graph feature representation with embedded regulatory knowledge based on the trained three-flow graph neural network model, and input the cross-modal fusion graph feature representation into the node-level category discrimination model;

[0134] S422. By using the learnable classification weight matrix in the node-level category discrimination model, the cross-modal fusion graph feature representation is mapped to the illegal construction change category space to obtain the mapping result;

[0135] S423. Normalize the mapping result using a normalization function to obtain a node-level set of probabilities for illegal construction change categories. Generate a node-level map of illegal construction change probabilities based on the node-level set of probabilities for illegal construction change categories. Under the mapping effect of the node-level category discrimination model, the graph feature representation can be mapped to the illegal construction change category space, thereby obtaining the node-level set of probabilities for illegal construction change categories.

[0136] S43. Perform conditional random field post-processing on the node-level illegal construction change probability map to obtain three types of illegal construction change information, and generate urban illegal construction alarm records based on the illegal construction change information; under the role of generating urban illegal construction alarm records, the monitoring method can enhance its ability to judge the legality of illegal construction.

[0137] Specifically, conditional random field post-processing is performed on the node-level illegal construction change probability map to obtain three types of illegal construction change information, and urban illegal construction alarm records are generated based on the illegal construction change information, including:

[0138] S431. Perform conditional random field post-processing on the node-level illegal construction change probability map to generate a pixel-level urban illegal construction change mask map.

[0139] S432. By performing a difference operation between the pixel-level urban illegal construction change mask map and the pre-acquired historical urban illegal construction change mask map, three types of illegal construction change information are obtained.

[0140] S433. Write the three types of illegal construction change information into the urban geographic information system to generate urban illegal construction alarm records containing spatial coordinates, change type and violation clause labels; it can obtain the three types of illegal construction change information through differential operation, and then identify the category of illegal construction change.

[0141] Specifically, a contrastive learning framework is built on the temporally consistent graph structure, a three-flow graph neural network model adapted to few samples is constructed and trained, and a cross-modal fusion graph feature representation embedded with regulatory knowledge is formed.

[0142] Based on the temporally consistent graph structure, a contrastive learning framework is constructed. Pseudo-labels are generated using unlabeled remote sensing images, and the parameters of the three-flow graph neural network are updated through self-supervised training to obtain a three-flow graph neural network model adapted to few samples. During the training process of the three-flow graph neural network model, knowledge of urban and rural planning regulations is embedded into the graph edge weight calculation. The edge weights are updated according to building height thresholds, plot ratio restrictions, and land use attribute conditions to form a cross-modal fusion graph feature representation with embedded regulatory knowledge.

[0143] Inference is performed using cross-modal fusion graph feature representations embedded with legal knowledge, outputting a node-level probability map of illegal construction changes. Conditional random field post-processing is then applied to the node-level probability map of illegal construction changes to determine three types of change information: newly added illegal constructions, demolition and restoration, and no change.

[0144] Inference is performed using cross-modal fusion graph feature representations embedded with legal knowledge to output a node-level probability map of illegal construction changes. Conditional random field post-processing is then applied to the node-level probability map to generate a pixel-level urban illegal construction change mask map. The pixel-level urban illegal construction change mask map is then compared with the historical urban illegal construction change mask map to determine three types of change information: newly added illegal construction, demolition and restoration, and no change. These three types of change information are then written into the urban geographic information system to generate urban illegal construction alarm records containing spatial coordinates, change type, and violation clause labels.

[0145] Specifically, a contrastive learning framework is introduced based on the temporally consistent graph structure to improve the node feature representation h. i (f) We construct positive and negative sample pairs and model similarity constraints, and then construct a contrastive learning loss function L. contrast :

[0146]

[0147] Among them, h i (f) Represents the fusion graph node v i (f) The cross-modal fusion graph feature representation, Representation and fusion graph node v i (f) Features of positive sample nodes aligned in time sequence For the negative sample node features randomly sampled in the graph domain, sim(·,·) represents the normalized cosine similarity function, τ is the temperature coefficient, and N i - For the fusion graph node v i(f) The set of negative samples.

[0148] Generating pseudo-labels in a time-consistent map structure using unlabeled remote sensing images from the training set Pseudo-label generation using cross-modal fusion graph feature representation h i (f) Based on the alignment results of the multi-temporal structure, a pseudo-label generation function is defined:

[0149]

[0150] Where C represents the set of illegal construction change categories, W (cls) This is the trainable parameter matrix of the pseudo-label classifier. For the fusion graph node v i (f) The corresponding prediction category labels include new illegal construction, no change, and demolition and restoration.

[0151] The parameters of the three-flow graph neural network model are optimized by combining pseudo-label supervision loss and contrastive learning loss. The joint loss function is defined as follows:

[0152] L total =L contrast +λ sup ·L supervised ;

[0153] Among them, L supervised λ is the cross-entropy loss after pseudo-label generation. sup L is the adjustment coefficient for the pseudo-supervision loss. tota This is the final optimization objective function used for training the three-flow graph neural network model.

[0154] In the training process of the three-flow graph neural network model, knowledge of urban and rural planning regulations is embedded into the graph edge weight calculation process to improve the time-consistent graph structure Galigned. (f) edge e in ij (f) edge connection weight w ij (law) The regulations were amended to define the edge connection weight calculation function as follows:

[0155] w ij (law) =w ij (f) ·γ ij ( h) ·γ ij ( f) ·γ ij (z) ;

[0156] Among them, w ij(f) For the original fusion edge weights, γ ij (h) The building height adjustment factor γ is used when the difference in building height between two connected nodes exceeds the prescribed height limit. ij (h) <1; γ ij (f) As a plot ratio adjustment factor, it sets limits on the plot ratio of building land; violations reduce the edge credibility. ij (z) This is a land use attribute adjustment factor used to reflect the consistency of land use properties among the plots belonging to different nodes.

[0157] Based on the updated edge connection weights w ij (law) Reconstructing cross-modal fusion graph feature representations embedded with urban and rural planning regulations knowledge

[0158] The node-level probability set of illegal construction change categories corresponding to each node in the cross-modal fusion graph feature representation embedded with urban and rural planning regulations knowledge is calculated. The node-level probability set of illegal construction change categories is used to represent the probability value of a fusion graph node belonging to the category of newly added illegal construction, demolition and restoration, or unchanged. The node-level probability set of illegal construction change categories is obtained by inputting the graph feature representation of the fusion graph node into a node-level category discrimination module composed of a learnable classification weight matrix, and performing Softmax normalization on the output result, thus forming a node-level urban illegal construction change probability map.

[0159] Among them, the graph feature representation of the fused graph node is the final representation after integrating spectral features, spatial structure features and radar scattering features and embedding urban and rural planning regulations. The learnable classification weight matrix in the node-level category discrimination module is the key parameter that maps the graph feature representation to the illegal construction change category space. Softmax normalization is used to convert the response value of each category into a probability value, forming a triplet of new illegal construction probability, demolition and restoration probability and unchanged probability, thus forming a node-level urban illegal construction change probability map.

[0160] A mapping relationship is established between the map domain and the pixel domain of the remote sensing image. The category probability value of each fused map node in the node-level urban illegal construction change probability map is projected onto its corresponding original remote sensing image pixel region to form a preliminary pixel-level urban illegal construction change map. Each pixel in the preliminary pixel-level urban illegal construction change map contains probability estimates for the categories of newly added illegal construction, demolition and restoration, and unchanged construction. Conditional random field (CRF) post-processing optimization is performed on the preliminary pixel-level urban illegal construction change map. The CRF post-processing optimization aims to enhance the spatial consistency and structural continuity of the pixel-level change map. The CRF post-processing optimization is implemented by defining a total energy function, which consists of two parts: the first part is the negative logarithm of the change probability of the pixel itself, representing the single-point confidence of the pixel belonging to a certain change category; the second part is the joint relationship potential function between the pixel and other pixels in its neighborhood, used to characterize the consistency constraints of adjacent pixels in terms of category.

[0161] In the post-processing optimization of Conditional Random Fields, the joint relation potential function between adjacent pixels is weighted based on spectral similarity and spatial adjacency, making it more likely that pixels with similar spectra and adjacent locations will have the same change category label, and finally outputting a pixel-level urban illegal construction change mask map with stronger structural consistency.

[0162] The obtained pixel-level urban illegal construction change mask Mt and the historical urban illegal construction change mask Mt (t-Δt) Perform pixel-level difference operations to identify the types of changes in illegal constructions, let M be... diff =M t -M (t-Δt) Then: If M diff (x, y) = +1 indicates that a new illegal structure has been added at location (x, y); if M diff (x, y) = -1 indicates that demolition and restoration occurred at location (x, y); if M diff (x, y) = 0 indicates that the position (x, y) remains unchanged; based on the classification results, three types of change information layers are constructed, and each layer records the spatial coordinate set corresponding to the change category.

[0163] The three types of change information layers are written into the city's geographic information system to generate an urban illegal construction alarm record table.

[0164] The urban illegal construction alarm record form includes: location number, coordinate information, change type, time label, and legal label. The coordinate information includes: a unique spatial identifier for the illegal construction area; and the coordinates of the spatial center point of the illegal construction area (x, y). i y i); Change type: new illegal construction, demolition and restoration, or no change; Time label: the date t of the remote sensing image acquisition in which the current change occurred; Regulatory label: the corresponding violation clause number Rk of urban and rural planning regulations, identified by the regulatory vector embedded in the cross-modal fusion graph feature representation that embeds knowledge of urban and rural planning regulations.

[0165] In summary, this invention enables the network to automatically suppress node associations that do not conform to urban and rural planning standards during training and inference, thereby enhancing the model's ability to determine the legality of illegal constructions.

[0166] By designing a weight adjustment mechanism based on node-level mutual information, the fusion weights can be dynamically adjusted according to the stability, autocorrelation, and changing trends of nodes in historical remote sensing image sequences when fusing spectral flow, spatial flow, and radar flow map features. Furthermore, regulatory factors are introduced into the edge connection weights of the graph neural network. By constructing building height adjustment factors, floor area ratio factors, and land use attribute factors, regulatory constraints are applied to the edge weights in the fused graph. This enables the network to automatically suppress node associations that do not conform to urban and rural planning standards during training and inference, enhancing the model's ability to determine the legality of illegal constructions.

[0167] Example:

[0168] The Urban Management and Law Enforcement Bureau of City A launched a special campaign to monitor the expansion of illegal buildings in key areas. In recent years, due to the renovation of urban villages and the relocation of surrounding industries, there have been many non-standard building activities such as hidden additions and carports converted into dormitories. Traditional remote sensing image manual identification methods often result in missed judgments and delays due to limited manpower and slow feedback.

[0169] Based on multi-source remote sensing data from July 2022 to June 2024, we comprehensively carried out three-flow graph feature modeling and graph neural network detection.

[0170] First, during the data acquisition phase, four phases of optical remote sensing data (0.5m resolution), three phases of C-band synthetic aperture radar (SAR) imagery (1m resolution), and building outlines and land parcel boundary information extracted from the urban geographic information database were procured from the National Resources Satellite Center. All data underwent a unified preprocessing process, including geometric correction, radiometric normalization, and resampling to the WGS84 / UTM coordinate system, to form a standard reference image dataset.

[0171] Subsequently, the three graph structures were constructed in their respective modes. The spectral flow graph structure uses 256×256 image blocks as nodes and calculates the edge weights between nodes through spectral angle similarity. The spatial flow graph structure uses building vector outlines and land parcel units as nodes and establishes edge weights through Euclidean geometric adjacency relationships. The radar flow graph structure is constructed based on the backscattering coefficient of SAR images. Each node represents a radar patch area, and the Gaussian kernel similarity of adjacent patches is used for edge connection.

[0172] Building upon this foundation, the three-flow graph neural network employs different graph convolutional structures for spectral flow, spatial flow, and radar flow inputs. In the spectral flow graph, an attention mechanism targeting spectral drift is introduced to effectively identify reflection changes caused by the addition of new building materials. In the spatial flow graph, a gating mechanism is used to characterize slight adjustments to the building outline, such as topological changes caused by adding floors or renovations. In the radar flow graph, residual graph convolution is used to enhance the focus on high-frequency changing regions, such as abrupt changes in scattering characteristics caused by temporary construction or building.

[0173] On August 1, 2024, the three-flow graph neural network was trained with the support of a cross-temporal structure alignment mechanism. The training set used labeled data from another street in Zone B between 2022 and 2023 (a total of approximately 4,825 illegal construction nodes, divided into three categories of changes: new, demolished, and unchanged). The model achieved an overall accuracy of 92.4% on the test set. Compared with the traditional scheme of using ResNet+ChangeNet to detect temporal changes in optical images, its IoU was improved by 12.7%.

[0174] In the actual deployment phase, this invention performed inference analysis on remote sensing images of a core area of ​​a town, covering a total of 8.7 square kilometers, and output a node-level probability map of illegal construction. After processing with a conditional random field, a pixel-level change mask map was finally generated. After comparison with images from the same period in 2023, 124 newly added building areas were automatically identified, of which 43 were classified as "potentially illegal constructions". Subsequently, the law enforcement bureau conducted on-site spot checks on 20 of these locations and found that 16 of them were indeed unapproved constructions. The accuracy rate of automatic identification was 80%, and the reporting time was within 48 hours, which is far better than the traditional manual investigation, which takes more than 3 weeks and covers less than 30% of the area.

[0175] Furthermore, with the help of the regulatory knowledge embedding module, the model can automatically identify areas that exceed building height or floor area ratio limits.

[0176] In a steel structure factory renovation project near a certain road, the model accurately marked it as an area where the height exceeded the 15-meter height limit line.

[0177] To further quantify the effectiveness of the method of this invention, the performance of the following three schemes in remote sensing data processing tasks between 2022 and 2024 was compared and tested:

[0178] Table 1. Data comparison between the method of the present invention and comparative methods 1 and 2.

[0179]

[0180]

[0181] As can be seen from the data in Table 1, the present invention has significant advantages in recall, precision and IoU, especially in handling low contrast and new class changes, showing higher sensitivity and accuracy, and a significant reduction in false alarm rate. At the same time, after being embedded in urban and rural regulations, its ability to judge legal and illegal additions has been qualitatively improved, and it has practical feasibility for direct application in urban law enforcement systems.

[0182] In summary, by utilizing the above-mentioned technical solution of this invention, this invention simultaneously constructs three heterogeneous graph structures—spectral flow graph, spatial flow graph, and radar flow graph—to build maps from three dimensions: texture spectral features, urban spatial structure relationships, and radar microwave scattering information. Through a dedicated graph construction strategy, physically meaningful edge weight functions are defined in each graph, fundamentally enhancing the multidimensional information characterization capability of the original image data. Furthermore, through cross-modal graph dynamic fusion operations, when fusing spectral flow, spatial flow, and radar flow graph features, the fusion operation can adjust the fusion based on the stability, autocorrelation, and changing trends of nodes in historical image sequences. By dynamically adjusting the graph structure and combining it with temporal consistency anchoring mapping, structural registration and feature alignment of graph nodes are achieved between different time nodes, significantly improving the adaptability and robustness of time-varying graph structures in cross-time change detection tasks. By embedding urban and rural planning regulations into the edge weight calculation process of the three-flow graph, the edge weights in the fused graph can be adjusted according to regulations by constructing building height adjustment factors, floor area ratio factors, and land use attribute factors in practical applications. This enables the network to automatically suppress node associations that do not conform to urban and rural planning standards during training and inference, enhancing the monitoring method's ability to determine the legality of illegal constructions.

[0183] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A remote sensing method for monitoring changes in urban illegal construction expansion based on a three-flow graph neural network, characterized in that, The method includes: S1. Acquire multi-source image data and preprocess the multi-source image data to obtain a standard image dataset; S2. Construct a heterogeneous graph structure based on a standard image dataset, and input the heterogeneous graph structure into a neural network branch to obtain multi-stream feature representations. Then, fuse the multi-stream feature representations through a cross-modal graph dynamic fusion operation to generate a cross-modal fused graph feature representation. S2 includes: S21. Construct heterogeneous graph structures based on standard image datasets to obtain spectral flow graph structures, spatial flow graph structures, and radar flow graph structures; S3. Construct structural anchoring relationships using pre-acquired historical image sequences, and align the graph node sets in the cross-modal fusion graph feature representation according to the structural anchoring relationships to obtain a temporally consistent graph structure; S3 includes: S31. By utilizing the consistency of the geographical location of the same spatial entity at different time points in a pre-acquired historical image sequence, construct a structural anchoring relationship between time points; S32. Align the graph node set in the cross-modal fusion graph feature representation according to the structural anchoring relationship to obtain the aligned fusion node set and cross-time edge connections; S33. Construct a graph structure using the aligned fused node set and cross-time edge connections to obtain a time-consistent graph structure; S4. Construct and train a neural network model through a time-consistent graph structure, use the trained neural network model to generate a node-level illegal construction change probability map, obtain illegal construction change information based on the node-level illegal construction change probability map, and generate urban illegal construction alarm records based on the illegal construction change information.

2. The method for monitoring urban illegal construction expansion using remote sensing based on a three-flow graph neural network according to claim 1, characterized in that, The process of acquiring multi-source image data and preprocessing it to obtain a standard image dataset includes: S11. Acquire multi-source image data and perform geometric registration on the multi-source image data to obtain aligned multi-source image data; S12. Radiometric difference elimination is performed on the aligned multi-source image data using radiometric normalization to obtain multi-source image data after difference elimination. S13. After eliminating the differences, the multi-source image data are uniformly resampled to a common coordinate system to obtain a standard image dataset.

3. The method for monitoring urban illegal construction expansion using remote sensing based on a three-flow graph neural network according to claim 1, characterized in that, Following S21, the following is also included: S22. Input the spectral flow graph structure, spatial flow graph structure and radar flow graph structure into the three-flow graph neural network branch respectively to obtain the spectral flow feature representation, spatial flow feature representation and radar flow feature representation; S23. By performing a cross-modal graph dynamic fusion operation inside the three-flow graph neural network, the spectral flow feature representation, spatial flow feature representation, and radar flow feature representation are fused to generate a cross-modal fused graph feature representation.

4. The method for monitoring urban illegal construction expansion using remote sensing based on a three-flow graph neural network according to claim 3, characterized in that, The construction of heterogeneous graph structures based on standard image datasets, resulting in spectral flow graph structures, spatial flow graph structures, and radar flow graph structures, includes: S211. Based on the image modal types of the standard image dataset, the graph node set is divided to obtain the spectral flow graph node set, the spatial flow graph node set, and the radar flow graph node set; S212. Calculate the spectral angle similarity between spectral feature vectors based on the spectral flow graph node set, and use the spectral angle similarity combined with the preset spectral edge connection threshold to construct spectral edge connection relationships. Generate the spectral flow graph structure based on the spectral edge connection relationships and the spectral flow graph node set. S213. Obtain the Euclidean distance between the geometric centroids of the corresponding building outlines using the spatial flow graph node set, and construct the spatial flow graph edge connection relationship based on the Euclidean distance and the preset adjacency threshold. Generate the spatial flow graph structure based on the spatial flow graph edge connection relationship and the spatial flow graph node set. S214. Calculate the radar backscattering similarity based on the average backscattering coefficient of the radar patch region in the radar flow graph node set, and use the radar backscattering similarity to construct the radar flow graph edge connection relationship. Generate the radar flow graph structure based on the radar flow graph edge connection relationship and the radar flow graph node set.

5. The method for monitoring urban illegal construction expansion using remote sensing based on a three-flow graph neural network according to claim 3, characterized in that, The step of inputting the spectral flow graph structure, spatial flow graph structure, and radar flow graph structure into the three-flow graph neural network branch respectively to obtain the spectral flow feature representation, spatial flow feature representation, and radar flow feature representation includes: S221. Input the spectral flow graph structure into the spectral branch of the three-flow graph neural network, and introduce a spectral attention graph convolutional unit with non-constructed spectral drift to update the features of the nodes in the spectral flow graph structure to obtain the spectral flow feature representation. S222. Input the spatial flow graph structure into the spatial flow graph branch of the three-flow graph neural network, and introduce a spatial gated graph convolutional unit enhanced by illegal geometric topology changes to update the features of the nodes of the spatial flow graph structure, thereby obtaining the spatial flow feature representation; S223. Input the radar flow graph structure into the radar flow graph branch of the three-flow graph neural network, and introduce the radar flow residual graph convolution unit with dynamic adjustment of the backscattering structure of the illegal construction area to update the features of the nodes of the radar flow graph structure, so as to obtain the radar flow feature representation.

6. The method for monitoring urban illegal construction expansion using remote sensing based on a three-flow graph neural network according to claim 1, characterized in that, The process of constructing and training a neural network model using a time-consistent graph structure, generating a node-level probability map of illegal construction changes using the trained neural network model, obtaining illegal construction change information based on the node-level probability map of illegal construction changes, and generating urban illegal construction alarm records based on the illegal construction change information includes: S41. Construct a contrastive learning framework based on the temporally consistent graph structure, and use the contrastive learning framework to construct and train a three-flow graph neural network model. S42. Use the trained three-flow graph neural network model to generate cross-modal fusion graph feature representations embedded with regulatory knowledge, and infer the node-level illegal construction change probability map from the cross-modal fusion graph feature representations. S43. Perform conditional random field post-processing on the node-level illegal construction change probability map to obtain three types of illegal construction change information, and generate urban illegal construction alarm records based on the illegal construction change information.

7. The method for monitoring urban illegal construction expansion using remote sensing based on a three-flow graph neural network according to claim 6, characterized in that, The step of constructing a contrastive learning framework based on the temporally consistent graph structure, and using the contrastive learning framework to construct and train a three-flow graph neural network model includes: S411. Based on the temporally consistent graph structure, a contrastive learning framework is introduced. The contrastive learning framework is used to construct positive and negative sample pairs and model similarity constraints on the node feature representation of the temporally consistent graph structure, and the contrastive learning loss function is obtained. S412. Using the pre-acquired unlabeled image data, set a pseudo-label generation function in the temporal consistent graph structure, generate pseudo-labels according to the pseudo-label generation function, and construct a pseudo-label supervision loss function through the pseudo-labels. S413. Combine the pseudo-label supervision loss function and the contrastive learning loss function to obtain the joint loss function. Optimize the parameters of the three-flow graph neural network model using the joint loss function to obtain the optimized three-flow graph neural network model. S414. Train the optimized three-flow graph neural network model, embed urban and rural planning regulations into the three-flow graph edge weight calculation process, and perform regulatory correction on the edge connection weights in the time-series consistent graph structure to obtain the trained three-flow graph neural network model.

8. The method for monitoring urban illegal construction expansion using remote sensing based on a three-flow graph neural network according to claim 6, characterized in that, The method utilizes a trained three-flow graph neural network model to generate cross-modal fusion graph feature representations embedding regulatory knowledge. Furthermore, reasoning is performed on the cross-modal fusion graph feature representation to obtain a node-level probability graph of illegal construction changes, including: S421. Generate a cross-modal fusion graph feature representation with embedded regulatory knowledge based on the trained three-flow graph neural network model, and input the cross-modal fusion graph feature representation into the node-level category discrimination model; S422. By using the learnable classification weight matrix in the node-level category discrimination model, the cross-modal fusion graph feature representation is mapped to the illegal construction change category space to obtain the mapping result; S423. Normalize the mapping result using a normalization function to obtain a set of probabilities for node-level illegal construction change categories, and generate a node-level illegal construction change probability map based on the set of probabilities for node-level illegal construction change categories.

9. The method for monitoring urban illegal construction expansion using remote sensing based on a three-flow graph neural network according to claim 6, characterized in that, The conditional random field post-processing of the node-level illegal construction change probability map yields three types of illegal construction change information, and the city's illegal construction alarm records are generated based on this information, including: S431. Perform conditional random field post-processing on the node-level illegal construction change probability map to generate a pixel-level urban illegal construction change mask map. S432. By performing a difference operation between the pixel-level urban illegal construction change mask map and the pre-acquired historical urban illegal construction change mask map, three types of illegal construction change information are obtained. S433. Write the three types of illegal construction change information into the city geographic information system to generate urban illegal construction alarm records containing spatial coordinates, change type and violation clause labels.

Images