An intelligent monitoring system and method based on multi-source fusion and meta learning

Abstract

The application belongs to the technical field of remote sensing monitoring and artificial intelligence, and particularly relates to an intelligent monitoring system and method based on multi-source fusion and meta learning, which comprises: a targeted verification module, which converts unstructured location description into a vector space range by analyzing project scheme text containing prior knowledge, generates a self-adaptive buffer zone to determine a monitoring target area, and performs accurate spatio-temporal matching on multi-source remote sensing images, and then extracts disturbance patches through a multi-modal feature pyramid network model; a meta learning module, which automatically constructs a high-quality training sample library through a generative adversarial active learning framework based on the disturbance patches; and a compliance diagnosis module, which constructs a spatio-temporal graph based on the monitoring target area and the disturbance patches, analyzes the spatio-temporal correlation and violation mode of the disturbance patches through a spatio-temporal graph convolution network, analyzes the causes of the violation through a causal model, and provides interpretable violation diagnosis and decision support. The application can significantly improve monitoring accuracy and has strong self-adaptive capability.

Description

Technical Field

[0001] This invention relates to the field of remote sensing monitoring and artificial intelligence, specifically to an intelligent monitoring system and method based on multi-source fusion and meta-learning. Background Technology

[0002] The current supervision of production and construction projects faces the following problems: First, the geographical location descriptions of project planning schemes are mostly unstructured texts, and the spatial scope definition relies on manual interpretation, which is inefficient and prone to errors. Second, the time-series images of traditional remote sensing monitoring have low matching degree with the project construction cycle, which easily misjudges non-engineering disturbances as construction activities, resulting in a large number of false alarms. Third, the characteristics of production and construction disturbances are multi-scale, multi-form, and dynamically evolving, and the generalization ability of a single detection model is poor. Fourth, deep learning detection methods rely on a large number of professionally labeled samples, but the cost of obtaining production and construction disturbance samples is extremely high, which restricts the improvement of model performance. Fifth, existing technologies can only achieve simple change detection, cannot automatically associate project approval information to identify specific violation patterns, and cannot analyze the causes of violations, resulting in weak support for regulatory decision-making.

[0003] Existing monitoring solutions for disturbances in production and construction mainly fall into three categories: pixel-based change detection, object-oriented image analysis, and deep learning segmentation networks. Pixel-based change detection methods, such as image difference and principal component analysis, are sensitive to radiometric differences, easily affected by atmospheric and sensor factors, and cannot distinguish the type of change. While object-oriented image analysis methods can suppress salt-and-pepper noise, the selection of image segmentation scale is difficult, and feature design relies on expert experience. Deep learning segmentation networks, such as U-Net and DeepLab, can automatically learn features, but they are highly dependent on the quantity and quality of labeled samples, and their generalization ability decreases significantly when applied across regions.

[0004] In summary, existing technologies lack a complete framework that deeply integrates project business knowledge, multi-source remote sensing data, and adaptive machine learning algorithms, resulting in problems such as low accuracy, insufficient automation, and lack of regulatory closed loop in production and construction disturbance monitoring, which cannot meet the current needs for intelligent and precise supervision. Summary of the Invention

[0005] In view of the technical deficiencies mentioned in the background art, the purpose of this invention is to provide an intelligent monitoring system and method based on multi-source fusion and meta-learning, aiming to at least partially solve one of the technical problems in the related art.

[0006] To achieve the above objectives, in a first aspect, embodiments of the present invention provide an intelligent monitoring system based on multi-source fusion and meta-learning, the system comprising: The target verification module is used to convert unstructured location descriptions into vector space ranges by parsing project plan texts containing prior knowledge, generate adaptive buffers to determine monitoring target areas, achieve accurate spatiotemporal matching of multi-source remote sensing images through a pre-built multi-scale temporal image alignment model, and then carry out change detection within the monitoring target area through a constructed multimodal feature pyramid network model to extract disturbance patches. The meta-learning module is used to automatically construct a high-quality training sample library based on the perturbation patches by generating an adversarial active learning framework, so as to drive the iterative optimization of the multimodal feature pyramid network model. The compliance diagnosis module is used to construct a spatiotemporal map based on the monitoring target area and the disturbance patches, analyze the spatiotemporal correlation and violation patterns of the disturbance patches through a preset spatiotemporal map convolutional network, and then analyze the causes of violations through a constructed structured causal model, providing interpretable violation diagnosis and decision support.

[0007] As a specific implementation of this application, the generation of the adaptive buffer is also based on the dominant wind direction and water flow direction factors to form a non-uniform buffer, so as to more realistically reflect the diffusion range of the construction impact.

[0008] As a specific implementation of this application, the multi-scale temporal image alignment model models the image registration problem as a spatiotemporal transformation function. The optimization problem, in which: ; The image data is aligned to time t; The original input image (observation data at time t) includes optical images, SAR images, and nighttime light remote sensing data. It is a spacetime transformation function, derived from the parameter set. Control, including geometric correction and radiometric normalization operations; The noise term represents the residual error during the registration process; It is a composite transformation function, containing three key components: (1) Rigid transformation corrects the differences in translation, rotation and scaling of the image and solves the geometric error of the sensor; (2) Seasonal adjustment item The simulation includes periodic changes in vegetation phenology and solar altitude angle; (3) Long-term trend item : Capturing the long-term evolution of land cover; The optimization objective is to minimize the feature matching error and the transformation smoothness constraint.

[0009] in It is a deep feature extractor based on convolutional neural networks, which is more robust to radiation differences than traditional SIFT features; For reference image at position The characteristic response at the location; N is the number of feature matching points; Regularization coefficients control the smoothness of the transformation function and prevent overfitting; For parameters The gradient of the regularization term constrains the local smoothness of the transformation.

[0010] As a specific implementation of this application, the multimodal feature pyramid network model fuses multi-source remote sensing data and extracts multi-scale, multi-level perturbation features. It adopts a three-stage architecture of encoding, fusion, and decoding, uses a multi-head attention mechanism, adaptively weights the feature contributions of different modalities, and performs pyramid feature fusion to decode and output probability maps and classification maps.

[0011] As a specific implementation of this application, the pyramid feature fusion specifically includes: By integrating multi-scale features through top-down and horizontal connections, a balance between large-scale localization and fine-grained boundary segmentation is achieved. The top-down approach involves upsampling high-level semantic features to lower levels, enhancing the semantic information of low-level features and improving small target detection. The lateral connection mechanism, through 1×1 convolution and feature addition / concatenation, preserves the original spatial details while introducing semantic context; By integrating multi-scale features, it achieves high spatial resolution in shallow convolutional layers, making it suitable for fine boundary segmentation; and in deep convolutional layers, it has a large receptive field, making it adept at localizing large-scale targets; and it dynamically adjusts the contribution of each scale through learnable weights.

[0012] As a specific implementation of this application, the spatiotemporal graph is modeled as a graph structure G=(V,E,A); where V represents a node, each perturbation patch is a node, and the corresponding attributes include area, type and detection time; E represents an edge, indicating spatial adjacency or temporal continuity, and A represents an adjacency matrix, describing the relationship between nodes; The spatiotemporal correlation and violation patterns of disturbed patches are analyzed using a pre-defined spatiotemporal graph convolutional network, specifically including: In spatiotemporal graph convolutional networks, multi-layer graph convolutions achieve intelligent recognition from local features to global violation patterns through a hierarchical information aggregation mechanism; this mechanism adopts a layered and progressive architecture: In the first convolutional layer, each node only aggregates information from its direct neighbors to learn local spatial patterns and basic violation features, such as area anomaly detection of a single perturbation patch. After entering the second convolutional layer, the receptive field of the node expands to the second-order neighbor, which can capture regional contextual information and spatial clustering effects, thereby identifying complex patterns such as cooperative violations of adjacent patches. Finally, in the third and deeper convolutional layers, information propagation covers the entire graph, and node features are integrated with the global spatiotemporal context, enabling the understanding of complex spatiotemporal violation patterns, including cross-period and overdue construction and large-scale boundary violations. This allows the network to adaptively learn violation features at different scales, ensuring both accurate capture of local details and intelligent diagnosis of global violation patterns.

[0013] As a specific implementation of this application, the structured causal model constructs a description of the causal relationships between variables and provides interpretable violation diagnosis, specifically including: Construction beyond the scope = f1 (regulatory intensity, company size, project profit, ); Overdue construction = f2 (weather conditions, financial situation, frequency of supervision, ); Intrusion into restricted areas = f3 (geographical awareness, cost pressure, severity of punishment) ); in, Unobserved confounding factors; Furthermore, counterfactual reasoning can answer questions about regulatory decisions, thereby providing decision support.

[0014] Secondly, embodiments of the present invention also provide an intelligent monitoring method based on multi-source fusion and meta-learning, applied to the intelligent monitoring system based on multi-source fusion and meta-learning described in the first aspect, the method comprising the following steps: (1) Prior knowledge-guided target verification: By parsing the project plan text containing prior knowledge, the unstructured location description is transformed into a vector space range, an adaptive buffer is generated to determine the monitoring target area, and a pre-constructed multi-scale temporal image alignment model is used to achieve accurate spatiotemporal matching of multi-source remote sensing images. Then, a multi-modal feature pyramid network model is constructed to carry out change detection in the monitoring target area and extract disturbance patches. (2) Meta-learning adaptive extraction: Based on the perturbation patches, a high-quality training sample library is automatically constructed by generating an adversarial active learning framework to drive the iterative optimization of the multimodal feature pyramid network model; (3) Spatiotemporal causal compliance diagnosis: Based on the monitoring target area and the disturbance patch, a spatiotemporal map is constructed. The spatiotemporal correlation and violation pattern of the disturbance patch are analyzed by a preset spatiotemporal map convolutional network. Then, the cause of violation is analyzed by the constructed structured causal model, providing interpretable violation diagnosis and decision support.

[0015] As one specific implementation of this application, the buffer radius R of the adaptive buffer is not a fixed value, but is based on multi-factor dynamic calculation: ; in: R0 is the base radius, determined by the project type; The terrain relief is calculated using a digital elevation model (DEM). Sc is an ecological sensitivity index that integrates soil erodibility, vegetation cover, and distance from water bodies. α, β, γ are adjustment coefficients.

[0016] The technical solution provided by the embodiments of the present invention has the following significant advantages: 1. Significantly improved monitoring accuracy: Change detection guided by prior knowledge avoids noise interference from the whole-domain search. Combined with the multimodal feature pyramid network model, the Intersection over Union (IoU) of disturbance detection is improved. 2. Significantly reduced sample acquisition costs: The active learning framework transforms the regulatory business process into an automated sample production process. Each verification project is a labeled sample, enabling the sample library to expand continuously at zero cost and solving the problem of deep learning's high dependence on labeled data. 3. Strong adaptability: Through meta-learning and active learning, the system can quickly adapt to new regions and new project types, improving cross-regional generalization performance; 4. Intelligent violation identification: Spatiotemporal graph convolutional networks and causal models can not only identify violation types, but also analyze the causes of violations and potential risks, providing decision support for precise supervision; This solution enables the entire process—from parsing project plan texts, image matching, and change detection to interpretable violation diagnosis—to be highly automated, significantly reducing the workload of manual verification and greatly improving regulatory efficiency. Attached Figure Description

[0017] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below.

[0018] Figure 1This is a block diagram illustrating the principle of an intelligent monitoring system based on multi-source fusion and meta-learning provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the overall working process of an intelligent monitoring system based on multi-source fusion and meta-learning provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the generation of an adaptive buffer from text coordinates provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of a multi-scale temporal image alignment model provided in an embodiment of the present invention; Figure 5 This is an architecture diagram of a multimodal feature pyramid network provided in an embodiment of the present invention; Figure 6 This is a flowchart of a generative adversarial active learning framework provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of a spatiotemporal diagram provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of a spatiotemporal graph convolutional network for violation identification provided in an embodiment of the present invention; Figure 9 This is a flowchart of an intelligent monitoring method based on multi-source fusion and meta-learning provided in an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0021] Throughout this specification, references to "an embodiment," "an example," or "an example" mean that a particular feature, structure, or characteristic described in connection with that embodiment or example is included in at least one embodiment of the invention. Therefore, the phrases "in an embodiment," "in an embodiment," "an example," or "an example" appearing in various places throughout the specification do not necessarily refer to the same embodiment or example. Furthermore, specific features, structures, or characteristics can be combined in one or more embodiments or examples in any suitable combination and / or sub-combination.

[0022] It should be noted that, unless otherwise stated, the technical terms used in this embodiment have the common meaning as understood in the relevant technical field.

[0023] Please refer to Figures 1 to 5 This invention provides an intelligent monitoring system based on multi-source fusion and meta-learning, the system comprising: The target verification module is used to convert unstructured location descriptions into vector space ranges by parsing project plan texts containing prior knowledge, generate adaptive buffers to determine monitoring target areas, achieve accurate spatiotemporal matching of multi-source remote sensing images through a pre-built multi-scale temporal image alignment model, and then carry out change detection within the monitoring target area through a constructed multimodal feature pyramid network model to extract disturbance patches. The meta-learning module is used to automatically construct a high-quality training sample library based on the perturbation patches by generating an adversarial active learning framework, so as to drive the iterative optimization of the multimodal feature pyramid network model. The compliance diagnosis module is used to construct a spatiotemporal map based on the monitoring target area and the disturbance patches, analyze the spatiotemporal correlation and violation patterns of the disturbance patches through a preset spatiotemporal map convolutional network, and then analyze the causes of violations through a constructed structured causal model, providing interpretable violation diagnosis and decision support.

[0024] In this embodiment, prior knowledge refers to information such as approval information, plan text, red line scope, construction period, project type, ecological protection area / ecological red line, etc. in the project plan; The geographical location descriptions in the project plan (such as "center point coordinates: E116°20′15″, N39°55′30″") are first converted into standard WGS84 coordinates using a multi-resolver fusion algorithm. The resolvers include: Regular expression parser: Matches common coordinate formats (degrees, minutes, seconds, decimal degrees); Named Entity Recognition (NER) Parser: Identifies location names in text based on a pre-trained geographic model; Large Language Model (LLM) parser: Understands complex and ambiguous location descriptions by fine-tuning a large geographic model through instructions.

[0025] After coordinate transformation, the system automatically generates an adaptive buffer; the buffer radius R of the adaptive buffer is not a fixed value, but is based on dynamic calculation of multiple factors. ; in: R0 is the base radius, determined by the project type; The terrain relief is calculated using a digital elevation model (DEM). Sc is an ecological sensitivity index that integrates soil erodibility, vegetation cover, and distance from water bodies. α, β, γ are adjustment coefficients.

[0026] In application, the adaptive buffer is also generated based on the prevailing wind direction and water flow direction to form a non-uniform buffer, so as to more realistically reflect the diffusion range of the construction impact.

[0027] To achieve accurate matching of images from different times and different sensors, this invention proposes a multi-scale temporal image alignment model; this model models the image registration problem as a spatiotemporal transformation function. The optimization problem, in which: ; The image data is aligned to time t; The original input image (observation data at time t) includes optical images, SAR images, and nighttime light remote sensing data. It is a spacetime transformation function, derived from the parameter set. Control, including geometric correction and radiometric normalization operations; The noise term represents the residual error during the registration process; It is a composite transformation function, containing three key components: (1) Rigid transformation corrects the differences in translation, rotation and scaling of the image and solves the geometric error of the sensor; (2) Seasonal adjustment item The simulation includes periodic changes in vegetation phenology and solar altitude angle; (3) Long-term trend item : Capturing the long-term evolution of land cover; The optimization objective is to minimize the feature matching error and the transformation smoothness constraint.

[0028] in It is a deep feature extractor based on convolutional neural networks, which is more robust to radiation differences than traditional SIFT features; For reference image at position The characteristic response at the location; N is the number of feature matching points; Regularization coefficients control the smoothness of the transformation function and prevent overfitting; For parameters The gradient of the regularization term constrains the local smoothness of the transformation.

[0029] To address the shortcomings of traditional methods in multimodal data collaboration and multi-scale feature extraction, the proposed multimodal feature pyramid network model integrates multi-source remote sensing data and extracts multi-scale, multi-level perturbation features. It adopts a three-stage architecture of encoding, fusion, and decoding, employs a multi-head attention mechanism, adaptively weights the feature contributions of different modalities, and performs pyramid feature fusion to decode and output probability maps and classification maps.

[0030] 1. Multimodal input encoding: Optical image feature extraction: Input channels: RGB, near-infrared, and short-wave infrared multispectral bands; Feature types: spectral features, vegetation indices (such as NDVI), and water body indices, etc. Encoding network: Use shallow convolutional layers of ResNet or VGG to extract low-level visual features.

[0031] SAR Image Feature Analysis: Polarization channels: VV (vertical transmit-vertical receive), VH (vertical transmit-horizontal receive) polarization data; VV polarization: sensitive to surface roughness, suitable for identifying construction machinery and building materials; VH polarization: It responds significantly to changes in vegetation structure and soil moisture.

[0032] Nighttime light remote sensing data integration: Data sources: Nighttime light remote sensing satellites such as VIIRS and DMSP; Characteristic value: Directly reflects the intensity of human activity, especially suitable for monitoring nighttime construction activities; Normalization process: Eliminate interference from natural light sources such as moonlight and haze.

[0033] 2. Multi-level feature extraction: Low-level features (convolutional shallow layer P2): basic visual elements such as edges, textures, and color spots, used for fine boundary detection, such as the identification of the transition zone between construction areas and natural vegetation; Mid-layer features (convolutional mid-layer P3-P4): object components, structural information, local context, to identify typical construction elements (such as temporary sheds, material storage areas). High-level features (deep convolutional layer P5): semantic information, global context, and abstract patterns, to understand the overall concept of "construction disturbance" and distinguish different types of projects.

[0034] 3-modal attention fusion: A multi-head attention mechanism is employed to adaptively weight the feature contributions of different modalities; ; Where Q, K, and V are feature maps from different modalities; Q (Query) comes from the feature map of the dominant mode (usually an optical image); K (Key) is the feature key of the auxiliary mode (such as SAR or night light data). V (Value) is the eigenvalue to be weighted; Feature dimension scaling factor to prevent the inner product from becoming too large; In this context, T refers to the matrix transpose operation; Simultaneously, adaptive weight allocation is implemented, specifically including: Optical-dominated mode: In densely vegetated areas, optical images are given higher weight. SAR Enhancement Mode: The Importance of Enhancing SAR Features in Cloud-Covered Areas Night Light Assist Mode: Enhances the contribution of night light data for nighttime construction.

[0035] 4. Pyramid Feature Fusion: By integrating multi-scale features (P2-P5) through top-down and horizontal connections, a balance between large-scale localization and fine-grained boundary segmentation is achieved. The top-down approach involves upsampling high-level semantic features (P5) to lower levels to enhance the semantic information of low-level features and improve small target detection. The lateral connection mechanism, through 1×1 convolution and feature addition / concatenation, preserves the original spatial details while introducing semantic context; By integrating multi-scale features, it achieves high spatial resolution at P2 level, making it suitable for fine boundary segmentation; at P5 level, it has a large receptive field and excels at localizing large-scale targets; and it dynamically adjusts the contribution of each scale through learnable weights.

[0036] It should be noted that in the prior knowledge-guided targeted verification process, "targeted verification" refers to the process of verifying and confirming suspected disturbed patches in the time-series image matching results based on the prior knowledge output from the project plan, spatial analysis, and buffer generation (such as the spatial scope, functional zoning, and temporal arrangement of the project plan).

[0037] Specifically, firstly, spatial constraints on disturbance behaviors (such as the boundaries of construction areas and the avoidance range of ecologically sensitive areas) are extracted from the project plan through "spatial analysis". Then, the spatially related areas (such as the possible spread range of construction impact) are expanded through "buffer zone generation". Subsequently, based on the candidate disturbance patches obtained by "temporal image matching", and combined with the spatial, temporal, and functional constraints of the above-mentioned prior knowledge, patches that have potential conflicts or deviations from the logic of the project plan (such as construction sequence and spatial layout) are selected as "target objects". The "disturbance attributes (whether it is illegal, the type of disturbance, etc.)" of these target objects are accurately verified through high-resolution image comparison, on-site survey data verification or domain expert experience judgment.

[0038] In the GAAL framework, "targeted verification" refers to the process of human expert annotation and verification of high-value samples selected based on active learning strategies.

[0039] In this approach, meta-learning and active learning are not isolated but embedded in the generative adversarial active learning framework (GAAL) to jointly address the two core pain points of extremely scarce samples and poor cross-regional generalization ability.

[0040] In this approach, meta-learning is primarily used as a model initialization and pre-training strategy, aiming to enable the detection model to "learn how to learn," a concept referred to as "learning the learning of learning." This approach finds optimal initial model parameters through pre-training on a large amount of source domain data from different regions and of different types. Thus, when faced with a new production construction project (target domain), even with only a very small number of labeled samples, the model can quickly adapt to local terrain features through a few iterations.

[0041] In this approach, active learning serves as a sample selection mechanism, used to choose the most valuable samples for model improvement from the massive datasets generated by Generative Adversarial Networks (GANs) for manual verification. Active learning simulates the inquisitive behavior of humans. Unlike traditional deep learning, which requires large amounts of randomly labeled data, the active learning algorithm proactively identifies the samples it is most uncertain about and most prone to confusion, and requests expert (human) annotation. It learns to select samples based on uncertainty, representativeness, and diversity.

[0042] In summary, meta-learning is responsible for "speed" (quickly adapting to new regions and projects); active learning is responsible for "efficiency" (significantly reducing annotation costs and improving sample utilization); the two are deeply integrated through the GAAL framework, jointly achieving a technological leap from "sample scarcity" to "self-generation and self-optimization".

[0043] In this embodiment, to address the problem of sample scarcity, the present invention proposes a Generative Adversarial Active Learning (GAAL) framework, which combines Generative Adversarial Networks (GANs) and active learning to achieve automatic generation and intelligent selection of training samples.

[0044] Combination Figure 6 Its core process is as follows: Generator (G): Based on a small number of real samples, learns the distribution pdata(x) of perturbation patches and generates realistic synthetic samples G(z) (z is a noise vector).

[0045] Discriminator (D): Receives both real and generated samples and outputs the probability that a sample is real. Through adversarial training, the generated samples are encouraged to continuously approximate the real distribution.

[0046] Active learning sample selection: Although many samples are generated, it is necessary to select the most valuable samples for improving the model; this invention adopts a multi-criteria fusion strategy: Uncertainty: Select the sample with the largest prediction entropy H(x).

[0047] Diversity: Select the sample that is furthest from the existing samples in the feature space, and avoid selecting similar samples to ensure the diversity of the training set.

[0048] Representativeness: Select the sample that is closest to the distribution of unlabeled data (measured by the maximum mean difference, MMD).

[0049] The final selection score is: ; For the sample The final selection score; The generated sample is to be evaluated.

[0050] in, The prediction entropy represents the uncertainty of the model's classification of that sample. The uncertainty weighting coefficient is (0.4-0.6). To control for diversity weights, they are typically set to 0.2-0.3. The representativeness weight is usually set to 0.2-0.3; see Table 1 for details.

[0051] ; In the formula, Number of categories (perturbation / non-perturbation). Let c be the probability that the sample belongs to class c. For the diversity criterion, ; Let L be the embedding representation of the sample in the feature space, and L be the set of labeled samples. As a representative criterion; ; A collection of individual generated samples; Unlabeled dataset ; MMD, or Maximum Mean Discrepancy, is a statistical test used to measure the difference between two probability distributions. In the representativeness criterion, MMD is used to measure the distribution difference between a single generated sample x and an unlabeled dataset U.

[0052] The specific formula for calculating MMD in the representativeness criterion is as follows: ; P: Distribution of the generated sample x (single sample empirical distribution); Q: Unlabeled dataset The empirical distribution; : A feature mapping function that maps samples to the reproducing kernel Hilbert space (RKHS); Regenerating kernel Hilbert space to guarantee computational truncation. The mathematical expectation of a distribution P; The mathematical expectation on the distribution Q; Table 1 Parameter Range and Explanation

[0053] High-scoring samples were selected for rapid manual validation, and then added to the training set after validation.

[0054] In this embodiment, the spatiotemporal graph is modeled as a graph structure G=(V,E,A); where V represents a node, each perturbation patch is a node, and the corresponding attributes include area, type and detection time; E represents an edge, indicating spatial adjacency or temporal continuity, and A represents an adjacency matrix, describing the relationship between nodes; The spatiotemporal correlation and violation patterns of disturbed patches are analyzed using a pre-defined spatiotemporal graph convolutional network, specifically including: In spatiotemporal graph convolutional networks, multi-layer graph convolutions achieve intelligent recognition from local features to global violation patterns through a hierarchical information aggregation mechanism; this mechanism adopts a layered and progressive architecture: In the first convolutional layer, each node only aggregates information from its direct neighbors to learn local spatial patterns and basic violation features, such as area anomaly detection of a single perturbation patch. After entering the second convolutional layer, the receptive field of the node expands to the second-order neighbor, which can capture regional contextual information and spatial clustering effects, thereby identifying complex patterns such as cooperative violations of adjacent patches. Finally, in the third and deeper convolutional layers, information propagation covers the entire graph, and node features are integrated with the global spatiotemporal context, enabling the understanding of complex spatiotemporal violation patterns, including cross-period and overdue construction and large-scale boundary violations. This allows the network to adaptively learn violation features at different scales, ensuring both accurate capture of local details and intelligent diagnosis of global violation patterns.

[0055] Graph structure G=(V,E,A); node v i Each perturbation patch has attributes including area, type, and detection time. Spatial attributes: center coordinates, area, and shape index of the map patch; Time attributes: detect timestamp, duration, and rate of change; Feature attributes: spectral features, texture features, and intensity of variation; Business attributes: Project ID, Project type, and Approval status.

[0056] Side e ij Spatial adjacency (distance less than a threshold) or temporal continuity (same location at different times); Spatial adjacency: Pieces with a distance less than a threshold are connected; Temporally continuous: Patches detected at the same location at different times are connected; Hybrid relation: A strong association that simultaneously satisfies spatial proximity and temporal continuity.

[0057] Adjacency matrix A: describes the relationships between nodes.

[0058] ; The i-th node in the graph represents a specific perturbation patch. The j-th node in the graph represents a specific perturbation patch; d is the Euclidean distance between nodes; This represents a preset spatial proximity threshold. If the distance between two patches is less than this value, they are considered spatially adjacent. The time difference between nodes (based on their detection timestamp); Indicates the preset spatial proximity threshold This represents a preset temporal proximity threshold. If the detection time interval between two patches is less than this value, they are considered to be temporally continuous.

[0059] Spatiotemporal Graph Convolutional Networks (ST-GCN) aggregate neighborhood information through graph convolutional layers: ; in, Adjacency matrix for adding self-loops; It is an identity matrix to ensure the transmission of information within each node; This is the output feature matrix of the (l+1)th layer; For degree matrix, ; The degree matrix raised to the power of -1 / 2 is used for normalization; Let l be the node feature matrix of the l-th layer. ; Let be the learnable weight matrix of the l-th layer. ; The activation function is typically ReLU or LeakyReLU.

[0060] The node features, after being processed by multi-layer graph convolution, are then used by a classifier to determine the final violation type. The mathematical expression for the classifier is: ; In the formula, Let i be the feature vector output by the graph convolution at layer L (the last layer) of the i-th node. MLP stands for Multi-Layer Perceptron, responsible for feature transformation and nonlinear mapping. Its mathematical expression is: ; In the formula, , For learnable weight matrix, , For bias vectors, The activation function is typically ReLU or LeakyReLU.

[0061] The mathematical expression for converting the raw scores output by the MLP into a probability distribution is as follows: ; In the formula, Let be the original score vector output by the MLP, where is the original score for the j-th category. This represents the total number of violation categories (usually 5).

[0062] Let be the probability distribution of violation types for node i, where each element represents the confidence probability of the corresponding violation category. Typically, there are 5 categories: The probability of compliant construction; The probability of construction exceeding the permitted scope; The probability of construction exceeding the schedule; : The probability of intrusion into an ecological protection zone; The probability of construction without prior approval.

[0063] Reference Figure 7 By using a hierarchical and temporal connection approach, the perturbation patches within the same spatial region are visually represented, continuously evolving over time to form a path in the spatiotemporal graph. If different spatial regions are adjacent, connections should also be established to form a complete spatiotemporal graph network. This is the basic data structure of the ST-GCN violation identification module in this scheme, providing graph structure support for subsequent spatiotemporal feature aggregation and violation pattern mining. In the figure, the horizontal axis (X) represents the time dimension, the vertical axis (Y) represents the spatial dimension, the circles represent nodes, and A, B, C, and D represent perturbation patch instances at different times and spatial locations. Horizontal straight lines connect nodes in the same row (same spatial level), indicating that there is a temporal correlation between them.

[0064] Reference Figure 8 This figure illustrates the confidence (probability) of the violation identification model constructed in this invention in determining violations for different types of production and construction disturbance behaviors, and introduces a high-risk threshold to assist regulatory decision-making. The horizontal axis of the chart represents the probability value (range 0.0 to 1.0), and the vertical axis represents five typical construction behaviors, including "compliant construction," "construction without approval," "intrusion into ecological protection zones," "construction beyond the construction period," and "construction beyond the scope." Each behavior corresponds to a bar, and its length and the numerical value at the end indicate the probability that the behavior is determined to be a violation by the model.

[0065] The final violation determination is based on a probability threshold:

[0066] Basic threshold parameters: The compliance threshold is typically set at 0.7. The threshold for construction beyond the permitted scope is typically 0.6. The threshold for construction exceeding the schedule is typically 0.6. Intrusion threshold for protected areas, typically 0.55 (more sensitive); The threshold for construction without prior approval is typically 0.65. Minimum probability threshold for violations, typically 0.3 (to prevent low-probability violations from interfering with compliance assessments). The threshold for determining multiple violations usually requires two or more violations with a probability ≥ 0.5.

[0067] Furthermore, to provide interpretable violation diagnoses, the structured causal model constructs descriptions of causal relationships between variables, specifically including: Construction beyond the scope = f1 (regulatory intensity, company size, project profit, ); Overdue construction = f2 (weather conditions, financial situation, frequency of supervision, ); Intrusion into restricted areas = f3 (geographical awareness, cost pressure, severity of punishment) ); in, Unobserved confounding factors; Furthermore, counterfactual reasoning can answer questions about regulatory decisions, thereby providing decision support; For example: "If the frequency of supervision is increased by 50%, by how much will the probability of construction exceeding the permitted scope decrease?" Calculate the average treatment effect (ATE): ; Where Y(1) and Y(0) represent the results of intervention and no intervention, respectively; finally, a corresponding report is generated.

[0068] The above solution has the following significant advantages: 1. Significantly improved monitoring accuracy: Change detection guided by prior knowledge avoids noise interference from the whole-domain search. Combined with the multimodal feature pyramid network model, the Intersection over Union (IoU) of disturbance detection is improved. 2. Significantly reduced sample acquisition costs: The active learning framework transforms the regulatory business process into an automated sample production process. Each verification project is a labeled sample, enabling the sample library to expand continuously at zero cost and solving the problem of deep learning's high dependence on labeled data. 3. Strong adaptability: Through meta-learning and active learning, the system can quickly adapt to new regions and new project types, improving cross-regional generalization performance; 4. Intelligent violation identification: Spatiotemporal graph convolutional networks and causal models can not only identify violation types, but also analyze the causes of violations and potential risks, providing decision support for precise supervision; This solution enables the entire process—from parsing project plan texts, image matching, and change detection to interpretable violation diagnosis—to be highly automated, significantly reducing the workload of manual verification and greatly improving regulatory efficiency.

[0069] Based on the same inventive concept, this embodiment of the invention also provides an intelligent monitoring method based on multi-source fusion and meta-learning, applied to the intelligent monitoring system based on multi-source fusion and meta-learning described in the first aspect, with reference to... Figure 9 The method includes the following steps: S101, Prior Knowledge-Guided Target Verification: By parsing the project plan text containing prior knowledge, the unstructured location description is transformed into a vector space range, an adaptive buffer is generated to determine the monitoring target area, and a pre-built multi-scale temporal image alignment model is used to achieve accurate spatiotemporal matching of multi-source remote sensing images. Then, a multi-modal feature pyramid network model is constructed to carry out change detection in the monitoring target area and extract disturbance patches. S102, Meta-learning adaptive extraction: Based on the perturbation patch, a high-quality training sample library is automatically constructed by generating an adversarial active learning framework to drive the iterative optimization of the multimodal feature pyramid network model; S103, Spatiotemporal Causal Compliance Diagnosis: Based on the monitoring target area and the disturbance patches, a spatiotemporal map is constructed. The spatiotemporal correlation and violation patterns of the disturbance patches are analyzed through a preset spatiotemporal map convolutional network. Then, the causes of violations are analyzed through the constructed structured causal model, providing interpretable violation diagnosis and decision support.

[0070] Furthermore, the buffer radius R of the adaptive buffer is not a fixed value, but is calculated dynamically based on multiple factors: ; in: R0 is the base radius, determined by the project type; The terrain relief is calculated using a digital elevation model (DEM). Sc is an ecological sensitivity index that integrates soil erodibility, vegetation cover, and distance from water bodies. α, β, γ are adjustment coefficients.

[0071] The multimodal feature pyramid network model fuses multi-source remote sensing data and extracts multi-scale, multi-level perturbation features. It adopts a three-stage architecture of encoding, fusion, and decoding, uses a multi-head attention mechanism, adaptively weights the feature contributions of different modalities, and performs pyramid feature fusion to decode and output probability maps and classification maps.

[0072] In this embodiment, the spatiotemporal graph is modeled as a graph structure G=(V,E,A); where V represents a node, each perturbation patch is a node, and the corresponding attributes include area, type and detection time; E represents an edge, indicating spatial adjacency or temporal continuity, and A represents an adjacency matrix, describing the relationship between nodes; The spatiotemporal correlation and violation patterns of disturbed patches are analyzed using a pre-defined spatiotemporal graph convolutional network, specifically including: In spatiotemporal graph convolutional networks, multi-layer graph convolutions achieve intelligent recognition from local features to global violation patterns through a hierarchical information aggregation mechanism; this mechanism adopts a layered and progressive architecture: In the first convolutional layer, each node only aggregates information from its direct neighbors to learn local spatial patterns and basic violation features, such as area anomaly detection of a single perturbation patch. After entering the second convolutional layer, the receptive field of the node expands to the second-order neighbor, which can capture regional contextual information and spatial clustering effects, thereby identifying complex patterns such as cooperative violations of adjacent patches. Finally, in the third and deeper convolutional layers, information propagation covers the entire graph, and node features are integrated with the global spatiotemporal context, enabling the understanding of complex spatiotemporal violation patterns, including cross-period and overdue construction and large-scale boundary violations. This allows the network to adaptively learn violation features at different scales, ensuring both accurate capture of local details and intelligent diagnosis of global violation patterns.

[0073] It should be noted that for a more detailed description of the workflow of the method embodiments, please refer to the aforementioned system embodiments section, which will not be repeated here.

[0074] The specific beneficial effects of the entire plan include: 1. Significantly improved monitoring accuracy: Change detection guided by prior knowledge avoids noise interference from the whole-domain search. Combined with the multimodal feature pyramid network model, the Intersection over Union (IoU) of disturbance detection is improved. 2. Significantly reduced sample acquisition costs: The active learning framework transforms the regulatory business process into an automated sample production process. Each verification project is a labeled sample, enabling the sample library to expand continuously at zero cost and solving the problem of deep learning's high dependence on labeled data. 3. Strong model adaptability: Through meta-learning and active learning, the system can quickly adapt to new regions and new project types, improving cross-regional generalization performance; 4. Intelligent violation identification: Spatiotemporal graph convolutional networks and causal models can not only identify violation types, but also analyze the causes of violations and potential risks, providing decision support for precise supervision; This solution enables the entire process—from parsing project plan texts, image matching, and change detection to interpretable violation diagnosis—to be highly automated, significantly reducing the workload of manual verification and greatly improving regulatory efficiency.

[0075] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can also be implemented in other ways. The embodiments described above are merely illustrative. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the figures. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified functions or actions, or using a combination of dedicated hardware and computer instructions.

[0076] Furthermore, the functional modules in the various embodiments of this invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. When using each module, information is collected and stored only with the full authorization of the relevant user or organization and in compliance with relevant laws and regulations, and the security and privacy of the data are protected. Unauthorized access is strictly prohibited. Data processing will be carried out within the scope stipulated by law and will not exceed the authorized purpose and scope. At the same time, the authorizing party has the right to access, correct, delete, restrict processing, refuse, etc., of its personal data, and strictly comply with applicable laws and regulations and conduct compliance reviews.

[0077] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. For those skilled in the art, this invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An intelligent monitoring system based on multi-source fusion and meta-learning, characterized in that, The system includes: The target verification module is used to convert unstructured location descriptions into vector space ranges by parsing project plan texts containing prior knowledge, generate adaptive buffers to determine monitoring target areas, achieve accurate spatiotemporal matching of multi-source remote sensing images through a pre-built multi-scale temporal image alignment model, and then carry out change detection within the monitoring target area through a constructed multimodal feature pyramid network model to extract disturbance patches. The meta-learning module is used to automatically construct a high-quality training sample library based on the perturbation patches by generating an adversarial active learning framework, so as to drive the iterative optimization of the multimodal feature pyramid network model. The compliance diagnosis module is used to construct a spatiotemporal map based on the monitoring target area and the disturbance patches, analyze the spatiotemporal correlation and violation patterns of the disturbance patches through a preset spatiotemporal map convolutional network, and then analyze the causes of violations through a constructed structured causal model, providing interpretable violation diagnosis and decision support. The multi-scale temporal image alignment model models the image registration problem as a spatiotemporal transformation function. The optimization problem, in which: ； For time t Aligned image data; The original input images include optical images, SAR images, and nighttime light remote sensing data. It is a spacetime transformation function, derived from the parameter set. Control, including geometric correction and radiometric normalization operations; The noise term represents the residual error during the registration process; It is a composite transformation function, containing three key components: (1) Rigid transformation corrects the differences in translation, rotation and scaling of the image and solves the geometric error of the sensor; (2) Seasonal adjustment item The simulation includes periodic changes in vegetation phenology and solar altitude angle; (3) Long-term trend item : Capturing the long-term evolution of land cover; The optimization objective is to minimize the feature matching error and the transformation smoothness constraint. in It is a deep feature extractor based on convolutional neural networks, which is more robust to radiation differences than traditional SIFT features; For reference image at position The characteristic response at the location; N This represents the number of feature matching points; Regularization coefficients control the smoothness of the transformation function and prevent overfitting; For parameters The gradient of the regularization term constrains the local smoothness of the transformation.

2. The system as described in claim 1, characterized in that, The adaptive buffer is also generated based on the prevailing wind direction and water flow direction, forming a non-uniform buffer to more realistically reflect the spread of construction impacts.

3. The system as described in claim 2, characterized in that, The multimodal feature pyramid network model fuses multi-source remote sensing data and extracts multi-scale, multi-level perturbation features. It adopts a three-stage architecture of encoding, fusion, and decoding, uses a multi-head attention mechanism, adaptively weights the feature contributions of different modalities, and performs pyramid feature fusion to decode and output probability maps and classification maps.

4. The system as described in claim 3, characterized in that, The pyramid feature fusion specifically includes: By integrating multi-scale features through top-down and horizontal connections, a balance between large-scale localization and fine-grained boundary segmentation is achieved. The top-down approach involves upsampling high-level semantic features to lower levels, enhancing the semantic information of low-level features and improving small target detection. The lateral connection mechanism, through 1×1 convolution and feature addition / concatenation, preserves the original spatial details while introducing semantic context; By integrating multi-scale features, it achieves high spatial resolution in shallow convolutional layers, making it suitable for fine boundary segmentation; and in deep convolutional layers, it has a large receptive field, making it adept at localizing large-scale targets; and it dynamically adjusts the contribution of each scale through learnable weights.

5. The system as described in claim 4, characterized in that, The spatiotemporal graph is modeled as a graph structure G=(V,E,A); where V represents a node, each perturbation patch is a node, and the corresponding attributes include area, type and detection time; E represents an edge, indicating spatial adjacency or temporal continuity, and A represents an adjacency matrix, describing the relationship between nodes; The spatiotemporal correlation and violation patterns of disturbed patches are analyzed using a pre-defined spatiotemporal graph convolutional network, specifically including: In spatiotemporal graph convolutional networks, multi-layer graph convolutions achieve intelligent recognition from local features to global violation patterns through a hierarchical information aggregation mechanism; this mechanism adopts a layered and progressive architecture: In the first convolutional layer, each node only aggregates information from its immediate neighbors to learn local spatial patterns and basic violation features; After entering the second convolutional layer, the receptive field of the node expands to the second-order neighbor, which can capture regional contextual information and spatial clustering effects, thereby identifying complex patterns of cooperative violations by adjacent patches. Finally, in the third and deeper convolutional layers, information propagation covers the entire graph, and node features are integrated with the global spatiotemporal context, enabling the understanding of complex spatiotemporal violation patterns, including cross-period and overdue construction and large-scale boundary violations. This allows the network to adaptively learn violation features at different scales, ensuring both accurate capture of local details and intelligent diagnosis of global violation patterns.

6. The system as described in claim 5, characterized in that, The structured causal model constructs descriptions of causal relationships between variables and provides interpretable violation diagnoses, specifically including: Construction beyond the scope = f 1 (Regulatory intensity, enterprise size, project profit, ε1); Overdue construction = f 2 (weather conditions, financial situation, frequency of regulation, ε2); Invading the restricted area = f 3 (Geographical awareness, cost pressure, penalty intensity, ε1); Where ε represents unobserved confounding factors; Furthermore, counterfactual reasoning can answer questions about regulatory decisions, thereby providing decision support.

7. An intelligent monitoring method based on multi-source fusion and meta-learning, characterized in that, The method applied to the intelligent monitoring system based on multi-source fusion and meta-learning as described in claim 1 includes the following steps: (1) Prior knowledge-guided target verification: By parsing the project plan text containing prior knowledge, the unstructured location description is transformed into a vector space range, an adaptive buffer is generated to determine the monitoring target area, and a pre-constructed multi-scale temporal image alignment model is used to achieve accurate spatiotemporal matching of multi-source remote sensing images. Then, a multi-modal feature pyramid network model is constructed to carry out change detection in the monitoring target area and extract disturbance patches. (2) Meta-learning adaptive extraction: Based on the perturbation patches, a high-quality training sample library is automatically constructed by generating an adversarial active learning framework to drive the iterative optimization of the multimodal feature pyramid network model; (3) Spatiotemporal causal compliance diagnosis: Based on the monitoring target area and the disturbance patch, a spatiotemporal map is constructed. The spatiotemporal correlation and violation pattern of the disturbance patch are analyzed by a preset spatiotemporal map convolutional network. Then, the cause of violation is analyzed by the constructed structured causal model, providing interpretable violation diagnosis and decision support.

8. The method as described in claim 7, characterized in that, The buffer radius R of the adaptive buffer is not a fixed value, but is calculated dynamically based on multiple factors: ； in: R0 is the base radius, determined by the project type; ▽z represents the terrain relief, calculated using a digital elevation model; Sc is an ecological sensitivity index that integrates soil erodibility, vegetation cover, and distance from water bodies. α, β, γ are adjustment coefficients.

9. The method as described in claim 8, characterized in that, The multimodal feature pyramid network model fuses multi-source remote sensing data and extracts multi-scale, multi-level perturbation features. It adopts a three-stage architecture of encoding, fusion, and decoding, uses a multi-head attention mechanism, adaptively weights the feature contributions of different modalities, and performs pyramid feature fusion to decode and output probability maps and classification maps.

Images