Pollution event assessment method based on multi-modal incremental learning and dynamic optimization

Abstract

The application provides a pollution event evaluation method based on multi-modal incremental learning and dynamic optimization, comprising: processing multi-modal data by using a self-supervised pre-training model introducing an incremental learning mechanism to obtain deep embedding features of each modal semantic level; constructing a Laplacian matrix of pollution monitoring nodes by using geographical topological relations, introducing a time period coding module to decompose time information, and obtaining joint representation of pollution events in spatial layout and time evolution through fusion mapping of topological embedding and period embedding; using a learnable compression module to perform information quantity perception compression on the embedding vectors of each modal, and dynamically adjusting the feature dimension; introducing a domain adaptive gated fusion network, combining environmental domain knowledge and data-driven attention mechanism, dynamically adjusting the contribution degree of different modal features, and using the fusion features to classify pollution events. The application fully utilizes multi-source information, and improves the accuracy of pollution event recognition.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and in particular relates to a pollution event assessment method based on multimodal incremental learning and dynamic optimization. Background Technology

[0002] Environmental pollution incidents are typically sudden, multi-source, heterogeneous, and strongly correlated in time and space, making their monitoring and assessment particularly complex. Traditional pollution incident detection methods mainly rely on a single data source, such as data from air quality monitoring instruments, or on images and video data reported by manual observation.

[0003] Although existing technologies have made some progress in the identification and assessment of environmental pollution incidents, there are still many limitations in practical applications, especially in the processing of multimodal data fusion, spatiotemporal modeling and incremental learning.

[0004] Existing technologies mainly face the following drawbacks:

[0005] Most existing technologies rely on traditional feature concatenation or simple weighting methods for multimodal data fusion, lacking the ability to adaptively learn the dynamic relationships between different modalities. These methods usually fuse data based on predetermined weights, ignoring the differences in importance of each modality in different environmental pollution events. They often cannot dynamically adjust the modality fusion strategy in each specific event, resulting in unstable model performance in multimodal data environments, especially in the identification and assessment of new types of pollution events.

[0006] Most existing spatiotemporal modeling methods process spatial and temporal information separately, failing to effectively integrate spatiotemporal dependencies. Traditional spatiotemporal models often ignore the periodic characteristics of pollution events at different time scales, resulting in the models being unable to accurately capture the long-term trends and periodic fluctuations of pollutant diffusion.

[0007] Concept drift and data flow problems are very common in environmental pollution monitoring, especially with the emergence of new pollution sources. Most existing incremental learning methods rely on static model adjustments and lack the ability to continuously adapt to new data sources and scenarios. Although some methods attempt to use lightweight incremental updates to cope with changes in data distribution, they are inadequate when dealing with dynamic data flows across modalities due to the lack of collaborative incremental update strategies for multimodal data. In addition, existing incremental learning frameworks are usually unable to effectively handle large-scale data or data flows over long periods of time, making it difficult to maintain model performance in the real-time identification and assessment of pollution events. Summary of the Invention

[0008] The purpose of this invention is to provide a pollution event assessment method based on multimodal incremental learning and dynamic optimization, aiming to solve the technical problems mentioned in the background.

[0009] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions.

[0010] According to one embodiment of this application, a pollution event assessment method based on multimodal incremental learning and dynamic optimization is provided, comprising the following steps:

[0011] Step S101: Collect multimodal data from the environmental pollution detection platform, and process the multimodal data using a self-supervised pre-trained model with an incremental learning mechanism to obtain deep embedding features at the semantic level of each modality.

[0012] Step S102: Construct the Laplace matrix of pollution monitoring nodes using geographic topological relationships, and introduce a time period coding module to decompose time information into daily, weekly and seasonal components. Through the fusion mapping of topological embedding and periodic embedding, obtain the joint representation of pollution events in spatial layout and temporal evolution.

[0013] Step S103: Use a learnable compression module to compress the embedding vectors of each modality with information awareness and dynamically adjust the feature dimension to balance information preservation and computational efficiency. In the compression module, mutual information estimation and reconstruction error are optimized to output the intra-modal representation matrix. Semantic aggregation is driven by a parameterized attention mechanism to fuse the information of the corresponding modality into a concise intra-modal summary vector.

[0014] Step S104: Introduce a domain-adaptive gating fusion network, combining environmental domain knowledge with a data-driven attention mechanism to dynamically adjust the contribution of different modal features, thereby obtaining fused features;

[0015] Step S105: Classify pollution events using fusion features. Based on the classification results, classify the same event into different levels from multiple dimensions and output independent level results. Adaptively weight the level results of each dimension using an attention mechanism to form a comprehensive pollution level conclusion.

[0016] Furthermore, multimodal data includes text modality, image modality, and time-series modality;

[0017] The modality feature extractor is used to extract initial features from the data in each modality; for the text modality, sentence vectors are extracted using a pre-trained Chinese language model; for the image modality, frame-level features are extracted using ResNet; and for the temporal modality, window representation is extracted using a sliding window combined with Transformer.

[0018] Furthermore, in the self-supervised pre-training model of step S101, a distribution drift detection mechanism is introduced, setting the time... The feature distribution of the time-time model is The new sample distribution is ,when When this occurs, the incremental update mechanism is triggered. Indicates the drift threshold. The KL divergence is used to measure the difference between two probability distributions. In the incremental learning update mechanism, a lightweight parameter adaptation strategy is employed to adjust the existing model parameters. Perform a partial update.

[0019] Furthermore, in step S102, the step of constructing the Laplace matrix of the pollution monitoring nodes using geographical topological relationships includes:

[0020] A spatial topology map is constructed based on pollution monitoring points and their geographical adjacency.

[0021] The node topological embedding is obtained by performing spectral decomposition on the Laplacian matrix.

[0022] Furthermore, in step S102, the time period encoding module introduces a multi-scale time period for any timestamp t. Periodic sine and cosine coding is used to obtain the periodic sensing node embedding, which is represented as:

[0023]

[0024] in, It has multiple time periods;

[0025] The joint representation of a pollution event in terms of spatial distribution and temporal evolution is represented by a spatiotemporal joint embedding, and the final spatiotemporal joint embedding vector is represented as follows: ,in, This represents a vector concatenation operation. To fuse the weight matrix, Represents topological embedding. This indicates a periodic embedding.

[0026] Furthermore, in step S103, a learnable bottleneck compression module is designed for mode m. First, channel attention gating is used to perform statistical pooling on the embedding vector to obtain the summary vector, represented as:

[0027]

[0028] in, This represents the global summary vector for mode m. This represents the number of embedding vectors for modality m, and j is an index variable used to iterate through all embedding vectors of modality m. This represents the j-th embedding vector of mode m;

[0029] The dimension-gated vector is calculated using a small MLP and is represented as follows:

[0030]

[0031] in, Represents a dimension-gated vector. Indicates the embedding dimension. Indicates element-wise Sigmoid activation. , This represents the parameters of the gated network; where the closer the vector element is to 1, the more important the k-th dimension is.

[0032] The training objective is to compress the reconstruction error and information loss. , represented as:

[0033]

[0034] in, The features reconstructed by the decoder For embedding information With gate information mutual information, To weigh the parameters;

[0035] An aggregation module is constructed, in which, for each modality m, semantic aggregation is performed using parameterized attention to obtain the summary vector of that modality, represented as:

[0036]

[0037]

[0038] in, It is the i-th sub-feature within the mode; Attention weights; The aggregated modal summary vector representation; Is with Relevant learnable parameters, It is the result of performing an exponential operation on the parameter; The learnable parameters corresponding to the j-th sub-feature The result after performing an exponential operation.

[0039] Furthermore, in step S104, the modality summary vectors are concatenated and mapped to a global context, represented as follows: ,in, This represents a vector concatenation operation. Represents a text modal summary vector; Represents an image modality summary vector; Represents a spatiotemporal modal summary vector; This is the concatenated vector after multimodal concatenation;

[0040] Through the fully connected layer and nonlinear functions The posterior probability distribution for predicting pollution event types is expressed as:

[0041]

[0042] Where y represents the set of event type labels, For bias terms, Let K represent the set of real numbers, and let K represent the number of categories of pollution event types. This represents the dimension of a single modality summary vector.

[0043] Furthermore, in step S104, a prior weight vector that varies with type is constructed. Corresponding to three modes, among which, For a learnable mapping function, the parameters Learned through training, y represents the set of pollution event type labels;

[0044] The prior rules are the modal importance corresponding to different types, and then the data-driven modal scores are calculated. , represented as:

[0045]

[0046] in, , For mode-specific linear transformation parameters, The aggregated modal summary vector representation;

[0047] By fusing prior and data signals, the final modal weights are obtained, expressed as:

[0048]

[0049] in, These are the prior strength coefficients, used to balance prior constraints and adaptive learning based on data. Prior weight vector The components corresponding to mode m, It is an index variable that iterates through all modalities; when the type of contamination event is uncertain, the model adaptively reduces... To enhance data-driven approaches;

[0050] After obtaining the modal weights, the modal-level fusion representation is calculated, expressed as:

[0051]

[0052] in, This represents modal-level fusion characterization. , , These correspond to the modal weights for text modality, image modality, and spatiotemporal modality, respectively.

[0053] Meanwhile, to preserve the global context and enhance separability while preventing gradient vanishing, a residual correction term is introduced, expressed as:

[0054]

[0055] in, To introduce the final fusion characterization after residual correction, , For residual mapping parameters, It is a non-linear activation function.

[0056] Furthermore, in step S104, a regularization term is introduced, represented as:

[0057]

[0058] in, The loss function is the regularization term. This represents the final weight of the m-th mode. This represents a vector consisting of the weights of all modes. This indicates the calculation of the L1 norm. Denotes the square of the L2 norm. , These are the sparse constraint coefficient and the prior consistency weight, respectively;

[0059] The joint optimization objective in the gating fusion phase is expressed as:

[0060]

[0061] in, , All are balance coefficients greater than 0. For type classification loss, The loss is the decision loss for the hierarchical dimension.

[0062] Furthermore, in step S105, for pollution event classification, the fused representation vectors are... The input is fed into the classifier, and the output is the posterior distribution of the types of contamination events, represented as follows:

[0063]

[0064]

[0065] in, This represents the posterior probability distribution prediction result for the type of pollution event. , The weight matrix and bias terms of the classifier, and the set of event types. Categories include illegal discharge of pollutants, chemical leaks, pollution caused by traffic accidents, and secondary pollution caused by safety accidents;

[0066] Cross-entropy loss is used during training, expressed as:

[0067]

[0068] in, For real labels, To use cross-entropy loss; Indicates the predicted label;

[0069] After the classification results are determined, the event is evaluated in a multi-dimensional manner.

[0070] Compared with existing technologies, the beneficial effects of the pollution event assessment method based on multimodal incremental learning and dynamic optimization in this application are:

[0071] The domain-adaptive gating fusion mechanism proposed in this invention can dynamically adjust the weights of each modality according to the characteristics of different contamination events. Through this mechanism, the importance of each modality can be automatically identified in different types of contamination events, and intelligent optimization can be performed during the fusion process, thereby significantly improving the identification accuracy.

[0072] This invention combines the Graph Laplace spectroscopy method and multi-scale periodic coding to effectively combine spatial and temporal factors, forming a unified spatiotemporal modeling framework. Through the Graph Laplace spectroscopy method, this invention can capture the regularity of pollutant diffusion in space, while multi-scale periodic coding can accurately reflect the temporal variation characteristics of pollutants, especially their diurnal, weekly, and seasonal fluctuations.

[0073] This invention introduces a sliding window drift detection mechanism, which can detect changes in data distribution in real time and perform targeted incremental updates when concept drift is detected, without retraining the entire dataset. Combined with lightweight incremental update technology, this invention can efficiently update new knowledge while maintaining the stability of the original model parameters, avoiding catastrophic forgetting problems. This allows the invention to adapt quickly to new pollution events or sources, ensuring recognition accuracy and system stability.

[0074] This invention significantly improves the interpretability of the model by introducing a multi-dimensional hierarchical evaluation and attention-weighted comprehensive ranking mechanism. After event identification, this invention can provide a detailed explanation of each evaluation dimension and demonstrate the importance of each dimension through visualization. This not only enhances the transparency of the model's decision-making process but also enables environmental monitoring personnel to more clearly understand the basis of the model's judgment, thereby improving the trustworthiness and operability of the decision. Attached Figure Description

[0075] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0076] In the attached diagram:

[0077] Figure 1 This is a system architecture diagram of the pollution event assessment method based on multimodal incremental learning and dynamic optimization of the present invention;

[0078] Figure 2 This is a flowchart illustrating the implementation of the pollution event assessment method based on multimodal incremental learning and dynamic optimization of this invention.

[0079] Figure 3 A structural block diagram of a computer device provided by the present invention. Detailed Implementation

[0080] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0081] The technical solutions in the exemplary embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described exemplary embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0082] like Figure 1 and Figure 2 As shown, this embodiment of the invention provides a pollution event assessment method based on multimodal incremental learning and dynamic optimization, which includes the following steps:

[0083] Step S101: Collect multimodal data from the environmental pollution detection platform, and process the multimodal data using a self-supervised pre-trained model with an incremental learning mechanism to obtain deep embedding features at the semantic level of each modality.

[0084] In this embodiment, the multimodal data includes text modality, image modality, and time-series modality; wherein, the text modality includes pollution accident reports, monitoring logs, and related public opinion texts; the image modality includes remote sensing images of the polluted area and video frames of pollution source monitoring; the time-series modality includes pollutant concentration monitoring curves, geographical coordinates of pollution events, and topographic and meteorological information affecting diffusion; the multimodal data is represented as follows: ,in, These represent text, image, and time-series datasets, respectively.

[0085] All input data are standardized and normalized, and are represented as follows:

[0086]

[0087] in, Represents the original eigenvalues ​​of mode m. , Modal The mean and standard deviation; the mean and standard deviation can be estimated from historical data batches or updated online using a sliding window.

[0088] Furthermore, a modality feature extractor is used to extract initial features from the data within each modality. The modality feature extractor is defined as a set of mapping functions, expressed as: ,in, The neural network or feature extraction function representing mode m. For the parameters of the mapping function, It is the set of initial embedding representation matrices / vectors for mode m. The original dataset representing mode m;

[0089] In this embodiment, the text modality uses a pre-trained Chinese language model (such as BERT, ChineseBERT, or RoBERTa) to extract sentence vectors. If the input is a collection of short texts... ,but ,in, For the sample size, The vector representation of the j-th text; for image / video modalities, ResNet or ViT is used to extract frame-level features, and video can also be represented by temporal pooling at the frame level to obtain video segment representations; for temporal modalities, a sliding window is used. Where L represents the window length, This represents the timing data in the u-th sliding window; It is the reading at the start of the window; It is the end time reading of the window, and then the window representation is extracted using TCN, 1D-Conv, or Transformer. ;

[0090] Furthermore, in the self-supervised pre-training model of this invention embodiment, a distribution drift detection mechanism is introduced to address the continuous emergence of new pollution sources in a dynamic environment, setting a time... The feature distribution of the time-time model is The new sample distribution is ,when When a new pollution scenario is identified, the incremental update mechanism is triggered. The drift threshold, The KL divergence is used to measure the difference between two probability distributions. In the incremental learning update mechanism, a lightweight parameter adaptation strategy is employed to adjust the existing model parameters. A partial update is represented as:

[0091]

[0092] in, For learning rate, This represents the optimal increment in the gradient direction of the new sample.

[0093] This embodiment uses this mechanism to achieve continuous evolution and adaptive updating of the self-supervised pre-trained model without retraining the entire dataset.

[0094] In step S101 of this embodiment of the invention, a self-supervised pre-trained model (such as BERT, ResNet, TCN, etc.) is used within each modality to extract initial high-quality representations and obtain deep embedding features at the semantic level. To address the problem of insufficient adaptability of existing models in novel pollution scenarios, this step introduces a multimodal incremental learning mechanism, which enables continuous evolution of the model through parameter transfer and dynamic weight updates without retraining the full set of samples.

[0095] The multimodal incremental learning mechanism provided in this embodiment uses a sliding time window strategy to detect the distribution drift of new event samples and updates the model representation space through a lightweight adaptation layer (such as a LoRA module or an incremental parameter pool), thereby ensuring that the model can absorb the feature information of new pollution source types and rare pollution scenarios over time, significantly improving the long-term adaptability and generalization ability of the system.

[0096] Please continue to refer to Figure 1 and Figure 2 The pollution event assessment method based on multimodal incremental learning and dynamic optimization provided in this embodiment also includes the following steps:

[0097] Step S102: Construct the Laplace matrix of pollution monitoring nodes using geographic topological relationships, and introduce a time period coding module to decompose time information into daily, weekly and seasonal components. Through the fusion mapping of topological embedding and periodic embedding, obtain the joint representation of pollution events in spatial layout and temporal evolution.

[0098] Step S102 aims to simultaneously capture the spatial dependence and temporal evolution of pollution events, providing a unified spatiotemporal representation for subsequent multimodal feature alignment, and solving the technical problem that traditional methods only focus on spatial proximity relationships and cannot effectively model the temporal dynamic characteristics of pollution propagation.

[0099] In one implementation of the present invention, the step of constructing the Laplace matrix of pollution monitoring nodes using geographical topological relationships includes:

[0100] Construct a spatial topology map based on pollution monitoring points and their geographical adjacency. , represented as: Where V is the set of nodes and E is the set of edges. It is an adjacency matrix. Let N represent the set of real numbers, and N represent the total number of nodes.

[0101] Performing spectral decomposition on the Laplacian matrix L yields the node topological embedding, expressed as:

[0102]

[0103]

[0104]

[0105] In the formula, Input features for the nodes, For learnable parameter matrix, The Laplace matrix is ​​decomposed as follows: .

[0106] Furthermore, in the step of introducing the time period encoding module, in order to characterize the periodic patterns of pollution (such as day and night, week, season), a multi-scale time period is introduced for any timestamp t. Periodic sine and cosine coding is used to obtain the periodic sensing node embedding, which is represented as:

[0107]

[0108] in, For multi-scale time periods, such as 24 hours, 7 days, and 12 months;

[0109] The time period encoding in this embodiment can be directly incorporated into temporal features, enabling the model to have periodic awareness and time-dependent modeling capabilities.

[0110] Furthermore, in step S102 of this embodiment, the joint representation of the pollution event in terms of spatial layout and temporal evolution is a spatiotemporal joint embedding representation, and the final spatiotemporal joint embedding vector is represented as follows: ,in, This represents a vector concatenation operation. The weight matrix is ​​used for fusion; this spatiotemporal joint embedding representation serves as a unified spatiotemporal feature input, supporting subsequent intramodal compression and cross-modal fusion.

[0111] Therefore, this embodiment of the invention utilizes geographical topological relationships to construct the Laplace matrix of pollution monitoring nodes, realizing structured modeling between regional nodes and characterizing the spatial dependence of pollution diffusion. Secondly, a temporal periodic encoding module is introduced to decompose time information into daily, weekly, and seasonal components to capture the temporal regularity and periodic variation trend of pollution processes. Finally, through the fusion mapping of topological embedding and periodic embedding, the joint representation of pollution events in spatial layout and temporal evolution is realized, providing a stable spatiotemporal basis for subsequent modal feature fusion and dynamic assessment.

[0112] Please continue to refer to Figure 1 and Figure 2 To address the issues of high-dimensional feature redundancy, high noise, and high computational cost in multimodal data, the pollution event assessment method based on multimodal incremental learning and dynamic optimization provided in this embodiment further includes the following steps:

[0113] Step S103: Use a learnable compression module to compress the embedding vectors of each modality with information awareness and dynamically adjust the feature dimension to balance information preservation and computational efficiency. In the compression module, mutual information estimation and reconstruction error are optimized to output the intra-modal representation matrix. Semantic aggregation is driven by a parameterized attention mechanism to fuse the information of the corresponding modality into a concise intra-modal summary vector.

[0114] Specifically, step S103 aims to reduce redundant information and high-dimensional feature burden in multimodal data, achieve information compression and semantic aggregation within the modality, effectively reduce the complexity of cross-modal alignment and fusion, and provide a high-quality, low-redundancy input foundation for subsequent dynamic gating fusion.

[0115] Specifically, a learnable bottleneck compression module is designed for mode m. ,in, Indicates the compression module;

[0116] First, through channel attention / dimensionality gating, the embedding vector is statistically pooled (e.g., average pooling) to obtain the summary vector, represented as:

[0117]

[0118] in, This represents the global summary vector for mode m. This represents the number of embedding vectors for modality m, and j is an index variable used to iterate through all embedding vectors of modality m. This represents the j-th embedding vector of mode m;

[0119] Then, the dimension-gated vector is calculated using a small MLP and represented as:

[0120]

[0121] in, Represents a dimension-gated vector. Indicates the embedding dimension. Indicates element-wise Sigmoid activation. , This represents the parameters of the gated network; where the closer the vector element is to 1, the more important the k-th dimension is.

[0122] Furthermore, the training objective is compressed, where the training objective includes minimizing the reconstruction error and information loss, expressed as:

[0123]

[0124] in, The features reconstructed by the decoder For embedding information With gate information mutual information, To weigh the parameters;

[0125] This mechanism can suppress redundant dimensions while preserving key information, achieving efficient intramodal compression and semantic summarization.

[0126] Construct an aggregation module, where the aggregation module performs an aggregation on each modality. Let t represent the text modality, i represent the image modality, and s represent the spatiotemporal modality. Semantic aggregation is performed using parameterized attention to obtain the summary vector for that modality, denoted as:

[0127]

[0128]

[0129] in, It is the i-th sub-feature within the mode; Attention weights; This is the vector representation of the aggregated modal summaries. Is with Relevant learnable parameters, For parameters The result after performing exponentiation; The learnable parameters corresponding to the j-th sub-feature The result after performing an exponential operation.

[0130] Step S104: Introduce a domain-adaptive gating fusion network, combining environmental domain knowledge with a data-driven attention mechanism to dynamically adjust the contribution of different modal features, thereby obtaining fused features;

[0131] Step S104 in this embodiment can achieve adaptive weighted fusion of multimodal features and introduce type-aware domain priors to improve model interpretability and robustness.

[0132] Specifically, in this embodiment, the fusion network adaptively allocates weights for text, images, time series, and geographic information based on the type and contextual features of the pollution event. For example, in a chemical spill event, the influence of image and sensor features is automatically enhanced, while in an illegal discharge event, the weights of text and time series features are increased. This embodiment achieves flexibility and specificity in feature fusion through a gating mechanism, ensuring that the model can make optimal judgments in different pollution scenarios.

[0133] Specifically, in step S104 of this embodiment, the modality summary vectors are first concatenated and mapped to a global context, as follows: ,in, This represents a vector concatenation operation; Represents a text modal summary vector; Represents an image modality summary vector; Represents a spatiotemporal modal summary vector; This is the concatenated vector after multimodal concatenation;

[0134] Through the fully connected layer and nonlinear functions The posterior probability distribution for predicting pollution event types is expressed as:

[0135]

[0136] Where y represents the set of event type labels, For bias terms, Let K represent the set of real numbers, and let K represent the number of categories of pollution event types. The dimension representing a single modality summary vector;

[0137] To introduce domain-specific prior rules, such as highlighting the visual modality for chemical leaks and enhancing the text / sensor modality for illegal discharges, a type-dependent prior weight vector is constructed. Corresponding to three modes, among which, For a learnable mapping function, the parameters Learned through training, y represents the set of pollution event type labels;

[0138] Prior rules can be represented by the modal importance corresponding to different types, followed by the calculation of data-driven modal scores. , represented as:

[0139]

[0140] in , For mode-specific linear transformation parameters, This is the vector representation of the aggregated modal summaries. ;

[0141] By further integrating the prior and data signals, the final modal weights are obtained, expressed as:

[0142]

[0143] in, These are the prior strength coefficients, used to balance prior constraints and adaptive learning based on data. Prior weight vector The components corresponding to mode m, It is an index variable that iterates through all modalities; when the type of contamination event is uncertain, the model adaptively reduces... To enhance data-driven approaches;

[0144] Furthermore, in this embodiment, after obtaining the modal weights, the modal-level fusion characterization is calculated, expressed as:

[0145]

[0146] in, This represents modal-level fusion characterization. , , These correspond to the modal weights for text modality, image modality, and spatiotemporal modality, respectively.

[0147] Meanwhile, to preserve the global context and enhance separability while preventing gradient vanishing, a residual correction term is introduced, expressed as:

[0148]

[0149] in, To introduce the final fusion characterization after residual correction, , For residual mapping parameters, It is a non-linear activation function;

[0150] Furthermore, to avoid weight collapse to a single modality and improve interpretability, embodiments of the present invention introduce the following regularization term, expressed as:

[0151]

[0152] in, The loss function is the regularization term. This represents the final weight of the m-th mode. This represents a vector consisting of the weights of all modes. This indicates the calculation of the L1 norm. Denotes the square of the L2 norm. , These are the sparse constraint coefficient and the prior consistency weight, respectively;

[0153] This encourages models to learn sparse but non-extreme weight distributions and remain consistent when the priors are reasonable, and to have the ability to self-correct when new features are introduced into the data;

[0154] Furthermore, let the type classification loss be... The loss is determined by the following hierarchical dimensions (economic loss, social impact, ecological pollution, etc.). Then the joint optimization objective of the gating fusion stage is expressed as:

[0155]

[0156] in, , All are balance coefficients greater than 0. For type classification loss, The loss is the decision loss for the hierarchical dimension.

[0157] Step S105: Classify pollution events using fusion features. Based on the classification results, classify the same event into different levels from multiple dimensions and output independent level results. Adaptively weight the level results of each dimension using an attention mechanism to form a comprehensive pollution level conclusion.

[0158] Step S105 of this application embodiment constructs a multi-level evaluation system after completing the fusion characterization. This ensures accurate classification while comprehensively reflecting the real impact of pollution events from different loss perspectives, and significantly improves the scientificity and consistency of the final evaluation.

[0159] Specifically, this embodiment is based on fusion representation. This approach enables the identification of pollution incident types, multi-dimensional classification, and final comprehensive level assessment. Specifically, for pollution incident classification, this embodiment will integrate representation vectors. The input is fed into the classifier, and the output is the posterior distribution of the types of contamination events, represented as follows:

[0160]

[0161]

[0162] in, This represents the posterior probability distribution prediction result for the type of pollution event. , The weight matrix and bias terms of the classifier, and the set of event types. Categories include illegal discharge of pollutants, chemical leaks, pollution caused by traffic accidents, and secondary pollution caused by safety accidents;

[0163] Furthermore, in this embodiment, cross-entropy loss is used during training, expressed as:

[0164]

[0165] in, For real labels, To use cross-entropy loss, Let C represent the predicted label, C represent the set of event types, and c represent the event type.

[0166] After the classification results are determined, the event is further evaluated using a multi-dimensional grading system. Based on current standards, three core grading dimensions are defined:

[0167] ① Economic loss dimension (such as direct property loss, losses due to work stoppage and production stoppage, and costs of remediation and restoration);

[0168] ② Social impact dimension (such as casualties, harm to residents' health, and impact on social stability);

[0169] ③ Ecological pollution dimension (such as the scope of water pollution, atmospheric diffusion intensity, soil pollution level, and biodiversity destruction).

[0170] For each dimension Define the set of levels These correspond to a classification from "extremely severe" to "mild"; this embodiment utilizes fusion characterization. Embedded with classification results Construct a level discrimination function:

[0171]

[0172]

[0173] in, This represents the predicted probability distribution of the k-th dimension. Let represent the learnable weight matrix of the k-th dimension; The learnable bias vector of the k-th dimension; The predicted probability of level g in the k-th dimension; arg max represents the index corresponding to the maximum value.

[0174] Each dimension is trained independently, and its loss function is:

[0175]

[0176] in, For the true level label of this dimension, For weights.

[0177] This embodiment ensures that each dimension follows the existing quantitative standards: economic loss is based on monetary value, social impact on the number of casualties, and ecological pollution on pollutant concentration or affected area, thus making the classification results objective and comparable.

[0178] In obtaining multi-dimensional hierarchical results Subsequently, this invention uses an attention mechanism to weight and fuse the various dimensions to generate the final comprehensive pollution level.

[0179] Specifically as follows:

[0180] First, the hierarchical prediction distribution for each dimension is... Projection is a dimension embedding vector:

[0181]

[0182] in, This represents the embedding vector in the k-th dimension; This represents the projection weight matrix for the k-th dimension; The bias vector of the k-th dimension;

[0183] Then, the attention weights are calculated:

[0184]

[0185] in, This represents the attention weight of the k-th dimension; Indicates attention projection weights, Indicates the context projection weights. represents the transpose of the attention vector; e represents the global context vector; tanh is the hyperbolic tangent activation function; exp is the exponential function;

[0186] Finally, the comprehensive grade prediction vector is:

[0187]

[0188] as a result As the final comprehensive level of a pollution incident; This represents the learnable weight matrix of the final projection layer; This represents the bias vector of the final projection layer; softmax is the normalization function.

[0189] The entire system adopts a joint optimization objective:

[0190]

[0191] in, Indicates the total loss; Indicates classification loss; Indicates graded loss; , , Represents the weighting coefficients of each loss term; The design incorporates a consistency regularization term between the overall rating and authoritative classification documents to ensure that the fusion strategy learned by the model conforms to industry standards and requirements.

[0192] like Figure 3 As shown, in this embodiment of the invention, a computer device is provided. At the hardware level, the terminal includes a processor, and optionally also includes an internal bus, a network interface, and a memory.

[0193] The memory may include main memory, such as high-speed random access memory, or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for its operations.

[0194] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA bus, PCI bus, or EISA bus, etc.

[0195] The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0196] The memory is used to store programs. Specifically, the program may include program code, which includes computer operation instructions.

[0197] Memory can include main memory and non-volatile memory, and provides instructions and data to the processor.

[0198] The processor executes the program stored in the memory, and specifically executes the pollution event assessment method based on multimodal incremental learning and dynamic optimization provided in the above embodiments.

[0199] The above is as stated in this application. Figure 1 The pollution event assessment method based on multimodal incremental learning and dynamic optimization disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor.

[0200] The processor may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed through integrated logic circuits in the processor's hardware or through software instructions. The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor, etc.; it can also be a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor.

[0201] The steps of the method disclosed in the embodiments of this application can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor.

[0202] The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0203] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform... Figure 1 The embodiment shown is a pollution event assessment method based on multimodal incremental learning and dynamic optimization.

[0204] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0205] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

Claims

1. A pollution event assessment method based on multimodal incremental learning and dynamic optimization, characterized in that, Includes the following steps: Step S101: Collect multimodal data from the environmental pollution monitoring platform, and process the multimodal data using a self-supervised pre-trained model with an incremental learning mechanism to obtain deep embedding features at the semantic level of each modality; wherein, the multimodal data includes text modality, image modality, and time-series modality; in the self-supervised pre-trained model: a distribution drift detection mechanism is introduced, and a time-series detection mechanism is set... The feature distribution of the time-time model is The new sample distribution is ,when When this occurs, the incremental update mechanism is triggered; Indicates the drift threshold. The KL divergence is used to measure the difference between two probability distributions. In the incremental learning update mechanism, a lightweight parameter adaptation strategy is employed to adjust the existing model parameters. Perform a partial update; Step S102: Construct the Laplace matrix of pollution monitoring nodes using geographic topological relationships, and introduce a time period coding module to decompose time information into daily, weekly and seasonal components. Through the fusion mapping of topological embedding and periodic embedding, obtain the joint representation of pollution events in spatial layout and temporal evolution. Step S103: Use a learnable compression module to compress the embedding vectors of each modality with information awareness and dynamically adjust the feature dimension to balance information preservation and computational efficiency. In the compression module, mutual information estimation and reconstruction error are optimized to output the intra-modal representation matrix. Semantic aggregation is driven by a parameterized attention mechanism to fuse the information of the corresponding modality into a concise intra-modal summary vector. Step S104: Introduce a domain-adaptive gating fusion network, combining environmental domain knowledge with a data-driven attention mechanism to dynamically adjust the contribution of different modal features, thereby obtaining fused features; Step S105: Classify pollution events using fusion features. Based on the classification results, classify the same event into different levels from multiple dimensions and output independent level results. Adaptively weight the level results of each dimension using an attention mechanism to form a comprehensive pollution level conclusion.

2. The method of claim 1, wherein, The modality feature extractor is used to extract initial features from the data in each modality; for the text modality, sentence vectors are extracted using a pre-trained Chinese language model; for the image modality, frame-level features are extracted using ResNet; and for the temporal modality, window representation is extracted using a sliding window combined with Transformer.

3. The method of claim 2, wherein, In step S102, the step of constructing the Laplace matrix of pollution monitoring nodes using geographical topological relationships includes: A spatial topology map is constructed based on pollution monitoring points and their geographical adjacency. The node topological embedding is obtained by performing spectral decomposition on the Laplacian matrix.

4. The method of claim 3, wherein, In step S102, in the time period encoding module, for any timestamp t, introduce multi-scale time period , using periodic cosine encoding, obtain periodic perception node embedding, denoted as: ； wherein, is a multiscale time period; The joint representation of a pollution event in terms of spatial distribution and temporal evolution is represented by a spatiotemporal joint embedding, and the final spatiotemporal joint embedding vector is represented as follows: ,in, This represents a vector concatenation operation. To fuse the weight matrix, Represents topological embedding. This indicates a periodic embedding.

5. The method of claim 4, wherein, In step S103, a learnable bottleneck compression module is designed for mode m. First, channel attention gating is used to perform statistical pooling on the embedding vector to obtain the summary vector, represented as: ； in, This represents the global summary vector for mode m. This represents the number of embedding vectors for modality m, and j is an index variable used to iterate through all embedding vectors of modality m. This represents the j-th embedding vector of mode m; The dimension-gated vector is calculated using a small MLP and is represented as follows: ； in, Represents a dimension-gated vector. Indicates the embedding dimension. Indicates element-wise Sigmoid activation. , Indicates the parameters of the gating network; Represents the global summary vector of mode m; compressing the training objective, the training objective comprising minimizing a reconstruction error and an information loss is expressed as: ； wherein, is a feature reconstructed by the decoder, is embedding information is gating information is mutual information, is a trade-off parameter; An aggregation module is constructed. For each modality m, the aggregation module performs semantic aggregation using parameterized attention to obtain the summary vector of that modality, represented as: ； ； in, It is the i-th sub-feature within the mode; Attention weights; The aggregated modal summary vector representation; Is with Relevant learnable parameters, For parameters The result after performing exponentiation; The learnable parameters corresponding to the j-th sub-feature The result after performing an exponential operation.

6. The method of claim 5, wherein, In step S104, the modality summary vectors are concatenated and mapped to a global context, as follows: ,in, This represents a vector concatenation operation; Represents a text modal summary vector; Represents an image modality summary vector; Represents a spatiotemporal modal summary vector; This is the concatenated vector after multimodal concatenation; through a fully connected layer with a non-linear function the posterior probability distribution of the type of pollution event, denoted as: ； Where y represents the set of event type labels, For bias terms, Let K represent the set of real numbers, and let K represent the number of categories of pollution event types. This represents the dimension of a single modality summary vector.

7. The pollution event assessment method based on multimodal incremental learning and dynamic optimization according to claim 6, characterized in that, In step S104, a prior weight vector that varies with type is constructed. Corresponding to three modes, among which, For a learnable mapping function, the parameters Learned through training, y represents the set of pollution event type labels; Prior rules are modality importance for different types, followed by computing data-driven modality scores , is represented as: ； wherein, , is a modal specific linear transformation parameter, is the aggregated modal summary vector representation; By fusing prior and data signals, the final modal weights are obtained, expressed as: ； in, These are the prior strength coefficients, used to balance prior constraints and adaptive learning based on data. Prior weight vector The components corresponding to mode m, It is an index variable that iterates through all modalities; when the type of contamination event is uncertain, the model adaptively reduces... To enhance data-driven approaches; After obtaining the modal weights, the modal-level fusion representation is calculated, expressed as: ； in, This represents modal-level fusion characterization. , , These correspond to the modal weights for text modality, image modality, and spatiotemporal modality, respectively. Meanwhile, to preserve the global context and enhance separability while preventing gradient vanishing, a residual correction term is introduced, expressed as: ； in, To introduce the final fusion characterization after residual correction, , For residual mapping parameters, It is a non-linear activation function.

8. The method of claim 7, wherein, In step S104, a regularization term is introduced, represented as: ； in, The loss function is the regularization term. This represents the final weight of the m-th mode. This represents a vector consisting of the weights of all modes. This indicates the calculation of the L1 norm. Denotes the square of the L2 norm. , These are the sparse constraint coefficient and the prior consistency weight, respectively; The joint optimization objective in the gating fusion phase is expressed as: ； wherein, , are both balance coefficients greater than 0, is a type classification loss, is a judgment loss of the grading dimension.

9. The pollution event assessment method based on multimodal incremental learning and dynamic optimization according to claim 8, characterized in that, In step S105, for the pollution event classification, the fused representation vector is input to the classifier, which outputs the type posterior distribution of the pollution event, denoted as: ； ； wherein, represents a posterior probability distribution prediction result of a pollution event type, , is a weight matrix and bias term of the classifier, and the event type set includes illegal sewage, chemical leakage, pollution caused by traffic accidents, secondary pollution caused by safety accidents; Cross-entropy loss is used during training, expressed as: ； wherein, is the true label, is adopted using cross-entropy loss, denotes the predicted label, C denotes a set of event types, and c denotes an event type; After the classification results are determined, the event is evaluated in a multi-dimensional manner.

Images