Pollution source monitoring abnormal data processing system

Abstract

The invention discloses a pollution source monitoring abnormal data processing system comprising a multi-dimensional data acquisition module used for acquiring multi-dimensional data; the data preprocessing module is used for preprocessing the collected multi-dimensional data; the multi-dimensional feature extraction module is used for extracting data features of the preprocessed multi-dimensional data; the feature fusion and anomaly recognition module is used for carrying out fusion and anomaly recognition on the extracted multi-dimensional features based on a bidirectional long-short-term memory network model of an improved attention mechanism; the self-adaptive updating and early warning module is used for realizing self-adaptive updating of model parameters and performing early warning on equipment fault type abnormal data; and the data reconstruction module is used for reconstructing the identified abnormal data. By adopting the system, the limitation that a traditional method only pays attention to numerical value features is broken through, the numerical value features, the equipment state features and the environment correlation features are fused, all-directional and multi-view-angle recognition of abnormal data is achieved, and the accuracy of abnormal recognition is fundamentally improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a pollution source monitoring anomaly data processing system. Background Technology

[0002] In pollution source monitoring, sensors are the core equipment for acquiring key data such as pollutant concentration and emission flow. The quality of the data collected directly determines the accuracy and reliability of the monitoring results, thus affecting the scientific nature of pollution source control decisions. However, existing pollution source monitoring systems generally face the technical challenge of interference from abnormal sensor data during actual operation. Specific problems are as follows: 1. Sensor malfunctions can easily lead to data distortion: Pollution source monitoring scenarios are typically complex environments with harsh conditions such as high temperature, high humidity, corrosive gases, and dust accumulation. Combined with factors such as sensor aging, zero-point drift, and circuit failures, the collected data is highly susceptible to anomalies. Such anomaly data does not reflect actual pollution source emissions. If used directly for monitoring and analysis, it can severely mislead the assessment of pollution source emissions. For example, false high-concentration data generated by sensor zero-point drift may be misjudged as excessive emissions, or sensor response delays may cause missed detections of genuine excessive emissions.

[0003] 2. Existing methods for handling abnormal data lack accuracy: Currently, most industry methods for handling anomalies in monitoring data employ a single-dimensional judgment approach. This includes setting a fixed threshold range, where data exceeding the threshold is considered abnormal and removed; or using simple statistical methods, such as the 3σ criterion, to identify anomalies based on the statistical distribution of the data. However, these methods have significant limitations: fixed thresholds cannot adapt to dynamic changes in different pollution source types and monitoring periods, easily leading to false positives or false negatives; statistical methods like the 3σ criterion only focus on the numerical distribution characteristics of the data, ignoring crucial information such as sensor operating status and environmental interference factors. This results in extremely poor identification of progressive anomalies caused by minor sensor malfunctions, failing to meet the demands of high-precision monitoring.

[0004] 3. Lack of dynamic adaptability in data processing: Most existing methods adopt static processing logic and have not established a correlation model between data characteristics and sensor status and environmental parameters. Summary of the Invention

[0005] The purpose of this invention is to provide a pollution source monitoring abnormal data processing system to solve the problems existing in the background art.

[0006] To achieve the above objectives, the present invention provides a pollution source monitoring anomaly data processing system, comprising: The multi-dimensional data acquisition module is used to simultaneously collect pollution source monitoring data, sensor operating status data, and environmental parameter data; The data preprocessing module is used to preprocess the collected multi-dimensional data; The multi-dimensional feature extraction module is used to extract data features from preprocessed multi-dimensional data; The feature fusion and anomaly detection module, based on an improved attention mechanism bidirectional long short-term memory network model, fuses and detects anomalies in the extracted multi-dimensional features. The adaptive update and early warning module is used to periodically adopt newly collected labeled data and fine-tune the bidirectional long short-term memory network model based on the attention mechanism to achieve adaptive updates of model parameters; when the same type of equipment failure-type abnormal data appears within 3 consecutive monitoring cycles, an early warning is issued. The data reconstruction module reconstructs the identified abnormal data and generates alternative data to ensure the continuity of the monitoring data sequence.

[0007] Preferably, the pollution source monitoring data collected in the multi-dimensional data acquisition module includes pollutant concentration and emission flow rate, sensor operating status data includes sensor operating voltage, current, response time and zero offset, and environmental parameter data includes temperature, humidity, air pressure and dust concentration of the monitoring scene.

[0008] Preferably, the preprocessing of the collected multi-dimensional data includes missing value imputation, data normalization, and noise filtering.

[0009] Preferably, the data features include numerical features, device status features, and environmental correlation features; Numerical features, including statistical and trend features of monitoring data, are used to reflect the numerical distribution and dynamic change patterns of the data. Among them, statistical features include mean, variance, extreme values, fluctuation frequency and quantile distribution, and trend features include first-order difference, second-order difference, slope change rate and sliding window fitting error. Equipment status characteristics, including deviation characteristics and stability characteristics of sensor operating parameters, are used to reflect the working status of the sensor. Deviation characteristics include the difference between the actual voltage and the rated voltage of the sensor, the ratio of the actual response time to the standard response time, and the cumulative amount of calibration deviation. Stability characteristics include voltage fluctuation coefficient, cumulative amount of zero-point offset, and parameter variation coefficient. Environmental correlation characteristics were analyzed using Pearson and Spearman correlation analysis to extract key environmental parameters with a correlation degree ≥ 0.6. The abrupt change characteristics of the environmental parameters were also extracted, including the magnitude of temperature abrupt change, the rate of humidity change, and the duration of abrupt change.

[0010] Preferably, the improved attention-based bidirectional long short-term memory network model includes: The input module combines numerical features, equipment status features, and environmental correlation features into a feature matrix, which serves as the model input. The dual-branch fusion module includes an environment-numerical branch, a device status branch, and a fusion layer. The environment-numerical branch learns the spatiotemporal correlation between numerical features and environment-related features through CNN and BiLSTM layers to obtain environment-numerical fusion features. The device status branch reduces the dimensionality of high-dimensional device status features to low-dimensionality through 2-3 fully connected layers and uses gated recurrent units to filter key status parameters with anomaly recognition contribution ≥0.3, obtaining device health features. The fusion layer weightedly fuses the environment-numerical fusion features and device health features to obtain a fusion feature vector. The attention module calculates global attention weights for the fused feature vectors through an attention mechanism to capture key features; The output module identifies abnormal data and outputs the abnormal data type.

[0011] Preferably, the fusion layer weighted fusion of environmental-numerical fusion features and device health features includes: The mutual information value of the output vectors of the computational environment - numerical branch and the device status branch. The formula is: ; In the formula, , These are the output parameters and output vector of the environment-numerical branch, respectively. , These are the output parameters and output vector of the device state branch, respectively. For joint probability distribution, , It represents a marginal probability distribution; Preset the initial values ​​for the fusion weights, and set the output vector weights of the environment-numerical branch to [values ​​to be specified]. The weights of the device status branch output vector are set to The fusion weights are dynamically adjusted based on the mutual information values. When the strong correlation is ≥0.7, the fusion weight will be adjusted to... ≥0.5、 ≥0.5; when 0.3≤I<0.7 indicates moderate correlation, maintain the initial weights or fine-tune them; when I<0.3 indicates weak correlation, adjust the weights of the environment-numerical branch output vector to... ≤0.3, increase the weight of the device status branch output vector to ≥0.7; The two branch vectors are then concatenated using a weighted average with adjusted weights, as shown in the formula: ; In the formula, The final fused feature vector is a one-dimensional feature vector. Output vector for the environment-numerical branch. This is the output vector for the device state branch.

[0012] Preferably, the attention module includes an adaptation unit, a weight calculation unit, a feature enhancement unit, and an output unit; The adaptation unit performs a fusion of feature vectors. Perform dimensional partitioning, dividing the one-dimensional feature vector The features are split into distinguishable sub-feature dimensions, and the sub-features are encoded with abnormal sensitivity. Discrete / continuous features are mapped to a unified semantic space through an embedding layer, while strongly correlated environmental features are labeled. The first d1 dimension is the environmental sub-feature dimension, the middle d2 dimension is the numerical sub-feature dimension, and the last d3 dimension is the device status sub-feature dimension. The weight calculation unit uses multi-head attention and gated attention to assign different weights to the split environmental sub-features, numerical sub-features and device status sub-features, and then normalizes the attention weights of the three types of sub-features after concatenation to ensure that the total weight is 1. The weighted loss function is: ; In the formula, , The penalty coefficient is... Attention weights for device state features. Weights for strongly correlated environmental features; The feature enhancement unit uses weighted averaging and residual connection to enhance the three types of sub-features. The output unit concatenates the enhanced three types of sub-features in their original order to generate an attention-enhanced fusion feature vector. Through LayerNorm Normalization is performed to obtain the output features. .

[0013] Preferably, the output module includes a feature projection unit, an abnormal data identification unit, an abnormal type identification unit, a comprehensive identification unit, and a correction unit; The feature projection unit will Linear projection maps to a low-dimensional feature space, yielding the features. ; The abnormal data identification unit identifies abnormal data, classifying data with an abnormality probability ≥ 0.85 as abnormal, using the following formula: ; In the formula, This is the probability branch weight matrix; For bias terms; It is the Sigmoid activation function. , representing the probability of an anomaly; The loss function is: ; In the formula, The labels are real, 0 = normal, 1 = abnormal; The anomaly type identification unit identifies anomaly data types, specifically including: The anomaly types are classified into three primary categories: equipment failure (0), environmental interference (1), and actual exceedance (2). Secondary categories are further set based on the primary categories, and the anomaly types are encoded and represented. Through linear layers Mapping to the type dimension, the formula is: ; In the formula, , Indicates a secondary category, , ; The secondary anomaly type is determined by using the output type probability distribution activated by Softmax. The secondary anomaly types are then mapped to the primary classification, using the following formula: ; ; ; In the formula, For the first The probability of each secondary category, As a primary category, , Indicates exponentiation; The loss function used is multi-class cross-entropy loss, and the formula is: ; In the formula, For type real labels; The integrated identification unit is based on anomaly probability. and type probability Calculate the overall confidence level ,when When, it indicates that the data is abnormal. At that time, retrieve the abnormal results of the data at the same position in the last 5 frames. If ≥3 frames are consistent, confirm; otherwise, mark as pending review. At that time, the results are retained and marked as low confidence, and used as training samples for model iteration, combining the confidence scores. The calculation formula is: ; In the formula, and These are the abnormal probabilities. and type probability The weights; The correction unit constructs a feature causal graph based on a Bayesian network and generates a causal mask. , The probability distribution of the anomaly type is corrected to determine the final anomaly type. The corrected formula is as follows: ; ; In the formula, To perform element-wise multiplication, the result is corrected and renormalized to ensure that the sum of probabilities is 1.

[0014] Preferably, the total loss function of the output module is: ; ; In the formula, Indicates the loss weight for the anomaly type. Indicates the confidence loss weight. Represents the confidence loss function. For type probability.

[0015] Preferably, the data reconstruction module uses a dual criterion of cosine similarity and Euclidean distance to screen similar sequences, selecting historical normal sequences with environmental parameters and equipment status deviations ≤10%; weights are redistributed according to sequence similarity to generate alternative data; consistency with the trend of previous and subsequent data ≥0.85 is considered qualified, and manual review is triggered when it is unqualified to avoid reconstruction distortion.

[0016] Therefore, the pollution source monitoring abnormal data processing system described above has the following beneficial effects: (1) Breaking through the limitations of traditional methods that only focus on numerical features, it integrates numerical features, equipment status features, and environmental correlation features to achieve comprehensive and multi-perspective identification of abnormal data, fundamentally improving the accuracy of anomaly identification; (2) Use a bidirectional long short-term memory network model with an improved attention mechanism to identify data, thereby improving the accuracy of data identification; (3) It can achieve adaptive adjustment of parameters through incremental learning to adapt to different monitoring scenarios; at the same time, it can link equipment fault early warning, combine data processing with equipment operation and maintenance, reduce the overall operation and maintenance cost of the monitoring system, and improve the intelligence level of the system.

[0017] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the abnormal data processing of pollution source monitoring according to an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can be arranged and designed in various different configurations, and therefore should not be construed as limiting the present invention.

[0020] Example Reference Figure 1 This invention provides a pollution source monitoring anomaly data processing system, comprising: The multi-dimensional data acquisition module is used to simultaneously collect pollution source monitoring data, sensor operating status data, and environmental parameter data. Pollution source monitoring data includes pollutant concentration and emission flow rate; sensor operating status data includes sensor operating voltage, current, response time, and zero-point offset; and environmental parameter data includes temperature, humidity, air pressure, and dust concentration in the monitoring environment.

[0021] The data preprocessing module is used to preprocess the collected multi-dimensional data. Preprocessing includes missing value imputation, data normalization, and noise filtering.

[0022] Missing value imputation: The interpolation method is adaptively selected according to the type of missing data: cubic spline interpolation is used for continuous missing data, and time series GRU prediction interpolation is used for random missing data. A data validity warning is triggered when the missing rate is ≥30%. Data normalization: An improved Min-Max normalization algorithm is used to map data of different dimensions to the [0,1] interval, while preserving the original distribution characteristics of the data (such as the relative positions of extreme points). Noise filtering: A scene-adaptive wavelet denoising algorithm is used.

[0023] The multi-dimensional feature extraction module is used to extract data features from the preprocessed multi-dimensional data. These data features include numerical features, equipment status features, and environmental correlation features. Numerical features, including statistical and trend features of monitoring data, are used to reflect the numerical distribution and dynamic change patterns of the data. Among them, statistical features include features such as mean, variance, extreme values, fluctuation frequency, and quantile distribution, while trend features include features such as first-order difference, second-order difference, slope change rate, and sliding window fitting error.

[0024] Equipment status characteristics, including deviation and stability characteristics of sensor operating parameters, are used to reflect the working status of the sensor. Deviation characteristics include the difference between the actual voltage and the rated voltage, the ratio of the actual response time to the standard response time, and the cumulative amount of calibration deviation. Stability characteristics include voltage fluctuation coefficient, cumulative zero-point offset, and parameter variation coefficient.

[0025] Environmental correlation characteristics were analyzed using Pearson and Spearman correlation analysis to extract key environmental parameters with a correlation degree ≥ 0.6. The abrupt change characteristics of the environmental parameters were also extracted, including the magnitude of temperature abrupt change, the rate of humidity change, and the duration of abrupt change.

[0026] The feature fusion and anomaly detection module, based on an improved attention mechanism bidirectional long short-term memory network model, fuses and detects anomalies in the extracted multi-dimensional features.

[0027] Improved attention-based bidirectional long short-term memory network models include: The input module combines numerical features, equipment status features, and environmental correlation features into a feature matrix, which serves as the model input.

[0028] The dual-branch fusion module includes an environment-numerical branch, a device status branch, and a fusion layer.

[0029] The environment-numerical branch concatenates environmental and numerical features into a two-dimensional feature matrix (rows = time steps, columns = feature dimensions) by time steps. It then uses the convolutional kernels of a convolutional neural network (CNN) to capture the spatial correlation features between environmental and numerical features within the same time step. The spatial correlation features output by the CNN are then input into a bidirectional long short-term memory network (Bi-LSTM). The forward LSTM learns the impact of historical environment-numerical features on the current moment, while the backward LSTM predicts future trends, capturing the temporal dependence of gradual environmental changes on slow numerical shifts. Finally, it outputs a joint vector of disturbance and numerical features with unified dimensions.

[0030] The device status branch uses 2-3 fully connected layers to reduce the dimensionality of high-dimensional device status features to low-dimensional features, including environmental and numerical features. A gated recurrent unit is used to filter key status parameters with an anomaly identification contribution of ≥0.3, and the output device health feature vector matches the dimension of the environment-numerical branch. The fusion layer weightedly fuses environmental-numerical fusion features and device health features to obtain a fused feature vector. Specifically, the fusion layer weightedly fuses environmental-numerical fusion features and device health features, including: The mutual information value of the output vectors of the computational environment - numerical branch and the device status branch. The formula is: ; In the formula, , These are the output parameters and output vector of the environment-numerical branch, respectively. , These are the output parameters and output vector of the device state branch, respectively. For joint probability distribution, , It represents a marginal probability distribution; Preset the initial values ​​for the fusion weights, and set the output vector weights of the environment-numerical branch to [values ​​to be specified]. The weights of the device status branch output vector are set to The fusion weights are dynamically adjusted based on the mutual information values. When the correlation is ≥0.7 (strong correlation, such as aging equipment components + high dust concentration → abnormal value), the fusion weight will be adjusted to... ≥0.5、 ≥0.5; when 0.3≤I<0.7 (moderate correlation), maintain the initial weights or fine-tune them. ±0.05, ±0.05); when I < 0.3 (weak correlation, such as no environmental anomalies but equipment malfunction), adjust the weights of the environment-numerical branch output vector to... ≤0.3, increase the weight of the device status branch output vector to ≥0.7.

[0031] The two branch vectors are then concatenated using a weighted average with adjusted weights, as shown in the formula: ; In the formula, The final fused feature vector is a one-dimensional feature vector. Output vector for the environment-numerical branch. This is the output vector for the device state branch.

[0032] The fused feature vector contains both the spatiotemporal correlation of "environment-value" and retains the key information of "device status", and the weights are dynamically adapted according to the feature correlation degree.

[0033] The attention module calculates global attention weights for the fused feature vectors using an attention mechanism to capture key features. The attention module includes an adaptation unit, a weight calculation unit, a feature enhancement unit, and an output unit.

[0034] The adaptation unit, for the fused feature vector Perform dimensional partitioning, dividing the one-dimensional feature vector The feature is broken down into distinguishable sub-feature dimensions, and the sub-features are encoded with exceptional sensitivity. An embedding layer maps discrete / continuous features to a unified semantic space, while strongly correlated environmental features are labeled. Specifically, the first d1 dimension (e.g., dimensions 1-64) represents the environmental sub-feature dimension, the middle d2 dimension (e.g., dimensions 65-96) represents the numerical sub-feature dimension, and the last d3 dimension (e.g., dimensions 97-128) represents the device status sub-feature dimension. d1+d2+d3 is the total dimension of the fused vector, which can be adjusted according to the actual scenario.

[0035] The weight calculation unit employs multi-head attention and gated attention to process the split environmental sub-features. Numerical sub-features and device status sub-features Different weights are assigned, and the attention weights of the three sub-features are concatenated and normalized to ensure that the sum of the weights is 1. Specifically, this includes: Based on the split environmental sub-features, numerical sub-features, and device status sub-features, construct attention query vector Q, attention key vector K, and value vector V.

[0036] Attention query vector Q from fusion Generated through linear transformation. ,in, This is a learnable weight matrix.

[0037] Attention key vector K and value vector V are generated for the three types of sub-features respectively, using the following formula: ; ; In the formula, Represents environmental sub-features. Represents numerical sub-features, This represents a device state sub-feature.

[0038] Self-attention and bias weights are used to strengthen the basic weights of device sub-state features: ; in: The dimension is the key vector to ensure gradient stability; The bias term (initial value set to 0.45, corresponding to the preset basic weights of device state features) can be optimized through training iterations; ∈R 1×d3 The attention weights for device state features should have an average output weight of ≥0.45.

[0039] Filter strongly correlated environmental sub-features and label environmental features. F eStrongly correlated dimensions (such as dust concentration and temperature abrupt changes) are used; gated attention is employed to assign dynamic high weights to strongly correlated dimensions and de-weight non-correlated dimensions. ; in, , For the gated weight matrix / bias, Use the Sigmoid activation function; For element-wise multiplication, the gated vector outputs a weight coefficient of ≥0.7 for strongly correlated dimensions and ≤0.3 for uncorrelated dimensions; the final weight of strongly correlated environmental sub-features must be ≥0.4 (which can be dynamically increased to 0.45 under high interference conditions).

[0040] Numerical sub-features are assigned using basic scaled dot product attention, with weights adaptively adjusted based on feature importance, and no preset bias. ; The weights of numerical sub-features need to be dynamically adjusted according to abnormal scenarios (e.g., the weight of voltage fluctuation numerical features is reduced to 0.15 under high interference conditions).

[0041] The attention weights of the three sub-features are concatenated and then normalized to ensure that the sum of the weights is 1. The formula is as follows: .

[0042] The weighted loss function is: ; In the formula, , This is the penalty coefficient (all set to 0.5). Attention weights for device state features. To assign weights to strongly correlated environmental features, ensure that the weight of equipment status features is ≥0.45 and the weight of strongly correlated environmental features is ≥0.4.

[0043] The feature enhancement unit employs weighted averaging and residual connections to enhance the three types of sub-features. Specifically, it includes: Weighted summation is applied to the three types of sub-features, respectively, to generate enhanced sub-features. The formula is as follows: ; Residual connections: To preserve information from the original fused features, residual connections are introduced. ; Residual connections can effectively alleviate the vanishing gradient problem in deep networks and ensure training stability.

[0044] The output unit concatenates the three enhanced sub-features in their original order to generate an attention-enhanced fusion feature vector. Through LayerNorm Normalization is performed to obtain the output features. The formula is: ; .

[0045] The output module identifies abnormal data and outputs the abnormal data type.

[0046] The output module includes a feature projection unit, an anomaly data recognition unit, an anomaly type recognition unit, a comprehensive recognition unit, and a correction unit.

[0047] Feature projection unit, which projects high-dimensional feature vectors Linear projection maps to a low-dimensional feature space, yielding the features. The formula is: ; In the formula, , for Dimensions, such as 192; The projection dimension is set to 64 / 128. This is a bias term; LayerNorm ensures stable feature distribution and avoids gradient explosion.

[0048] The abnormal data identification unit identifies abnormal data and classifies it into normal / abnormal categories. Data with an abnormality probability ≥ 0.85 is classified as abnormal. The calculation formula is as follows: ; In the formula, This is the probability branch weight matrix; For bias terms; It is the Sigmoid activation function. , representing the probability of an anomaly; The loss function used is the binary cross-entropy loss function, the formula of which is: ; In the formula, The labels are real, 0 = normal, 1 = abnormal; The exception type identification unit identifies exception data types, specifically including: The anomaly types are classified into three primary categories: equipment failure (0), environmental interference (1), and actual exceedance (2). Secondary categories are further set based on the primary categories, as shown in Table 1. The anomaly types are coded and represented.

[0049] Table 1 Examples of exception type encoding

[0050] Through linear layers Mapping to the type dimension, the formula is: ; In the formula, , Indicates a secondary category, This represents 11 secondary categories. ; The output type probability distribution is activated using Softmax, and the secondary anomaly type is determined based on the output type probability. The secondary anomaly types are then mapped to the primary classification, using the following formula: ; ; ; In the formula, For the first The probability of each secondary category, As a primary category, , This indicates exponentiation.

[0051] The loss function uses multi-class cross-entropy loss, taking into account both primary and secondary classification supervision. The formula is as follows: ; In the formula, This is a true label for the type.

[0052] Integrated identification unit, based on anomaly probability and type probability Calculate the overall confidence level Overall confidence level The calculation formula is: ; In the formula, and These are the abnormal probabilities. and type probability The weight.

[0053] Based on comprehensive confidence level A three-level judgment is performed, and the judgment rules are shown in Table 2.

[0054] Table 2 Judgment Rules

[0055] The correction unit constructs a feature causal graph based on a Bayesian network and generates a causal mask. , Each dimension value represents the "causal confidence level" (0~1) of the corresponding type. If there is a direct causal relationship between the feature and the type (e.g., "temperature sensor not calibrated" → "zero drift"), If it is a false association (such as "irrelevant environmental noise" → "abnormal ammonia nitrogen content"), (Suppressing weight).

[0056] Based on the generated causal mask and the corrected anomaly type probability distribution, the final anomaly type correction formula is determined as follows: ; ; In the formula, To perform element-wise multiplication, the result is corrected and renormalized to ensure that the sum of probabilities is 1.

[0057] Specifically, the total loss function of the output module is: ; ; In the formula, Indicates the loss weight for the anomaly type. , Indicates the confidence loss weight. , Represents the confidence loss function. For type probability.

[0058] The model is trained in an end-to-end manner, with the parameters of the output module, attention module, and dual-branch fusion module being jointly and iteratively optimized, and AdamW is selected as the optimizer.

[0059] The adaptive update and early warning module is used to periodically adopt newly collected labeled data and fine-tune the bidirectional long short-term memory network model based on the attention mechanism to achieve adaptive updates of model parameters; when the same type of equipment failure-type abnormal data appears within 3 consecutive monitoring cycles, an early warning is issued.

[0060] Specifically, adaptive updates employ a federated learning framework, with localized training using data from multiple monitoring points. Only the model parameter update amount is uploaded, ensuring data security. When the new scenario data volume reaches 5%, fine-tuning is triggered, freezing the feature extraction layer and updating only the fully connected layer parameters, shortening the adaptation time to within one hour. A multi-level fault warning mechanism is adopted: Level 1 warning is triggered when the same device malfunctions for two consecutive cycles, pushing maintenance suggestions (including details of the malfunction characteristics); Level 2 warning is triggered when there are three consecutive cycles of malfunction, locking the sensor number and generating an operation and maintenance work order; warnings are prioritized based on the pollution source level and the severity of the malfunction, ensuring that key points are handled first.

[0061] The data reconstruction module reconstructs identified abnormal data, generating alternative data to ensure the continuity of the monitoring data sequence. Specifically, it uses a dual criterion of cosine similarity and Euclidean distance to screen similar sequences, selecting historical normal sequences with environmental parameters and equipment status deviations ≤10%; it then reassigns weights based on sequence similarity to generate alternative data; and a consistency of ≥0.85 with the trend of preceding and following data is considered acceptable. If the data is unacceptable, manual review is triggered to prevent reconstruction distortion.

[0062] In this embodiment, when the monitoring scene changes drastically (such as switching the monitoring object from industrial waste gas to domestic sewage), data distribution shift detection is performed. KL divergence is used to calculate the difference between the current data distribution and the training data distribution. When the KL divergence is ≥0.3, it is determined to be a distribution shift. The model feature extraction layer parameters are fine-tuned by using a small amount of new scene labeled data, and the decision layer parameters are frozen. Without losing historical knowledge, the model can quickly adapt to the new scene.

[0063] Meanwhile, the system of this invention dynamically adjusts the model thresholds based on the characteristics of time, space, and environment. It calculates "time-period baseline thresholds" (e.g., production periods vs. shutdown periods for industrial enterprises, daytime vs. nighttime) based on historical data. For example, the threshold for production periods is increased to 0.9, and for shutdown periods it is decreased to 0.8. It also presets "scene threshold offsets" for different monitoring scenarios. For instance, industrial waste gas monitoring has an offset of -0.03 due to large data fluctuations; agricultural non-point source pollution monitoring has an offset of -0.05 due to numerous environmental interferences; and ecological park monitoring has an offset of +0.02 due to high data stability. The thresholds are then fine-tuned based on current environmental parameters (e.g., temperature, humidity). The final threshold = time-period baseline threshold + scene threshold offset + environmental correction.

[0064] Example 1 For industrial waste gas pollution source monitoring scenarios, the data processing flow of the pollution source monitoring anomaly data processing system is as follows: 1. Multi-dimensional data acquisition: Sulfur dioxide and nitrogen oxide sensors are used to collect pollutant concentration data (sampling frequency of 1 time / minute) and emission flow rate; sensor operating status data includes operating voltage (rated voltage 24V), response time (standard response time ≤5s), and zero offset; environmental parameter data includes monitoring point temperature (-10℃~50℃), humidity (10%~90%RH), air pressure, and dust concentration.

[0065] 2. Data Preprocessing: For a single missing voltage value, linear interpolation of the first 5 minutes of the sequence was used to fill the gap; pollutant concentration (mg / m³) was... 3 Multidimensional data such as voltage (V) are normalized to the [0,1] interval; noise is filtered out of the concentration data by db4 wavelet to remove high-frequency noise caused by dust interference.

[0066] 3. Multi-dimensional feature extraction: Numerical characteristics: 10-minute mean of sulfur dioxide concentration (0.35), variance (0.02), first difference (0.05), extreme values ​​(0.82), quantile distribution (0.75 quantile 0.42); Equipment status characteristics: voltage deviation (24.2V-24V=0.2V), response time ratio (6s / 5s=1.2), voltage fluctuation coefficient (0.015), cumulative zero-point offset (0.03). Environmental correlation characteristics: Through Pearson + Spearman double correlation analysis, the correlation between temperature and concentration was 0.72 (≥0.6), and the amplitude of temperature change (5℃ / 10min) and the rate of humidity change (8% / 10min) were extracted.

[0067] 4. Feature Fusion and Anomaly Detection: A 3×12 feature matrix composed of three types of features is input into the improved attention mechanism BiLSTM model; the dual-branch fusion module calculates the mutual information value I=0.75 (strong correlation) and adjusts the weights. W 1 = 0.5 W 2=0.5, generating a fused feature vector; the attention module assigns a mean weight of 0.46 (≥0.45) to the device state feature through multi-head attention; the output module calculates the anomaly probability. P =0.92 (≥0.85), overall confidence level C =0.94 (0.85≤C<0.95), which is determined to be a device failure type (abnormal response time, secondary code 01).

[0068] 5. Adaptive Updates and Early Warnings: If the same equipment malfunctions for two consecutive monitoring cycles, a "clean sensor probe" maintenance suggestion will be pushed; if the malfunction continues for the third cycle, a level 2 early warning will be triggered, and a maintenance work order will be generated to lock the sensor number.

[0069] 6. Data Reconstruction: Using a dual criterion of cosine similarity and Euclidean distance, historical normal sequences with environmental parameters and equipment status deviations ≤10% were screened; alternative data (mean concentration 0.32) were generated based on sequence similarity weights, with a consistency of 0.88 (≥0.85) with the trend of the preceding and following data, and the reconstruction was qualified.

[0070] Example 2 For agricultural pollution source monitoring scenarios, the data processing flow of the pollution source monitoring anomaly data processing system is as follows: 1. Multi-dimensional data acquisition: Soil / irrigation water pollutant concentration (sampling frequency 1 time / 10 minutes) and discharge flow rate are collected using ammonia nitrogen, total phosphorus and pesticide residue sensors; sensor operating status data include working voltage (rated 12V), current, response time (standard ≤8s) and zero offset; environmental parameter data include soil temperature (5℃~35℃), air humidity (30%~95%RH), air pressure, light intensity and wind speed.

[0071] 2. Data preprocessing: Cubic spline interpolation was used to fill the two consecutive missing values ​​of total phosphorus concentration; pollutant concentration (mg / L), voltage (V) and other data were normalized to the [0,1] interval; the pesticide residue data were denoised by db6 wavelet and wind speed fluctuation interference was removed.

[0072] 3. Multi-dimensional feature extraction: Numerical characteristics: 30-minute mean of ammonia nitrogen concentration (0.28), variance (0.015), second difference (0.008), sliding window fitting error (0.02); Equipment status characteristics: voltage deviation (11.7V-12V=-0.3V), response time ratio (10s / 8s=1.25), cumulative calibration deviation (0.04), parameter coefficient of variation (0.025). Environmental correlation characteristics: soil temperature and ammonia nitrogen concentration were correlated at 0.75 (≥0.6), and the amplitude of temperature change (4℃ / 30min) and light intensity change (2000 lux / 30min) were also correlated.

[0073] 4. Feature Fusion and Anomaly Detection: After the feature matrix is ​​input into the model, the dual-branch fusion module calculates the mutual information value I=0.65 (moderate correlation), maintaining the initial weights. W 1 = 0.4 W 2=0.6; Attention module labels strongly correlated environmental features (soil temperature), with a mean weight of 0.42 (≥0.4); Output module anomaly probability. P =0.89 (≥0.85), overall confidence level C =0.91 (0.85≤C<0.95), retrieved data from the last 5 frames, 3 frames were consistent, confirming the anomaly type as equipment failure (voltage anomaly, secondary code 00).

[0074] 5. Adaptive Updates and Early Warnings: If the same anomaly is pushed for two consecutive cycles, a maintenance suggestion will be pushed, and an early warning will be triggered and a work order will be generated in the third cycle.

[0075] 6. Data Reconstruction: Screen similar historical sequences (voltage deviation -0.4V to -0.2V, soil temperature 20℃ to 23℃), generate alternative data (ammonia nitrogen mean 0.26), trend consistency 0.87, reconstruction qualified.

[0076] Example 3 For pollution source monitoring scenarios in ecological parks, the data processing flow of the pollution source monitoring anomaly data processing system is as follows: 1. Multi-dimensional data acquisition: COD (chemical oxygen demand), PM2.5, and VOCs sensors are used to collect pollutant concentrations (sampling frequency 1 time / 5 minutes) and emission flow rates; sensor operating status data includes operating voltage (rated 24V), current, response time (standard ≤6s), and zero offset; environmental parameter data includes air temperature (-5℃~40℃), humidity (40%~90%RH), air pressure, precipitation, and wind speed.

[0077] 2. Data preprocessing: The missing COD value after the rain was filled by interpolation of the historical sequence of the same rainfall level; the multidimensional data were normalized to the [0,1] interval; the VOCs data were denoised by db5 wavelet to remove instantaneous noise from the flow of people.

[0078] 3. Multi-dimensional feature extraction: Numerical characteristics: 20-minute mean PM2.5 concentration (0.32), variance (0.018), extreme values ​​(0.78), and fluctuation frequency (0.3 times / 20min); Equipment status characteristics: voltage deviation (24.3V-24V=0.3V), response time ratio (7.5s / 6s=1.25), cumulative zero-point offset (0.05), voltage fluctuation coefficient (0.02). Environmental correlation characteristics: The correlation between precipitation and COD concentration was 0.71 (≥0.6). The abrupt change amplitudes of precipitation (15 mm / 20 min) and wind speed (3 m / s / 20 min) were extracted.

[0079] 4. Feature Fusion and Anomaly Detection: The dual-branch fusion module calculates a mutual information value I=0.68 (moderate correlation) and fine-tunes the weights. W 1 = 0.42 W 2=0.58; mean weight of device state features in the attention module is 0.45 (meets the standard); output module anomaly probability P =0.93 (≥0.85), overall confidence level C =0.96 (≥0.95), judged as equipment failure type (zero point offset abnormal, secondary code 02).

[0080] 5. Adaptive Updates and Early Warnings: If the same anomaly occurs for three consecutive cycles, a Level 2 early warning will be triggered and an operation and maintenance work order will be generated.

[0081] 6. Data Reconstruction: Filter similar historical sequences (voltage deviation 0.2V~0.4V, precipitation 5mm~20mm), generate alternative data (PM2.5 mean 0.30), trend consistency 0.89, reconstruction qualified.

[0082] Therefore, the present invention adopts the above-mentioned pollution source monitoring abnormal data processing system, which breaks through the limitations of traditional methods that only focus on numerical features, integrates numerical features, equipment status features, and environmental correlation features, and realizes comprehensive and multi-perspective identification of abnormal data, fundamentally improving the accuracy of abnormal identification.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A pollution source monitoring anomaly data processing system, characterized in that, include: The multi-dimensional data acquisition module is used to simultaneously collect pollution source monitoring data, sensor operating status data, and environmental parameter data; The data preprocessing module is used to preprocess the collected multi-dimensional data; The multi-dimensional feature extraction module is used to extract data features from preprocessed multi-dimensional data; The feature fusion and anomaly detection module, based on an improved attention mechanism bidirectional long short-term memory network model, fuses and detects anomalies in the extracted multi-dimensional features. The adaptive update and early warning module is used to periodically adopt newly collected labeled data and fine-tune the bidirectional long short-term memory network model based on the attention mechanism to achieve adaptive updates of model parameters; when the same type of equipment failure-type abnormal data appears within 3 consecutive monitoring cycles, an early warning is issued. The data reconstruction module reconstructs the identified abnormal data and generates alternative data to ensure the continuity of the monitoring data sequence.

2. The pollution source monitoring anomaly data processing system according to claim 1, characterized in that: The pollution source monitoring data collected in the multi-dimensional data acquisition module includes pollutant concentration and emission flow rate; sensor operating status data includes sensor operating voltage, current, response time and zero offset; and environmental parameter data includes temperature, humidity, air pressure and dust concentration of the monitoring scene.

3. The pollution source monitoring anomaly data processing system according to claim 1, characterized in that: The preprocessing of the collected multi-dimensional data includes missing value imputation, data normalization, and noise filtering.

4. The pollution source monitoring anomaly data processing system according to claim 1, characterized in that: The data features include numerical features, device status features, and environmental correlation features; Numerical features, including statistical and trend features of monitoring data, are used to reflect the numerical distribution and dynamic change patterns of the data. Among them, statistical features include mean, variance, extreme values, fluctuation frequency and quantile distribution, and trend features include first-order difference, second-order difference, slope change rate and sliding window fitting error. Equipment status characteristics, including deviation characteristics and stability characteristics of sensor operating parameters, are used to reflect the working status of the sensor. Deviation characteristics include the difference between the actual voltage and the rated voltage of the sensor, the ratio of the actual response time to the standard response time, and the cumulative amount of calibration deviation. Stability characteristics include voltage fluctuation coefficient, cumulative amount of zero-point offset, and parameter variation coefficient. Environmental correlation characteristics were analyzed using Pearson and Spearman correlation analysis to extract key environmental parameters with a correlation degree ≥ 0.

6. The abrupt change characteristics of the environmental parameters were also extracted, including the magnitude of temperature abrupt change, the rate of humidity change, and the duration of abrupt change.

5. The pollution source monitoring anomaly data processing system according to claim 4, characterized in that: The improved attention-based bidirectional long short-term memory network model includes: The input module combines numerical features, equipment status features, and environmental correlation features into a feature matrix, which serves as the model input. The dual-branch fusion module includes an environment-numerical branch, a device status branch, and a fusion layer. The environment-numerical branch learns the spatiotemporal correlation between numerical features and environment-related features through CNN and BiLSTM layers to obtain environment-numerical fusion features. The device status branch reduces the dimensionality of high-dimensional device status features to low-dimensionality through 2-3 fully connected layers and uses gated recurrent units to filter key status parameters with anomaly recognition contribution ≥0.3, obtaining device health features. The fusion layer weightedly fuses the environment-numerical fusion features and device health features to obtain a fusion feature vector. The attention module calculates global attention weights for the fused feature vectors through an attention mechanism to capture key features; The output module identifies abnormal data and outputs the abnormal data type.

6. The pollution source monitoring anomaly data processing system according to claim 5, characterized in that, The fusion layer weightedly fuses environmental-numerical fusion features and device health features, including: The mutual information value of the output vectors of the computational environment - numerical branch and the device status branch. The formula is: ； In the formula, , These are the output parameters and output vector of the environment-numerical branch, respectively. , These are the output parameters and output vector of the device state branch, respectively. For joint probability distribution, , It represents a marginal probability distribution; Preset the initial values ​​for the fusion weights, and set the output vector weights of the environment-numerical branch to [values ​​to be specified]. The weights of the device status branch output vector are set to The fusion weights are dynamically adjusted based on the mutual information values. When the strong correlation is ≥0.7, the fusion weight will be adjusted to... ≥0.5、 ≥0.5; when 0.3≤I<0.7 indicates moderate correlation, maintain the initial weights or fine-tune them; when I<0.3 indicates weak correlation, adjust the weights of the environment-numerical branch output vector to... ≤0.3, increase the weight of the device status branch output vector to ≥0.7; The two branch vectors are then concatenated using a weighted average with adjusted weights, as shown in the formula: ； In the formula, The final fused feature vector is a one-dimensional feature vector. Output vector for the environment-numerical branch. This is the output vector for the device state branch.

7. The pollution source monitoring anomaly data processing system according to claim 6, characterized in that: The attention module includes an adaptation unit, a weight calculation unit, a feature enhancement unit, and an output unit; The adaptation unit performs a fusion feature vector analysis. Perform dimensional partitioning, dividing the one-dimensional feature vector The features are split into distinguishable sub-feature dimensions, and the sub-features are encoded with abnormal sensitivity. Discrete / continuous features are mapped to a unified semantic space through an embedding layer, while strongly correlated environmental features are labeled. The first d1 dimension is the environmental sub-feature dimension, the middle d2 dimension is the numerical sub-feature dimension, and the last d3 dimension is the device status sub-feature dimension. The weight calculation unit uses multi-head attention and gated attention to assign different weights to the split environmental sub-features, numerical sub-features and device status sub-features, and then normalizes the attention weights of the three types of sub-features after concatenation to ensure that the total weight is 1. The weighted loss function is: ； In the formula, , The penalty coefficient is... Attention weights for device state features. Weights for strongly correlated environmental features; The feature enhancement unit uses weighted averaging and residual connection to enhance the three types of sub-features. The output unit concatenates the enhanced three types of sub-features in their original order to generate an attention-enhanced fusion feature vector. Through LayerNorm Normalization is performed to obtain the output features. .

8. The pollution source monitoring anomaly data processing system according to claim 7, characterized in that: The output module includes a feature projection unit, an abnormal data identification unit, an abnormal type identification unit, a comprehensive identification unit, and a correction unit; The feature projection unit will Linear projection maps to a low-dimensional feature space, yielding the features. ; The abnormal data identification unit identifies abnormal data, classifying data with an abnormality probability ≥ 0.85 as abnormal, using the following formula: ； In the formula, This is the probability branch weight matrix; For bias terms; It is the Sigmoid activation function. , representing the probability of an anomaly; The loss function is: ； In the formula, The labels are real, 0 = normal, 1 = abnormal; The anomaly type identification unit identifies anomaly data types, specifically including: The anomaly types are set into three primary categories: equipment failure type 0, environmental interference type 1, and actual exceedance type 2. Based on the primary categories, secondary categories are further set up to encode the anomaly types. Through linear layers Mapping to the type dimension, the formula is: ； In the formula, , Indicates a secondary category, , ; The secondary anomaly type is determined by using the output type probability distribution activated by Softmax. The secondary anomaly types are then mapped to the primary classification, using the following formula: ； ； ； In the formula, For the first The probability of each secondary category, As a primary category, , Indicates exponentiation; The loss function used is multi-class cross-entropy loss, and the formula is: ； In the formula, For type real labels; The integrated identification unit is based on anomaly probability. and type probability Calculate the overall confidence level ,when When, it indicates that the data is abnormal. At that time, retrieve the abnormal results of the data at the same position in the last 5 frames. If ≥3 frames are consistent, confirm; otherwise, mark as pending review. At that time, the results are retained and marked as low confidence, and used as training samples for model iteration, combining the confidence scores. The calculation formula is: ； In the formula, and These are the abnormal probabilities. and type probability The weights; The correction unit constructs a feature causal graph based on a Bayesian network and generates a causal mask. , The probability distribution of the anomaly type is corrected to determine the final anomaly type. The corrected formula is as follows: ； ； In the formula, To perform element-wise multiplication, the result is corrected and renormalized to ensure that the sum of probabilities is 1.

9. A pollution source monitoring anomaly data processing system according to claim 8, characterized in that, The total loss function of the output module is: ； ； In the formula, Indicates the loss weight for the anomaly type. Indicates the confidence loss weight. Represents the confidence loss function. For type probability.

10. A pollution source monitoring anomaly data processing system according to claim 8, characterized in that: The data reconstruction module uses a dual criterion of cosine similarity and Euclidean distance to screen similar sequences, selecting historical normal sequences with environmental parameters and equipment status deviations ≤10%; and reassigns weights based on sequence similarity to generate alternative data. A consistency of ≥0.85 with the trend of previous and subsequent data is considered acceptable. If the consistency is not acceptable, manual review will be triggered to avoid distortion during reconstruction.

Images