Data processing method and device based on environmental monitoring, and storage medium

Abstract

The present application relates to the technical field of data processing, and discloses a data processing method and device based on environmental monitoring and a storage medium, comprising the following steps: step one, receiving various monitoring data and pre-processing the obtained monitoring data; step two, constructing various monitoring data time series according to the collection time of the pre-processed monitoring data; step three, dividing a plurality of monitoring time periods according to a monitoring period, and determining whether the monitored environment has monitoring abnormalities in each monitoring time period according to the monitoring data time series, and if there are monitoring abnormalities, then entering step four. By pre-processing the environmental monitoring data, constructing the monitoring data time series, dividing the monitoring time periods according to the monitoring period, and determining and identifying the sudden monitoring abnormalities according to the plurality of monitoring data and the adjacent time periods, the type of monitoring abnormalities can be distinguished, and it can be determined whether it is an external factor interference or an environmental abnormality.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to data processing methods, equipment, and storage media based on environmental monitoring. Background Technology

[0002] Environmental monitoring data plays a crucial role in environmental protection and sustainable development. This data covers several key areas, including air quality, water quality, and soil pollution levels. It forms the basis for assessing the environmental situation, formulating relevant policies, and taking appropriate measures.

[0003] However, most existing environmental monitoring technologies use fixed thresholds to determine anomalies. In actual monitoring scenarios, the instantaneous changes caused by external interference and the continuous accumulation of anomalies in the environment itself may overlap in value. In addition, fixed thresholds cannot be dynamically adjusted according to on-site working conditions, interference frequency and anomaly degree. Therefore, the system cannot effectively identify whether the monitoring data is an interference signal formed by instantaneous fluctuations or a real anomaly formed by actual pollution accumulation. As a result, there is a common problem of difficulty in distinguishing between external interference and anomalies. Therefore, this invention proposes a data processing method, device, and storage medium based on environmental monitoring to address the shortcomings of existing technologies. Summary of the Invention

[0004] The purpose of this invention is to provide a data processing method, device, and storage medium based on environmental monitoring to solve the problems mentioned in the background art.

[0005] The objective of this invention can be achieved through the following technical solutions: Data processing methods based on environmental monitoring include the following steps: Step 1: Receive each monitoring data and preprocess the acquired monitoring data; Step 2: Construct time series of each monitoring data based on the acquisition time of each preprocessed monitoring data; Step 3: Divide the monitoring period into multiple monitoring time periods according to the monitoring cycle, and obtain whether there are any monitoring anomalies in the monitored environment during each monitoring time period based on the time series of the monitoring data. If there are any monitoring anomalies, proceed to Step 4. Step 4: Extract multiple monitoring time periods with monitoring anomalies, analyze the monitoring anomalies based on the monitoring data corresponding to each of the multiple monitoring time periods, and determine the anomaly type; Step 5: Adjust the base threshold according to the identified anomaly type, compare the monitoring data with the adjusted base threshold, and obtain an environmental monitoring report.

[0006] Preferably, the method for performing step three is as follows: The monitoring period is divided into multiple consecutive monitoring time periods of equal duration; For each monitoring time period, feature data of each monitoring data in each monitoring time period is extracted, and the difference between the feature data of each monitoring time period and the previous adjacent monitoring time period is calculated to obtain the change feature vector between monitoring time periods. Based on historical monitoring data, a baseline feature data and a change difference threshold are preset. The feature data of the current monitoring time period are compared with the baseline feature data, and the difference between the feature data of the current monitoring time period and the previous time period is compared with the change difference threshold. When the feature data exceeds the dynamic benchmark threshold or the change difference exceeds the difference threshold, it is determined that there is a monitoring anomaly. Extract monitoring data from adjacent monitoring time periods of the monitoring time period where an anomaly is determined to exist, obtain the correlation between the adjacent time periods and the monitoring data of the monitoring time period to which the anomaly occurred, and if the correlation meets the preset threshold, the existence of a monitoring anomaly is confirmed; if the correlation does not meet the threshold, it is determined to be transient interference, and false alarms are excluded.

[0007] Preferably, the feature data includes the mean, standard deviation, instantaneous rate of change, and extreme value deviation rate of the monitoring data. The method for obtaining the correlation between the monitoring data of adjacent time periods and the monitoring time period to which the monitoring anomaly occurred is as follows: Extract the two monitoring time periods before and the next monitoring time period from which anomalies are identified; The mean, standard deviation, instantaneous rate of change, and extreme value deviation rate of the monitoring time period with anomalies, the two preceding monitoring time periods, and the last monitoring time period are extracted respectively, and the data of the same characteristics in multiple time periods are normalized. Calculate the correlation coefficients between the monitoring time period with anomalies and multiple similar characteristic data of the two preceding monitoring time periods and the subsequent monitoring time period; The comprehensive correlation coefficient is calculated based on the obtained multiple correlation coefficients.

[0008] Preferably, the correlation coefficients include time-series trend correlation coefficients, fluctuation amplitude correlation coefficients, and extreme value deviation correlation coefficients, and the method for obtaining the correlation coefficients is as follows: The monitoring data sequence of the monitoring period with anomalies is compared with the monitoring data sequence of adjacent monitoring periods to obtain the time series trend correlation coefficient based on the degree of synchronization of the two monitoring data time series over time. The fluctuation characteristics of the monitoring data during the abnormal monitoring period are matched with the fluctuation characteristics of the monitoring data during adjacent monitoring periods, and the fluctuation amplitude correlation coefficient is obtained based on the similarity of the fluctuation magnitudes of the two. The deviation characteristics of the extreme values ​​and mean values ​​of the monitoring data during the abnormal monitoring period are compared with the deviation characteristics of the corresponding monitoring data during adjacent monitoring periods. Based on the similarity of the deviation patterns, the extreme value deviation correlation coefficient is obtained.

[0009] Preferably, the method for performing step four is as follows: An anomaly analysis sample set is constructed based on the monitoring data of the abnormal monitoring period, the adjacent normal monitoring period, the time series change characteristics, the fluctuation characteristics, and the extreme value deviation characteristics.

[0010] The trend and fluctuation of the monitoring data in the abnormal analysis sample set are compared with the external interference-related characteristics in the historical records to determine whether the abnormal monitoring data is an external interference feature. By comparing the changing trends and cumulative degree of monitoring data in the abnormal analysis sample set with the typical characteristics of its own environmental pollution, it can be determined whether the abnormal monitoring data are real environmental abnormalities. If it meets the characteristics of external interference, it is determined to be interference from external factors; If it matches the characteristics of its own environment, it is determined to be an abnormal environment.

[0011] Preferably, the method for performing step five is as follows: When the monitoring anomaly is caused by external interference, the basic threshold of each monitoring data is adjusted upward based on the magnitude of the corresponding monitoring data change, the duration of the change, and the frequency of the interference. When the environment itself is abnormal, the basic threshold of each monitoring data is adjusted downward based on the cumulative increase, duration and stability of the abnormality of the monitoring data. The adjusted threshold is verified to ensure that it is not lower than the minimum detection limit of the monitoring equipment and not higher than the limit specified in the environmental standard. Using the finalized adjusted thresholds, each monitoring data point is compared, and normal data, abnormal data, and their corresponding levels are marked to generate and output a complete environmental monitoring report.

[0012] Preferably, the process of adjusting the base threshold upwards is as follows: Based on the monitoring data during the analysis period where anomalies exist, obtain the interference peak value, interference duration, and interference frequency of the monitoring data, and obtain the interference exceedance difference value by comparing the interference peak value with the original base threshold. The floating coefficient is set according to the duration and frequency of the interference. Multiply the difference between the interference threshold and the floating coefficient to obtain the upward floating value of the base threshold; The threshold after upward floating is obtained by adding the base threshold to the threshold floating upward.

[0013] Preferably, the process of downwardly adjusting the base threshold is as follows: Based on the monitoring data within the analysis period where anomalies exist, obtain the stable anomaly mean of the anomaly data, and obtain the anomaly deviation difference between the stable anomaly mean and the original baseline threshold.

[0014] Adjustment coefficients are set based on the duration and accumulation of the anomaly. Multiply the abnormal deviation difference by the correction factor to obtain the threshold downward correction value.

[0015] Subtract the downward correction amount from the original base threshold to obtain the downward-corrected threshold.

[0016] A data processing device includes a memory, a processor, and a machine-executable program stored in the memory and running on the processor.

[0017] A storage medium, the storage medium being a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium.

[0018] The beneficial effects of this invention are: 1. This invention preprocesses environmental monitoring data, constructs a time series of monitoring data, divides monitoring time periods according to the monitoring cycle, and identifies sudden monitoring anomalies based on multiple types of monitoring data and adjacent time periods. It distinguishes between the types of monitoring anomalies and determines whether they are due to external interference or internal environmental anomalies. Based on the judgment results, it adaptively and dynamically adjusts the basic threshold by using upward floating adjustment and downward correction adjustment methods, effectively distinguishing between external interference and actual environmental pollution, significantly reducing the false alarm rate and missed alarm rate in the environmental monitoring process, and achieving the effect of dynamic adaptive optimization of monitoring thresholds according to actual working conditions.

[0019] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a three-dimensional schematic diagram of the data processing method, equipment, and storage medium based on environmental monitoring according to the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Please see Figure 1 As shown, this invention is a data processing method based on environmental monitoring. Environmental monitoring refers to the activities of environmental monitoring agencies in monitoring and measuring environmental quality through chemical, physical, biological, remote sensing, telemetry, and computer technologies. Its core task is to acquire indicator data reflecting environmental quality, assess environmental pollution status and environmental quality levels, and provide a scientific basis for environmental management, law enforcement supervision, pollution control, and policy formulation. After completing environmental monitoring, the acquired industrial information and data are processed to obtain a corresponding environmental monitoring report. The data processing method of this invention includes the following steps: Step 1: Receive and preprocess the acquired monitoring data. Specifically, this invention is mainly applied to online environmental monitoring scenarios at industrial park exhaust outlets. It uses volatile organic compound concentration sensors, particulate matter monitoring probes, temperature and humidity sensors, and wind speed and direction sensors deployed at the industrial park exhaust outlets to collect real-time environmental monitoring data such as exhaust gas concentration, particulate matter content, temperature, humidity, wind speed, and wind direction. After acquiring multiple types of environmental monitoring data, the data is uploaded to a data processing terminal. Because the original monitoring data may experience occasional data loss, random noise, instantaneous jumps, and transmission point drops during acquisition and transmission, it cannot be directly used for subsequent time series construction and anomaly detection. Therefore, preprocessing of the received monitoring data is necessary. The preprocessing process includes data deduplication, linear interpolation completion of missing values, outlier removal, sliding window filtering for noise reduction, and data normalization. Preprocessing removes invalid data, smooths noise interference, and ensures the continuity and validity of the monitoring data.

[0024] Step two involves constructing time series of each monitoring data based on the acquisition time of each preprocessed monitoring data. After preprocessing various environmental monitoring data from the industrial park's exhaust gas outlets, based on the precise acquisition timestamps corresponding to each preprocessed monitoring data, the monitoring data, including volatile organic compound concentration, flue gas particulate matter content, temperature, humidity, wind speed, and wind direction, are uniformly sorted and aligned in chronological order. This constructs an independent and continuous monitoring data time series for each type of environmental monitoring indicator, enabling multiple types of monitoring data at the same time to form corresponding correlations. At the same time, data fragments with abnormal time stamps or disordered time sequences are removed, ensuring that each monitoring data time series has completeness, consistency, and comparability in terms of time sequence.

[0025] Step 3: Divide the monitoring period into multiple monitoring time periods based on the monitoring cycle. Based on the time series of the monitoring data for each period, determine whether there are any monitoring anomalies in the monitored environment during each monitoring time period. If monitoring anomalies are found, proceed to Step 4. After completing Step 2 and constructing independent and continuous monitoring data time series for various environmental monitoring indicators, the monitoring cycle is divided into multiple monitoring time periods based on the monitoring needs and actual operating conditions of the industrial park's exhaust gas emission outlets. The environmental monitoring cycle is 24 hours. A 24-hour monitoring cycle can comprehensively cover the exhaust gas emission characteristics during both production and non-production periods in the industrial park, and can accurately capture the temporal patterns of instantaneous interference and environmental anomalies. Following the principle of continuous and equal duration, the monitoring time series is determined based on the data collection frequency. Each monitoring period lasts 30 minutes, and each period contains multiple continuously collected monitoring data samples. After dividing the monitoring into multiple periods, characteristic data for each type of monitoring data—volatile organic compound (VOC) concentration, particulate matter content in flue gas, temperature, humidity, wind speed, and wind direction—are extracted for each period. The specific characteristic data for each type of monitoring data includes the monitoring data mean, standard deviation, instantaneous rate of change, and extreme value deviation rate. The monitoring data mean reflects the overall level of the corresponding monitoring indicator within the monitoring period, accurately representing the average concentration of exhaust gas emissions and the average state of environmental parameters. For example, the average VOC concentration directly reflects the overall emission intensity of VOCs in the exhaust gas within that 30-minute period. The standard deviation is used to obtain... The dispersion of monitoring data within a monitoring period reflects the fluctuation range of the monitoring indicators. For example, a larger standard deviation of particulate matter content in flue gas indicates that the emission of particulate matter in the exhaust gas is more unstable during that period, and there may be abnormal fluctuations. The instantaneous rate of change is used to capture sudden jumps in monitoring data within that period, accurately identifying instantaneous interference or abnormal changes. For example, an excessively large instantaneous rate of change in wind speed may correspond to external interference such as instantaneous gusts, and an excessively large instantaneous rate of change in volatile organic compound concentration may correspond to abnormal exhaust gas emissions or instantaneous false transmissions from sensors. The extreme value deviation rate reflects the degree of deviation between the extreme values ​​and the mean values ​​of monitoring data within that period, avoiding the omission of anomalies caused by the neglect of extreme value data. For example, the deviation rate between the maximum value and the mean value of volatile organic compound concentration. Excessive concentrations of waste gas emissions may occur during periods of high concentration. After extracting the characteristic data for each monitoring period, the differences between the same type of characteristic data in the current monitoring period and the previous adjacent monitoring period are calculated for each type of monitoring indicator. These differences include the difference between the mean of the current monitoring period and the mean of the previous monitoring period, the difference between the standard deviation of the current period and the standard deviation of the previous period, the difference between the instantaneous rate of change of the current period and the instantaneous rate of change of the previous period, and the difference between the extreme value deviation rate of the current period and the extreme value deviation rate of the previous period. These four types of characteristic data differences are combined in sequence to form a change feature vector between monitoring periods. The change feature vector is used to reflect the magnitude and trend of changes in the monitoring indicators between two adjacent time periods.

[0026] Subsequently, based on the historical monitoring data of the industrial park's exhaust gas outlets over the past 12 months, and the typical characteristics of external interference and the typical characteristics of its own environmental anomalies under similar historical scenarios, baseline characteristic data and change difference thresholds were preset for each type of monitoring indicator. Typical characteristics of external interference include sudden changes in monitoring data caused by vehicle traffic, instantaneous gusts, and equipment start-up and shutdown. Typical characteristics of the own environmental anomalies include continuous increases in monitoring data caused by accumulated exhaust gas concentration and pipeline deposits. The baseline characteristic data is the statistical average of various characteristic data within a historical normal monitoring period, reflecting the normal environmental state of the monitoring point. The change difference threshold is the maximum difference between characteristic data in adjacent time periods within a historical normal monitoring period, used to distinguish between normal fluctuations and abnormal changes. Both the baseline characteristic data and the change difference threshold are dynamically adjusted based on updates to historical data. After presetting the baseline characteristic data and the change difference threshold, the current... The various feature data of the previous monitoring period are compared with the corresponding baseline feature data to determine whether the feature data of the current period exceeds the reasonable range of the baseline feature data. At the same time, the difference of each type of feature data in the change feature vector of the current monitoring period and the previous adjacent monitoring period is compared with the corresponding change difference threshold to determine whether the change amplitude exceeds the normal range. When the feature data of any type of monitoring indicator in the current monitoring period exceeds the corresponding baseline feature data and the feature data difference of the monitoring indicator exceeds the corresponding change difference threshold, it is preliminarily determined that there is a monitoring anomaly in the monitoring period. For example, if only the feature data exceeds the baseline but the change difference does not exceed the threshold, it may be a normal fluctuation of the environment. If only the change difference exceeds the threshold but the feature data does not exceed the baseline, it may be an instantaneous fluctuation of the data. Only when both conditions are met is it considered that there is a high probability of an anomaly.

[0027] To ensure the accuracy of anomaly detection and eliminate misjudgments caused by transient interference, adjacent monitoring time periods are extracted from the monitoring time period where an anomaly is initially identified. These adjacent monitoring time periods are the two consecutive monitoring time periods preceding and following the anomaly monitoring time period. The two consecutive monitoring time periods preceding and following the anomaly monitoring time period cover the environmental conditions before and after the anomaly. By analyzing the correlation between monitoring data before and after the anomaly, it is determined whether the anomaly is an isolated transient interference. Subsequently, four types of feature data are extracted from the monitoring time period where the anomaly exists, the two consecutive monitoring time periods preceding the anomaly, and the subsequent monitoring time period: mean, standard deviation, instantaneous rate of change, and extreme value deviation rate. Since the measurement units of the various feature data are different, direct comparison will lead to deviations in correlation calculation. Therefore, it is necessary to normalize the similar feature data of these four time periods and map all types of feature data to the [0,1] interval. After completing the normalization process, the correlation coefficients of multiple similar feature data of the monitoring time period where the anomaly exists and the two consecutive monitoring time periods preceding and following the anomaly are calculated respectively.

[0028] The correlation coefficients include time-series trend correlation coefficient, fluctuation amplitude correlation coefficient, and extreme value deviation correlation coefficient. The time-series trend correlation coefficient is obtained by comparing the trend consistency of monitoring data sequences from monitoring periods with those from adjacent monitoring periods, based on the degree of synchronization between the two time series over time. Specifically, all monitoring data within these two time periods are first extracted and then organized into two complete monitoring data sequences according to the order of data collection. Each data sequence corresponds one-to-one with the collection time. For each data sequence, each monitoring data point is multiplied by its corresponding collection sequence number. Multiplying the data collection sequence numbers gives greater weight to the later collected data, making the overall trend closer to the later changes in the data. Then, summing all the multiplication results gives the product sum of the sequence. Summing all the collection sequence numbers in the sequence gives the sequence number sum. Then, dividing the product sum of each data sequence by its corresponding sequence number sum calculates the time series trend value for the abnormal monitoring period and adjacent monitoring periods. Finally, dividing the smaller time series trend value by the larger time series trend value gives the time series trend correlation coefficient. If both time series trend values ​​are 0, the time series trend correlation coefficient is 1. The larger the coefficient, the higher the degree of synchronization of the time series changes of the monitoring data in the two time periods.

[0029] The correlation coefficient of fluctuation amplitude is obtained by matching the fluctuation characteristics of monitoring data in the abnormal monitoring period with the fluctuation characteristics of monitoring data in adjacent monitoring periods, based on the similarity of the fluctuation magnitudes. For example, if the standard deviation of flue gas particulate matter content in the abnormal period and the adjacent period are similar, it indicates that the fluctuation amplitudes are consistent and the correlation coefficient of fluctuation amplitude is high. If the standard deviation of the abnormal period is much larger than that of the adjacent period, it indicates abnormal fluctuations, which may be due to transient interference. Specifically, all monitoring data in the abnormal monitoring period and any adjacent monitoring period are first extracted separately. Combined with the previously preprocessed data and feature extraction results, the correlation coefficient is then calculated. Obtain the standard deviation of the monitoring data within these two time periods. Calculate the difference between the two standard deviations and take the absolute value of this difference to obtain the absolute difference in the fluctuation amplitude between the two time periods. Then, divide the calculated absolute difference by the larger of the two standard deviations to obtain the relative difference percentage of the fluctuation amplitude between the two time periods. Finally, subtract this relative difference percentage from 1 to obtain the fluctuation amplitude correlation coefficient. If both standard deviations are 0, the fluctuation amplitude correlation coefficient is 1. The closer the coefficient is to 1, the closer the fluctuation amplitudes of the monitoring data in the two time periods are, and the higher the matching degree of the fluctuation characteristics.

[0030] The extreme value deviation correlation coefficient is obtained by comparing the deviation characteristics of extreme values ​​from the mean of monitoring data in the abnormal monitoring period with the deviation characteristics of corresponding monitoring data in adjacent monitoring periods. The coefficient is based on the similarity of the deviation patterns. For example, if the deviation rates of extreme values ​​and mean values ​​of volatile organic compound concentrations in the abnormal period and adjacent periods are similar and the deviation directions are consistent, the extreme value deviation correlation coefficient is high. If the deviation rate of the abnormal period is much greater than that of the adjacent periods and the deviation direction is irregular, the extreme value deviation correlation coefficient is low. Specifically, all monitoring data within the abnormal monitoring period and any adjacent monitoring period are extracted, and the maximum, minimum, and mean values ​​of the monitoring data within each of the two periods are obtained. The maximum and minimum values ​​are the extreme values ​​of the monitoring data within the corresponding periods, and the mean value is the overall value of the monitoring data within the corresponding periods. For each time period, the deviation rates of the maximum and minimum values ​​from the mean are calculated separately. The calculation method is as follows: divide the absolute value of the difference between the maximum and the mean by the mean to obtain the maximum deviation rate; divide the absolute value of the difference between the minimum and the mean by the mean to obtain the minimum deviation rate. Subtract the maximum deviation rate of the abnormal monitoring time period from the maximum deviation rate of the adjacent monitoring time period to obtain the absolute value of the difference. Similarly, calculate the absolute value of the difference between the minimum deviation rates of the two time periods. Add these two absolute values ​​and take the average to obtain the average difference value of the extreme value deviation characteristics between the two time periods. Finally, subtract this average difference value from 1 to obtain the extreme value deviation correlation coefficient. The closer the extreme value deviation correlation coefficient is to 1, the more similar the deviation patterns of the extreme values ​​and the mean of the monitoring data in the two time periods are, and the higher the matching degree of the extreme value deviation characteristics.

[0031] After obtaining the time-series trend correlation coefficient, fluctuation amplitude correlation coefficient, and extreme value deviation correlation coefficient between the monitoring time period with anomalies and each adjacent time period, corresponding weights are assigned according to the importance of each type of correlation coefficient: time-series trend correlation coefficient weight 0.3, fluctuation amplitude correlation coefficient weight 0.25, and extreme value deviation correlation coefficient weight 0.25. The three types of correlation coefficients for each adjacent time period are multiplied by their respective weights and then summed to obtain the comprehensive correlation coefficient between the monitoring time period with anomalies and its adjacent time periods. Then, the average of the comprehensive correlation coefficients of all adjacent time periods is taken as the final comprehensive correlation coefficient. Finally, the obtained comprehensive correlation coefficient is... The coefficient is compared with a preset comprehensive correlation coefficient threshold, which is obtained based on historical normal and abnormal data and has a value of 0.6. If the final comprehensive correlation coefficient meets the preset threshold, it indicates that the monitoring data of the abnormal monitoring period is significantly correlated with the data of adjacent time periods, and the abnormality does not exist in isolation. This confirms the existence of a monitoring abnormality within the monitoring period. If the comprehensive correlation coefficient does not meet the preset threshold, it indicates that the data of the abnormal monitoring period is an isolated burst and is not related to the environmental state of adjacent time periods. This is determined to be transient interference, and the false alarm is excluded. Through this neighborhood correlation verification mechanism, the accuracy of monitoring abnormality judgment is further improved.

[0032] Step four involves extracting multiple monitoring time periods exhibiting anomalies. Based on the monitoring data corresponding to these time periods, the anomalies are analyzed to determine their types. After confirming the existence of genuine anomalies, all monitoring time periods marked as anomalies, along with their corresponding preceding and following normal monitoring time periods, are extracted. Simultaneously, the temporal variation characteristics, fluctuation characteristics, and extreme value deviation characteristics of these time periods are extracted to construct a complete and comparative anomaly analysis sample set. The characteristics of monitoring data change trends, fluctuation amplitudes, and durations in the anomaly analysis sample set are compared item by item with the typical characteristics of external interference in the pre-stored historical records. This determines whether the abnormal monitoring data exhibits external interference characteristics such as instantaneous changes, rapid declines, lack of sustained accumulation, and single-point outbreaks. For example, the change trend and fluctuation amplitude of the current abnormal data, which show an instantaneous surge in monitoring concentration, a rapid decline in a short period of time, lack of sustained accumulation, and only appearing in a single monitoring period, are compared with the typical sudden waveforms, short-term spikes, and rapid recovery characteristics formed by external interference such as passing vehicles, instantaneous gusts, and equipment start-up and shutdown in the historical records. If they are consistent, the monitoring data anomaly is caused by external interference.

[0033] Simultaneously, the characteristics of monitoring data change trends, cumulative increase magnitude, and spatial distribution consistency in the abnormal analysis sample set are compared in depth with the typical characteristics of real environmental anomalies corresponding to their own environmental pollution, such as continuous slow increase, stable fluctuation, gradual accumulation and diffusion, and multi-point synchronous response. This is to determine whether the abnormal monitoring data conforms to the change pattern of real environmental pollution. For example, the trend and degree of change of the current abnormal data, which shows a gradual increase in monitoring concentration, a sustained high level, relatively stable fluctuation, and continuous accumulation over multiple consecutive monitoring periods, are compared with the characteristics of slow rise, continuous accumulation, and stable high level formed by their own environmental pollution in historical records due to waste gas leakage, abnormal treatment facilities, and continuous increase in production load. If the two match, it is determined that the monitoring data is abnormal due to real environmental anomalies.

[0034] Step 5: Adjust the base threshold according to the determined anomaly type, and compare the monitoring data with the adjusted base threshold to obtain an environmental monitoring report. Specifically, based on the determined monitoring anomaly type, the base threshold is adaptively adjusted in a differentiated manner. When the monitoring anomaly is determined to be due to external interference, firstly, based on the monitoring data within the analysis period of the anomaly, obtain the interference peak value, interference duration, and interference frequency corresponding to this interference. Calculate the difference between the interference peak value and the original base threshold to obtain the interference exceeding the threshold difference. Then, set the corresponding floating coefficient according to the interference duration and interference frequency. The longer the interference duration and the higher the frequency, the larger the floating coefficient; the shorter the interference duration and the lower the frequency, the smaller the floating coefficient. Subsequently, multiply the interference exceeding the threshold difference by the floating coefficient to obtain the upward floating value of the base threshold. Finally, add the original base threshold and the upward floating value to obtain the new base threshold after upward floating. This improves the anti-interference capability of subsequent anomaly determination and avoids false alarms caused by similar external interference.

[0035] When a monitored anomaly is determined to be an environmental anomaly, the following steps are taken: First, based on the monitoring data within the analysis period of the anomaly, the stable anomaly mean value during the stable phase is obtained. The anomaly deviation difference is calculated by subtracting the stable anomaly mean value from the original baseline threshold. Then, a corresponding correction coefficient is set based on the duration and accumulation of the anomaly. The longer the anomaly duration and the larger the accumulation, the larger the correction coefficient; the shorter the anomaly duration and the smaller the accumulation, the smaller the correction coefficient. Subsequently, the anomaly deviation difference is multiplied by the correction coefficient to obtain the threshold downward correction value. Finally, the original baseline threshold is subtracted from the threshold downward correction value to obtain the new baseline threshold after downward correction. This improves the sensitivity of identifying real environmental pollution and avoids missed detections.

[0036] After completing the above threshold adjustments, the rationality of the new basic thresholds is verified to ensure that the adjusted thresholds are not lower than the minimum detection limit of the monitoring equipment and not higher than the limits stipulated by national or local environmental standards, thus ensuring that the thresholds are within a reasonable and effective range. Finally, the final adjusted thresholds are used to compare all environmental monitoring data point by point, marking normal data, abnormal data and their corresponding levels. Based on the comparison results, a complete environmental monitoring report is generated and output, thereby realizing dynamic threshold adaptive adjustment based on anomaly types, which reduces the false alarm rate while improving the accuracy of monitoring real environmental pollution.

[0037] Finally, the acquired monitoring data is compared with the adjusted monitoring data thresholds, and a complete environmental monitoring data report is output based on the comparison results of each data item.

[0038] This application also provides a data processing device, including a memory, a processor, and a machine-executable program stored in the memory and running on the processor, wherein the processor executes the machine-executable program to implement the above-described data processing method based on environmental monitoring.

[0039] This application also provides a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the above-described data processing method based on environmental monitoring.

[0040] The above description is merely an example and illustration of the concept of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described or use similar methods to replace them, as long as they do not deviate from the concept of the invention or exceed the scope defined in the claims, they should all fall within the protection scope of the present invention.

Claims

1. A data processing method based on environmental monitoring, characterized in that, Includes the following steps: Step 1: Receive each monitoring data and preprocess the acquired monitoring data; Step 2: Construct time series of each monitoring data based on the acquisition time of each preprocessed monitoring data; Step 3: Divide the monitoring period into multiple monitoring time periods according to the monitoring cycle, and obtain whether there are any monitoring anomalies in the monitored environment during each monitoring time period based on the time series of the monitoring data. If there are any monitoring anomalies, proceed to Step 4. Step 4: Extract multiple monitoring time periods with monitoring anomalies, analyze the monitoring anomalies based on the monitoring data corresponding to each of the multiple monitoring time periods, and determine the anomaly type; Step 5: Adjust the base threshold according to the identified anomaly type, compare the monitoring data with the adjusted base threshold, and obtain an environmental monitoring report.

2. The data processing method based on environmental monitoring according to claim 1, characterized in that, The method for performing step three is as follows: The monitoring period is divided into multiple consecutive monitoring time periods of equal duration; For each monitoring time period, feature data of each monitoring data in each monitoring time period is extracted, and the difference between the feature data of each monitoring time period and the previous adjacent monitoring time period is calculated to obtain the change feature vector between monitoring time periods. Based on historical monitoring data, a baseline feature data and a change difference threshold are preset. The feature data of the current monitoring time period are compared with the baseline feature data, and the difference between the feature data of the current monitoring time period and the previous time period is compared with the change difference threshold. When the feature data exceeds the dynamic benchmark threshold or the change difference exceeds the difference threshold, it is determined that there is a monitoring anomaly. Extract monitoring data from adjacent monitoring time periods of the monitoring time period where an anomaly is determined to exist, obtain the correlation between the adjacent time periods and the monitoring data of the monitoring time period to which the anomaly occurred, and if the correlation meets the preset threshold, the existence of a monitoring anomaly is confirmed; if the correlation does not meet the threshold, it is determined to be transient interference, and false alarms are excluded.

3. The data processing method based on environmental monitoring according to claim 2, characterized in that, The characteristic data includes the mean, standard deviation, instantaneous rate of change, and extreme value deviation rate of the monitoring data. The method for obtaining the correlation between the monitoring data of adjacent time periods and the monitoring time period to which the monitoring anomaly occurred is as follows: Extract the two monitoring time periods before and the next monitoring time period from which anomalies are identified; The mean, standard deviation, instantaneous rate of change, and extreme value deviation rate of the monitoring time period with anomalies, the two preceding monitoring time periods, and the last monitoring time period are extracted respectively, and the data of the same characteristics in multiple time periods are normalized. Calculate the correlation coefficients between the monitoring time period with anomalies and multiple similar characteristic data of the two preceding monitoring time periods and the subsequent monitoring time period; The comprehensive correlation coefficient is calculated based on the obtained multiple correlation coefficients.

4. The data processing method based on environmental monitoring according to claim 3, characterized in that, The correlation coefficients include time-series trend correlation coefficients, fluctuation amplitude correlation coefficients, and extreme value deviation correlation coefficients. The method for obtaining the correlation coefficients is as follows: The monitoring data sequence of the monitoring period with anomalies is compared with the monitoring data sequence of adjacent monitoring periods to obtain the time series trend correlation coefficient based on the degree of synchronization of the two monitoring data time series over time. The fluctuation characteristics of the monitoring data in the monitoring period with anomalies are matched with the fluctuation characteristics of the monitoring data in adjacent monitoring periods, and the fluctuation amplitude correlation coefficient is obtained based on the similarity of the fluctuation magnitudes of the two. The deviation characteristics of the extreme values ​​and mean values ​​of the monitoring data in the monitoring period with abnormality are compared with the deviation characteristics of the corresponding monitoring data in adjacent monitoring periods. Based on the similarity of the deviation patterns, the extreme value deviation correlation coefficient is obtained.

5. The data processing method based on environmental monitoring according to claim 1, characterized in that, The method for performing step four is as follows: Based on the monitoring data of the monitoring period with anomalies, the adjacent normal monitoring period, the time series change characteristics, the fluctuation characteristics, and the extreme value deviation characteristics, an anomaly analysis sample set is constructed. The trend and fluctuation of the monitoring data in the abnormal analysis sample set are compared with the external interference-related characteristics in the historical records to determine whether the abnormal monitoring data is an external interference feature. By comparing the changing trends and cumulative degree of monitoring data in the abnormal analysis sample set with the typical characteristics of its own environmental pollution, it can be determined whether the abnormal monitoring data are real environmental abnormalities. If it meets the characteristics of external interference, it is determined to be interference from external factors; If it matches the characteristics of its own environment, it is determined to be an abnormal environment.

6. The data processing method based on environmental monitoring according to claim 5, characterized in that, The method for performing step five is as follows: When the monitoring anomaly is caused by external interference, the basic threshold of each monitoring data is adjusted upward based on the magnitude of the corresponding monitoring data change, the duration of the change, and the frequency of the interference. When the environment itself is abnormal, the basic threshold of each monitoring data is adjusted downward based on the cumulative increase, duration and stability of the abnormality of the monitoring data. The adjusted threshold is verified to ensure that it is not lower than the minimum detection limit of the monitoring equipment and not higher than the limit specified in the environmental standard. Using the finalized adjusted thresholds, each monitoring data point is compared, and normal data, abnormal data, and their corresponding levels are marked to generate and output a complete environmental monitoring report.

7. The data processing method based on environmental monitoring according to claim 6, characterized in that, The process of adjusting the base threshold upwards is as follows: Based on the monitoring data during the analysis period where anomalies exist, obtain the interference peak value, interference duration, and interference frequency of the monitoring data, and obtain the interference exceedance difference value by comparing the interference peak value with the original base threshold. The floating coefficient is set according to the duration and frequency of the interference. Multiply the difference between the interference threshold and the floating coefficient to obtain the upward floating value of the base threshold; The threshold after upward floating is obtained by adding the base threshold to the threshold floating upward.

8. The data processing method based on environmental monitoring according to claim 7, characterized in that, The process of adjusting the base threshold downwards is as follows: Based on the monitoring data within the analysis period where anomalies exist, obtain the stable anomaly mean of the anomaly data, and obtain the anomaly deviation difference between the stable anomaly mean and the original baseline threshold. Adjustment coefficients are set based on the duration and accumulation of the anomaly. Multiply the abnormal deviation difference by the correction factor to obtain the threshold downward correction value; Subtract the threshold downward correction value from the original base threshold to obtain the downward-corrected threshold.

9. A data processing device, characterized in that, It includes a memory, a processor, and a machine-executable program stored in the memory and running on the processor, wherein the processor, when executing the machine-executable program, implements a data processing method based on environmental monitoring according to any one of claims 1-8.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements a data processing method based on environmental monitoring as described in any one of claims 1-8.

Images