An environmental monitoring data analysis method and related device

Abstract

The application discloses an environmental monitoring data analysis method and related device, relates to the technical field of environmental monitoring, obtains the standard sample data consistent with the attribute of the to-be-analyzed environmental monitoring data, calls a density-based clustering algorithm to process the two kinds of data, obtains a clustering result, and in the case that a turning point in a K-distance graph of the to-be-analyzed environmental monitoring data does not satisfy a preset turning point condition, determines a reference turning point by using a sliding window algorithm, so that the clustering result is more accurate; when the clustering result satisfies a condition, multi-dimensional quality analysis is performed on the to-be-analyzed environmental monitoring data, and the efficiency and accuracy of environmental monitoring data analysis are further improved; when the to-be-analyzed environmental monitoring data is subjected to multi-dimensional quality analysis, at least data stability and correlation of the to-be-analyzed environmental monitoring data are analyzed, so that the accuracy of an analysis result obtained by multi-dimensional analysis is greatly improved, and the accuracy of application of the analysis result to other scenes, such as data quality evaluation, is further improved.

Description

Technical Field

[0001] This application relates to the field of environmental monitoring technology, and in particular to a method and related apparatus for analyzing environmental monitoring data. Background Technology

[0002] With the increasing awareness of environmental protection and the development of technology, the scale of environmental monitoring networks continues to expand, and the monitoring data shows a trend of massive growth.

[0003] Among them, environmental monitoring data serves as the fundamental basis for objectively representing the state of environmental quality, reflecting the effectiveness of pollution control, and implementing environmental management and decision-making. The accuracy of its data analysis results is directly related to the scientific nature and effectiveness of environmental policies.

[0004] Currently, environmental monitoring data analysis mainly relies on manual analysis based on targets, or statistical testing of data according to predetermined dimensions. However, this method is inefficient, the accuracy of data analysis results is poor, and it cannot meet higher analysis needs, thus affecting the effectiveness of reusing the analysis results. Summary of the Invention

[0005] In view of the above problems, this application provides a method and related apparatus for analyzing environmental monitoring data to achieve high accuracy in environmental monitoring data analysis. The specific solution is as follows:

[0006] The first aspect of this application provides a method for analyzing environmental monitoring data, including:

[0007] Obtain the environmental monitoring data to be analyzed, as well as standard sample data with attributes consistent with the environmental monitoring data to be analyzed; the standard sample data is data stored in a standard sample database divided according to data attributes.

[0008] A density-based clustering algorithm is used to process the environmental monitoring data to be analyzed and the standard sample data to determine the K-distance map of the environmental monitoring data to be analyzed.

[0009] If the inflection point in the K-distance graph does not meet the preset inflection point conditions, a reference inflection point is determined using the sliding window algorithm, and the K-distance corresponding to the reference inflection point is used as the neighborhood radius of the environmental monitoring data to be analyzed, thus obtaining the clustering result of the environmental monitoring data to be analyzed; the clustering result includes at least one environmental monitoring data cluster;

[0010] If the clustering results meet the preset conditions, multidimensional quality analysis is performed on the environmental monitoring data to be analyzed. The multidimensional quality analysis includes at least data stability analysis and data correlation analysis.

[0011] Obtain the multidimensional quality analysis results of the environmental monitoring data to be analyzed. The multidimensional quality analysis results shall include at least: a first analysis result for characterizing the stability of the environmental monitoring data to be analyzed, and a second analysis result for characterizing the correlation of the environmental monitoring data to be analyzed.

[0012] In one possible implementation, the K-distance map is a distance map arranged in descending order. If an inflection point in the K-distance map does not meet a preset inflection point condition, a sliding window algorithm is used to determine a reference inflection point, and the K-distance corresponding to the reference inflection point is used as the neighborhood radius of the environmental monitoring data to be analyzed, including:

[0013] In the absence of obvious inflection points in the K-distance graph, the difference between any two adjacent environmental monitoring data points in the K-distance graph is calculated sequentially to obtain the difference set;

[0014] Calculate the relative rate of change between any two adjacent difference values ​​in the difference set to obtain the set of relative rates of change;

[0015] The first relative change rate in the set of relative change rates that is greater than a preset percentage is determined as the target relative change rate;

[0016] The environmental monitoring data point with the smallest K-distance among the four environmental monitoring data points corresponding to the target relative change rate is taken as the reference inflection point, and the K-distance corresponding to the reference inflection point is taken as the neighborhood radius of the environmental monitoring data to be analyzed.

[0017] In one possible implementation, when an inflection point in the K-distance graph does not meet the preset inflection point conditions, a sliding window algorithm is used to determine a reference inflection point, and the K-distance corresponding to the reference inflection point is used as the neighborhood radius of the environmental monitoring data to be analyzed. This also includes:

[0018] When all relative change rates in the relative change rate set are lower than a preset percentage, the normal distribution of the relative change rate set is determined.

[0019] The candidate set of reference inflection points in the normal distribution is determined according to the preset confidence interval, and the K distance corresponding to the first reference inflection point in the candidate set is selected as the neighborhood radius of the environmental monitoring data to be analyzed.

[0020] In one possible implementation, multidimensional quality analysis is performed on the environmental monitoring data to be analyzed to obtain quality analysis results, including:

[0021] Data stability analysis is performed on the environmental monitoring data to be analyzed, and the first analytical results are obtained to characterize the stability of the environmental monitoring data to be analyzed.

[0022] A data correlation analysis was performed on the environmental monitoring data to be analyzed, and a second analysis result was obtained to characterize the correlation of the environmental monitoring data to be analyzed.

[0023] In one possible implementation, a data correlation analysis is performed on the environmental monitoring data to be analyzed to obtain a second analysis result characterizing the correlation of the environmental monitoring data to be analyzed, including:

[0024] The environmental monitoring data to be analyzed is compared with relevant data to obtain the second analysis result; the relevant data includes the standard sample data corresponding to the environmental monitoring data to be analyzed, as well as the relevant attribute data corresponding to the environmental monitoring data to be analyzed.

[0025] One possible implementation also includes:

[0026] The results are generated based on the multidimensional quality analysis results; the results can be at least one of the following: multidimensional radar chart, analysis report, or preset output text.

[0027] A second aspect of this application provides an environmental monitoring data analysis apparatus, comprising:

[0028] The first acquisition unit is used to acquire the environmental monitoring data to be analyzed and standard sample data with the same attributes as the environmental monitoring data to be analyzed; the standard sample data is data stored in a standard sample database divided according to data attributes.

[0029] The clustering analysis unit is used to call a density-based clustering algorithm to process the environmental monitoring data and standard sample data to be analyzed, and to determine the K-distance map of the environmental monitoring data to be analyzed.

[0030] The inflection point determination unit is used to determine a reference inflection point using a sliding window algorithm when the inflection point in the K-distance map does not meet the preset inflection point conditions. The K-distance corresponding to the reference inflection point is used as the neighborhood radius of the environmental monitoring data to be analyzed, and the clustering result of the environmental monitoring data to be analyzed is obtained. The clustering result includes at least one environmental monitoring data cluster.

[0031] The quality analysis unit is used to perform multidimensional quality analysis on the environmental monitoring data to be analyzed, provided that the clustering results meet the preset conditions. The multidimensional quality analysis includes at least data stability analysis and data correlation analysis.

[0032] The second acquisition unit is used to acquire the multidimensional quality analysis results of the environmental monitoring data to be analyzed; the multidimensional quality analysis results include at least: a first analysis result used to characterize the stability of the environmental monitoring data to be analyzed, and a second analysis result used to characterize the correlation of the environmental monitoring data to be analyzed.

[0033] A third aspect of this application provides an environmental monitoring data analysis device, comprising at least one processor and a memory connected to the processor, wherein:

[0034] Memory is used to store computer programs;

[0035] The processor is used to execute computer programs so that the environmental monitoring data analysis device can implement the environmental monitoring data analysis method of the first aspect or any implementation thereof.

[0036] The fourth aspect of this application provides a computer program product, including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement the method for analyzing environmental monitoring data described in the first aspect or any implementation thereof.

[0037] The fifth aspect of this application provides a computer storage medium carrying one or more computer programs, which, when executed by an electronic device, enable the electronic device to perform an analysis method for environmental monitoring data as described in the first aspect or any implementation thereof.

[0038] Using the above technical solution, the environmental monitoring data analysis method and related apparatus provided in this application acquire the environmental monitoring data to be analyzed and standard sample data with the same attributes. A density-based clustering algorithm is then used to process these two types of data to obtain a clustering result that includes at least one environmental monitoring data cluster. If the inflection point in the K-distance graph of the environmental monitoring data to be analyzed does not meet the preset inflection point conditions, a sliding window algorithm is used to determine a reference inflection point to make the clustering result more accurate. When the clustering result meets the conditions, multidimensional quality analysis is then performed on the environmental monitoring data to be analyzed, further improving the efficiency and accuracy of environmental monitoring data analysis. Simultaneously, when performing multidimensional quality analysis on the environmental monitoring data to be analyzed, at least its data stability and data correlation are analyzed, thereby greatly improving the accuracy of the analysis results obtained through multidimensional analysis. This further improves the accuracy of applying the analysis results to other scenarios, such as data quality evaluation. Attached Figure Description

[0039] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0040] Figure 1 A flowchart illustrating an environmental monitoring data analysis method provided in this application;

[0041] Figure 2 Example diagram of the two-dimensional correlation coefficient matrix provided in this application;

[0042] Figure 3 A schematic diagram of the structure of the analysis device for environmental monitoring data provided in this application;

[0043] Figure 4A schematic diagram of the structure of the analysis equipment for the environmental monitoring data provided in this application. Detailed Implementation

[0044] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0045] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0046] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0047] With the rapid development of technologies such as the Internet of Things and cloud computing, the coverage and density of environmental monitoring networks are increasing day by day, and the sources, types and scale of monitoring data are showing an explosive growth trend.

[0048] Current environmental monitoring data exhibits typical characteristics of being "multi-source," "heterogeneous," and "high-concurrency." "Multi-source" refers to data originating from various channels, including satellite remote sensing, ground-based sensor networks, mobile monitoring equipment, and enterprise-mandated monitoring. "Heterogeneous" is reflected in the significant differences in data formats, structures, sampling frequencies, and units of measurement. "High-concurrency" is specifically manifested in the continuous, real-time generation of environmental monitoring data streams from a massive number of monitoring terminals.

[0049] Environmental monitoring data can serve as a fundamental basis for objectively representing the state of environmental quality, reflecting the effectiveness of pollution control, and implementing environmental management and decision-making. Therefore, efficient and reliable analysis of environmental monitoring data is the primary prerequisite for subsequent quality assessment and application.

[0050] Traditional environmental monitoring data analysis mainly relies on simple threshold checks, manual sampling verification, or statistical rules designed for single, well-structured data sources. These methods work well when processing small-scale, static environmental monitoring datasets with simple structures. However, they are less effective for complex environmental monitoring data that are multi-source, heterogeneous, and experience high concurrency, exhibiting low efficiency and poor accuracy in data analysis results.

[0051] To address the aforementioned problems, this application provides a method and related apparatus for analyzing environmental monitoring data.

[0052] Optional, see Figure 1 This application provides a flowchart illustrating a method for analyzing environmental monitoring data.

[0053] like Figure 1 As shown, the method for analyzing environmental monitoring data includes the following steps:

[0054] Step 101: Obtain the environmental monitoring data to be analyzed and standard sample data with attributes consistent with the environmental monitoring data to be analyzed.

[0055] It should be noted that the environmental monitoring data to be analyzed is a comprehensive, multi-dimensional big data collection, including but not limited to environmental quality monitoring data, pollution source monitoring data, and auxiliary data. Specifically, environmental quality monitoring data includes, but is not limited to, ambient air quality data, water quality data, soil quality data, acoustic quality data, and ecological quality data; pollution source monitoring data includes, but is not limited to, stationary pollution source data and mobile source data; and auxiliary data includes, but is not limited to, geographic information data, satellite remote sensing data, and socio-economic data.

[0056] Specifically, ambient air quality data includes concentration data for the six conventional parameters: fine particulate matter, inhalable particulate matter, ozone, sulfur dioxide, nitrogen dioxide, and carbon monoxide, as well as data on other characteristic pollutants such as volatile organic compounds, heavy metals, and greenhouse gases; water environment quality data includes data on surface water, drinking water sources, and groundwater, such as monitoring data for sections of rivers, lakes, and reservoirs in surface water data, physical indicators such as water temperature, turbidity, color, and transparency, chemical indicators such as pH value and dissolved oxygen, and biotoxicity indicators; acoustic environment quality data includes equivalent sound level data for functional zone noise, road traffic noise, and regional environmental noise; and ecological quality data includes ecological indicators obtained through remote sensing and other means, such as vegetation cover index and biodiversity monitoring data.

[0057] Standard sample data is stored in a standard sample database divided according to data attributes. Specifically, this database can be an automatic monitoring database of ecological and environmental pollution sources within the target area. This database includes various standard samples stored in a preset standard format, categorized by attributes. For example, this standard sample database could be a national database of automatic monitoring of ecological and environmental pollution sources, storing environmental monitoring data for each province. Therefore, standard sample data is broad-area data with the same attributes as the environmental monitoring data to be analyzed. Specifically, these shared attributes can include the same region, industry, data collection time, and type of treatment facility.

[0058] Optionally, acquire the environmental monitoring data to be analyzed, determine the attributes of the environmental monitoring data to be analyzed, and acquire the standard sample data corresponding to the environmental monitoring data to be analyzed from the standard sample database according to the attributes.

[0059] For example, when the environmental monitoring data to be analyzed is the sewage outlet data of Province A, standard data samples of the sewage outlet data of Province A are obtained from the standard sample database.

[0060] Step 102: Call the density-based clustering algorithm to process the environmental monitoring data to be analyzed and the standard sample data to determine the K-distance map of the environmental monitoring data to be analyzed.

[0061] It should be noted that this step uses the density-based clustering algorithm (DBSCAN, Density-Based Spatial Clustering of Applications with Noise) to perform preliminary data analysis on the environmental data to be analyzed. This algorithm is suitable for data clustering without a data center, such as the environmental monitoring data and standard data samples to be analyzed in this application.

[0062] This clustering algorithm primarily compares the environmental monitoring data to be analyzed with its corresponding standard sample data of the same attribute. This determines which sub-attribute (which sub-standard sample) the data in the environmental monitoring data belongs to under the standard sample data. By performing attribute judgment on the environmental monitoring data, the evaluation scope is narrowed down. Through clustering, it is preliminarily determined that the distribution of the environmental monitoring data matches the distribution of the standard sample data. In other words, the quality of the environmental monitoring data initially meets the quality of the standard sample data. It is easy to understand that this clustering algorithm can automatically identify the distribution attributes of the environmental monitoring data, reducing manual intervention and providing an accurate basis for the subsequent generation of multidimensional quality analysis results.

[0063] Optionally, a density-based clustering algorithm can be invoked to cluster the environmental monitoring data to be analyzed and the standard sample data together, to obtain a clustering result that includes at least one environmental monitoring data.

[0064] Specifically, this clustering algorithm treats each data point in the environmental monitoring data and standard sample data as a data point. It calculates the distance from each data point in the environmental monitoring data to all other data points, sorts all distances from farthest to nearest, and selects the Kth smallest distance as the K-distance point for that data point. The value of K can be preset. Then, after sorting the K-distances of all data points in the environmental monitoring data in descending order, a K-distance map is obtained.

[0065] Step 103: If the inflection point in the K-distance graph does not meet the preset inflection point conditions, a reference inflection point is determined using the sliding window algorithm, and the K-distance corresponding to the reference inflection point is used as the neighborhood radius of the environmental monitoring data to be analyzed, so as to obtain the clustering result of the environmental monitoring data to be analyzed.

[0066] It should be noted that the preset inflection point condition can be the K-distance of the data point corresponding to the position with obvious turning point in the K-distance graph as the neighborhood radius. If the preset inflection point condition is not met, it means that there is no obvious turning point in the K-distance graph.

[0067] There are two main methods for determining reference inflection points from a K-distance map using the sliding window algorithm. The first method is:

[0068] Calculate the difference between any two adjacent data points in descending order of K-distance in the K-distance graph to obtain the difference set. Then calculate the relative rate of change between any two adjacent difference values ​​in the difference set to obtain the relative rate of change set.

[0069] The first relative change rate in the set of relative change rates that is greater than a preset percentage is identified as the target change rate. The preset percentage is a reasonable value that can be set by the user, such as 20%.

[0070] The data point with the smallest K-distance among the four data points corresponding to the target rate of change is taken as the reference inflection point, and the K-distance corresponding to the reference inflection point is taken as the neighborhood radius of the environmental monitoring data to be analyzed, thus obtaining the environmental monitoring data cluster corresponding to the environmental monitoring data to be analyzed.

[0071] The above is the first method for determining the reference inflection point in the K-distance graph. When the above method cannot determine the reference inflection point, a second method can be used, which can be specifically described as follows:

[0072] When all relative change rates in the relative change rate set are lower than a preset percentage, statistical analysis is performed on all relative change rates in the relative change rate set. Assuming that the change rates of all data points follow a normal distribution, a normal distribution map corresponding to the relative change rate set is obtained. The mean and standard deviation of the relative change rates are calculated, and a preset confidence interval is determined with a 90% confidence level. Then, the critical value of the normal distribution, approximately 1.645 times the standard deviation, is used as the threshold to obtain a candidate set of reference inflection points. The K distance corresponding to the first reference inflection point in the candidate set is selected as the neighborhood radius of the environmental monitoring data to be analyzed, thus obtaining the environmental monitoring data cluster corresponding to the environmental monitoring data to be analyzed.

[0073] The candidate set of reference inflection points in the normal distribution is determined according to the preset confidence interval. The K distance corresponding to the first reference inflection point in the candidate set is selected as the neighborhood radius of the monitoring data to be analyzed. The environmental monitoring data cluster corresponding to the environmental monitoring data to be analyzed can then be obtained.

[0074] Optionally, when the inflection point in the K-distance plot is not obvious, a sliding window method is used to determine a reference inflection point from the K-distance plot, and the K-distance corresponding to the reference inflection point is used as the neighborhood radius of the environmental monitoring data to be analyzed, so as to obtain a clustering result that includes at least one environmental monitoring data cluster.

[0075] In summary, this application introduces a sliding window algorithm to optimize the selection of the neighborhood radius when performing cluster analysis on environmental monitoring data and standard sample data, thereby improving the robustness of the cluster analysis and increasing the accuracy of the cluster analysis results.

[0076] Step 104: If the clustering results meet the preset conditions, perform multidimensional quality analysis on the environmental monitoring data to be analyzed; the multidimensional quality analysis includes at least data stability analysis and data correlation analysis.

[0077] After clustering the environmental monitoring data to be analyzed based on the above steps, it is necessary to determine whether the clustering results meet the preset conditions. Specifically, meeting the preset conditions means that the similarity between the clustering results of the environmental monitoring data to be analyzed and the clustering results of the standard samples is higher than a threshold, such as 95%. In other words, the distribution of the environmental monitoring data to be analyzed is basically consistent with the sample data, which proves that its data quality meets certain requirements, making further data quality analysis more meaningful. If the similarity is lower than the threshold, for example, only 50%, it indicates that the distribution of the environmental monitoring data to be analyzed differs significantly from the sample data, indicating poor data quality. In this case, the results can be directly reported, saving the subsequent multi-dimensional data quality analysis process.

[0078] The multidimensional quality analysis types disclosed in this application include, but are not limited to, data quality analysis in two dimensions: data stability analysis and data correlation analysis of the environmental monitoring data to be analyzed. In other words, multidimensional quality analysis can specifically be two-dimensional quality analysis.

[0079] Specifically, data stability analysis can be conducted using a volatility review approach, meaning that the dimension of data stability can include a sub-dimension of volatility.

[0080] The volatility review is primarily used to analyze the degree of volatility in the environmental monitoring data to determine its stability and reliability. Specifically, it involves first calculating the standard deviation of the standard sample data, then calculating the standard deviation of the environmental monitoring data to be analyzed, calculating the absolute value of the relative deviation between the two, and then assigning a score to the absolute value of the relative deviation according to a pre-defined scoring rule. This score is then used as the primary analysis result.

[0081] Data correlation analysis primarily involves comparing the environmental monitoring data to be analyzed with relevant data. Relevant data includes, but is not limited to, the standard sample data corresponding to the environmental monitoring data to be analyzed, as well as the relevant attribute data corresponding to the environmental monitoring data to be analyzed. Specifically, the correlation analysis method for comparing the environmental monitoring data to be analyzed with its standard sample data can be a distributional review. Specifically, the upper and lower quartiles of the standard sample data are first calculated, and then the amount of data in the environmental monitoring data to be analyzed falling within the quartile range is counted. The percentage of data falling within the quartile range is used as the scoring result.

[0082] In correlation analysis, the environmental monitoring data to be analyzed is compared with its corresponding related attribute data. Specifically, this involves determining the related attribute data of the environmental monitoring data to be analyzed, determining the correlation between the environmental monitoring data to be analyzed and its related attribute data, obtaining a two-dimensional correlation coefficient matrix through linear data fitting, and then judging the degree of correlation between the environmental monitoring data to be analyzed and its related attribute data by the magnitude of the correlation coefficients in the two-dimensional correlation coefficient matrix, thus obtaining the correlation degree value.

[0083] For example, see Figure 2 Example diagram of the two-dimensional correlation coefficient matrix provided in this application.

[0084] like Figure 2 As shown, this correlation analysis uses the correlation between various attributes in the environmental monitoring data to be analyzed as the judgment condition, such as SO2 and smoke temperature, flow rate and NOx, flow rate and PM, etc.

[0085] The correlation coefficient matrix between the pollutant and the flue gas parameters was obtained through linear fitting. The degree of correlation of each attribute was judged by the magnitude of the correlation coefficient, and the weight of each degree of correlation was calculated as 1.

[0086] It should also be noted that the multidimensional quality analysis disclosed in the embodiments of this application also includes anomaly analysis. Subsequently, the data quality can be objectively evaluated by analyzing the data stability, data correlation and data anomalies of the environmental monitoring data to be analyzed.

[0087] When performing anomaly analysis, appropriate anomaly analysis methods can be selected to analyze whether the data is abnormal from different perspectives. In other words, the anomaly analysis dimension includes further subdivisions, such as anomaly detection analysis, labeled anomaly detection analysis, and transmission efficiency analysis. Specifically, anomaly detection analysis includes, but is not limited to, outlier detection, detection limit determination, breakpoint detection, and normalization evaluation. Among these, outlier detection primarily identifies environmental monitoring data that clearly does not conform to the actual situation. The judgment rule can be: negative values, zero values, or constant values ​​that remain unchanged over time are all considered outliers in the environmental monitoring data. Detection limit determination is mainly used to assess the measurability of the environmental monitoring data. When the value of the environmental monitoring data is lower than the preset detection limit, it is determined that the environmental monitoring data exceeds the detection limit, and its accuracy and reliability need to be carefully considered. Discontinuity detection includes steep rise and fall detection and discontinuity detection. Steep rise and fall detection is determined if the rate of change between each environmental monitoring data and the data before and after it exceeds ±30% under time-series conditions, then it is determined that there is abnormal fluctuation. Discontinuity detection is mainly for data missing situations, determining the number of consecutively missing environmental monitoring data in the time series, thereby identifying the discontinuities in the data records. Normalization evaluation mainly quantifies the overall data quality of the environmental monitoring data to be analyzed. This method calculates the ratio of the number of all data judged as outliers to the total amount of environmental monitoring data to be analyzed. This ratio can reflect the overall proportion of outliers in the environmental monitoring data to be analyzed.

[0088] Anomaly detection analysis mainly involves anomaly detection and labeling based on existing standards.

[0089] Transmission efficiency analysis primarily analyzes the ratio of the number of valid data points successfully transmitted to the data center within a statistical period to the theoretically required total number of data points. The calculation formula is as follows:

[0090] P = (S / M) × 100%;

[0091] Where P represents the transmission efficiency, S represents the number of valid data received within the statistical period, and M represents the number of data due within the statistical period. Transmission efficiency analysis can determine whether data loss or interruption occurred during transmission, and further, it can analyze whether the sampling frequency of the data changed within the statistical period.

[0092] In summary, data stability analysis and data correlation analysis represent multi-dimensional analysis of the data, yielding analytical results for environmental monitoring data across two dimensions. Adding data anomaly analysis allows for analysis of the environmental monitoring data from three dimensions: data stability, data correlation, and data anomalies. These analytical results can be used for subsequent processing, such as evaluating the reliability, accuracy, and quality of the collected environmental monitoring data.

[0093] In conjunction with the above-mentioned refinement of different dimensions, from another more detailed perspective, the data analysis method disclosed in this application can be seen as comprehensively analyzing environmental monitoring data from five specific sub-dimensions: volatility review, correlation analysis, anomaly detection analysis, anomaly detection analysis, and transmission efficiency analysis, to obtain data quality analysis results.

[0094] Of course, both the three-dimensional analysis at the overall level mentioned above and the five sub-dimensions containing the analysis types after refining these three dimensions are just examples. In actual monitoring scenarios, the analysis dimensions can be adjusted according to different needs and different monitoring objects, and then the sub-dimensions can be adjusted to ultimately generate a multi-dimensional analysis strategy that meets the current needs.

[0095] Step 105: Obtain the multidimensional quality analysis results of the environmental monitoring data to be analyzed; the multidimensional quality analysis results shall include at least: a first analysis result used to characterize the stability of the environmental monitoring data to be analyzed, and a second analysis result used to characterize the correlation of the environmental monitoring data to be analyzed.

[0096] In this embodiment, when the multidimensional quality analysis is a two-dimensional quality analysis, the multidimensional quality analysis results include at least a first analysis result used to characterize the stability of the environmental monitoring data to be analyzed, and a second analysis result used to characterize the correlation of the environmental monitoring data to be analyzed.

[0097] Based on the detailed description in step S104, the first analysis result is the volatility review result. The second analysis result can be the correlation analysis result between the environmental monitoring data to be analyzed and its corresponding standard sample data, or the correlation analysis result between the environmental monitoring data to be analyzed and its corresponding related attribute data, or the result of fusing the two analysis results. If fusing is used, different weights can be pre-assigned to the two correlation analyses, and a weighted summation can be used to obtain the fused value, which is then used as the correlation analysis result.

[0098] In cases where multidimensional quality analysis also includes data anomaly analysis, the results of multidimensional quality analysis also include a third analysis result used to characterize anomalies in the environmental monitoring data to be analyzed.

[0099] The third analysis result may include any one or a combination of several of the following: anomaly detection analysis results, marked anomaly detection analysis results, and transmission efficiency analysis results; or the result obtained by fusing these results. The fusion processing method can be found in the aforementioned example.

[0100] If the data from each dimension is not integrated, from a more detailed perspective, the data analysis results can be seen as a collection of results including volatility review, correlation analysis, anomaly detection analysis, labeled anomaly detection analysis, and transmission efficiency analysis.

[0101] Finally, the multidimensional quality analysis results are transformed into display results that meet the preset output requirements. These display results can be at least one of the following: a multidimensional radar chart, an analysis report, or a preset output text.

[0102] Specifically, the results of the multidimensional quality analysis can be normalized first, and then transformed into display results that meet the preset output requirements.

[0103] Among them, the multi-dimensional radar chart can display the data analysis results of the environmental monitoring data to be analyzed from multiple dimensions. Based on the description in the example above, if it is necessary to display the data analysis results from an overall perspective, a radar chart including two dimensions: data stability and data correlation can be generated, or a radar chart including three dimensions: data stability, data correlation, and data anomalies can be generated.

[0104] For a more detailed presentation, it can be displayed in the form of a five-dimensional radar chart, encompassing five dimensions: volatility review results, correlation analysis results, anomaly detection analysis results, marked anomaly detection analysis results, and transmission efficiency analysis results.

[0105] If a systematic presentation of the entire analysis process is required, an analysis report can be used. An analysis report is a document that can systematically present the entire analysis process and the final conclusions. Specifically, it may include a summary or core conclusions, analysis background and purpose, data source and processing instructions, detailed analysis and findings, and comprehensive conclusions and strategic recommendations.

[0106] Alternatively, the analysis results can be converted into automated reporting systems, data dashboards, or presented in the form of scores, based on the system's preset output text type.

[0107] In summary, the environmental monitoring data analysis method provided in this application acquires the environmental monitoring data to be analyzed and standard sample data with consistent attributes. A density-based clustering algorithm is then used to process these two types of data, resulting in a clustering result that includes at least one environmental monitoring data cluster. If the inflection point in the K-distance graph of the environmental monitoring data to be analyzed does not meet the preset inflection point conditions, a sliding window algorithm is used to determine a reference inflection point to make the clustering result more accurate. When the clustering result meets the conditions, multidimensional quality analysis is then performed on the environmental monitoring data to be analyzed, further improving the efficiency and accuracy of environmental monitoring data analysis. During the multidimensional quality analysis of the environmental monitoring data to be analyzed, at least its data stability and correlation are analyzed, thereby greatly improving the accuracy of the analysis results obtained through multidimensional analysis and further enhancing the accuracy of applying the analysis results to other scenarios, such as data quality evaluation.

[0108] Next, taking the application of the multidimensional quality analysis results obtained by the environmental monitoring data analysis method provided in the embodiments of this application to the specific scenario of data quality evaluation as an example, we will introduce a multidimensional quality evaluation scheme for environmental monitoring data.

[0109] The evaluation scheme includes a two-dimensional quality evaluation scheme based on two dimensions: data stability analysis and data correlation analysis. Specifically, the two-dimensional quality evaluation scheme can include volatility review and correlation analysis.

[0110] This evaluation scheme analyzes environmental monitoring data across three main dimensions: data stability, data correlation, and data anomalies. Building upon this foundation, if a more detailed breakdown of these dimensions is required, it is presented at a more granular level. For example, anomaly analysis can be subdivided into three sub-dimensions: anomaly detection analysis, marked anomaly detection analysis, and transmission efficiency analysis. This example uses the five-dimensional quality analysis results obtained from this granular analysis as the basis for the evaluation. The evaluation scheme generates five-dimensional quality analysis results based on the analysis obtained from these five dimensions: volatility review, correlation analysis, anomaly detection analysis, marked anomaly detection analysis, and transmission efficiency analysis.

[0111] Specifically, a normalized comprehensive evaluation system is constructed based on the results of the five-dimensional quality analysis. The quality analysis results of each dimension are aggregated into an intuitive quality score. This can be achieved by dynamically adjusting the weights and using rich visualization methods. For example, the weights of each dimension can change dynamically. A weighted summation model can be used to calculate the final comprehensive data quality score, thus obtaining the five-dimensional quality evaluation results of the environmental monitoring data.

[0112] There are many options for presenting the results of the five-dimensional quality assessment:

[0113] The five-dimensional quality evaluation results can be displayed using a data quality dashboard, such as a dashboard or scoreboard, to show the core indicators, and different colors can be used to indicate the quality level of environmental monitoring data.

[0114] Five-dimensional radar charts can also be used to display the scores of five dimensions simultaneously (which can be DQI, Data Quality Index). For example, the fuller the graph and the closer it is to the outer circle, the more balanced and higher the data quality is. This multi-dimensional radar chart format can be used to intuitively display the shortcomings of environmental monitoring data.

[0115] The results can also be displayed using line charts, with time as the horizontal axis, plotting the overall DQI and scores for each dimension. This visually shows the trend of environmental monitoring data quality over time, allowing for further assessment of the effectiveness of quality control measures or whether equipment performance is deteriorating. Spatial quality distribution heatmaps can also be used to display the five-dimensional quality assessment results. DQI scores for different monitoring points are displayed on a map as heatmaps or graded color dots, clearly identifying regional data quality issues and facilitating operation and maintenance scheduling.

[0116] The above describes a method for analyzing environmental monitoring data provided in this application. The following describes the apparatus for performing the above method for analyzing environmental monitoring data.

[0117] Please see Figure 3 , Figure 3 A schematic diagram of the structure of the analysis device for environmental monitoring data provided in this application. Figure 3 As shown, the device includes:

[0118] The system comprises a first acquisition unit 10, a cluster analysis unit 20, an inflection point determination unit 30, a quality analysis unit 40, and a second acquisition unit 50; wherein:

[0119] The first acquisition unit 10 is used to acquire the environmental monitoring data to be analyzed and standard sample data with the same attributes as the environmental monitoring data to be analyzed; the standard sample data is data stored in a standard sample database divided according to data attributes.

[0120] Clustering analysis unit 20 is used to call a density-based clustering algorithm to process the environmental monitoring data to be analyzed and the standard sample data, and to determine the K-distance map of the environmental monitoring data to be analyzed;

[0121] The inflection point determination unit 30 is used to determine a reference inflection point using a sliding window algorithm when the inflection point in the K-distance map does not meet the preset inflection point conditions, and to use the K-distance corresponding to the reference inflection point as the neighborhood radius of the environmental monitoring data to be analyzed, so as to obtain the clustering result of the environmental monitoring data to be analyzed; the clustering result includes at least one environmental monitoring data cluster;

[0122] The quality analysis unit 40 is used to perform multidimensional quality analysis on the environmental monitoring data to be analyzed when the clustering results meet preset conditions; the multidimensional quality analysis includes at least data stability analysis and data correlation analysis.

[0123] The second acquisition unit 50 is used to acquire the multidimensional quality analysis results of the environmental monitoring data to be analyzed; the multidimensional quality analysis results include at least: a first analysis result used to characterize the stability of the environmental monitoring data to be analyzed, and a second analysis result used to characterize the correlation of the environmental monitoring data to be analyzed.

[0124] In one embodiment, the K-distance map is a distance map arranged in descending order, and the inflection point determination unit 30 is specifically used for:

[0125] In the absence of obvious inflection points in the K-distance graph, the difference between any two adjacent environmental monitoring data points in the K-distance graph is calculated sequentially to obtain the difference set;

[0126] Calculate the relative rate of change between any two adjacent difference values ​​in the difference set to obtain the set of relative rates of change;

[0127] The first relative change rate in the set of relative change rates that is greater than a preset percentage is determined as the target relative change rate;

[0128] The environmental monitoring data point with the smallest K-distance among the four environmental monitoring data points corresponding to the target relative change rate is taken as the reference inflection point, and the K-distance corresponding to the reference inflection point is taken as the neighborhood radius of the environmental monitoring data to be analyzed.

[0129] In one embodiment, the inflection point determination unit 30 is specifically used for:

[0130] When all relative change rates in the relative change rate set are lower than a preset percentage, the normal distribution of the relative change rate set is determined.

[0131] The candidate set of reference inflection points in the normal distribution is determined according to the preset confidence interval, and the K distance corresponding to the first reference inflection point in the candidate set is selected as the neighborhood radius of the environmental monitoring data to be analyzed.

[0132] In one embodiment, the quality analysis unit 40 is specifically used for:

[0133] Data stability analysis is performed on the environmental monitoring data to be analyzed, and the first analytical results are obtained to characterize the stability of the environmental monitoring data to be analyzed.

[0134] A data correlation analysis was performed on the environmental monitoring data to be analyzed, and a second analysis result was obtained to characterize the correlation of the environmental monitoring data to be analyzed.

[0135] In one embodiment, the quality analysis unit 40 is specifically used for:

[0136] The environmental monitoring data to be analyzed is compared with relevant data to obtain the second analysis result; the relevant data includes the standard sample data corresponding to the environmental monitoring data to be analyzed, as well as the relevant attribute data corresponding to the environmental monitoring data to be analyzed.

[0137] In one embodiment, the above-mentioned environmental monitoring data analysis device further includes a display unit, which is specifically used for:

[0138] The results are generated based on the multidimensional quality analysis results; the results can be at least one of the following: multidimensional radar chart, analysis report, or preset output text.

[0139] This application also provides an environmental monitoring data analysis device in its embodiments. (See reference...) Figure 4 The diagram illustrates a structural schematic of an analysis device suitable for implementing the environmental monitoring data provided in this application. The environmental monitoring data analysis device in this application may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 4 The environmental monitoring data analysis device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0140] like Figure 4 As shown, the environmental monitoring data analysis device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. When the environmental monitoring data analysis device is powered on, the RAM 603 also stores various programs and data required for the operation of the environmental monitoring data analysis device. The processing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0141] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, memory cards, hard drives, etc.; and communication devices 609. Communication device 609 allows the environmental monitoring data analysis equipment to communicate wirelessly or wiredly with other devices to exchange data. Although... Figure 4An analysis apparatus for environmental monitoring data with various devices is shown; however, it should be understood that implementation or possession of all the devices shown is not required. More or fewer devices may be implemented alternatively.

[0142] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the environmental monitoring data analysis methods provided in this application.

[0143] This application also provides a computer storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the environmental monitoring data analysis methods provided in this application.

[0144] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0145] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0146] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0147] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A method of analyzing environmental monitoring data, characterized by, include: Acquire the environmental monitoring data to be analyzed and standard sample data with attributes consistent with the environmental monitoring data to be analyzed; The standard sample data is data stored in a standard sample database divided according to data attributes; the standard sample database is an automatic monitoring database of ecological and environmental pollution sources within the target area. A density-based clustering algorithm is invoked to process the environmental monitoring data to be analyzed and the standard sample data to determine the K-distance map of the environmental monitoring data to be analyzed. The clustering algorithm is used to compare the environmental monitoring data to be analyzed with the standard sample data of its corresponding attributes to determine which sub-standard sample under the standard sample data the data in the environmental monitoring data to be analyzed belongs to. If the inflection point in the K-distance graph does not meet the preset inflection point conditions, a reference inflection point is determined using a sliding window algorithm, and the K-distance corresponding to the reference inflection point is used as the neighborhood radius of the environmental monitoring data to be analyzed, thereby obtaining the clustering result of the environmental monitoring data to be analyzed; the clustering result includes at least one environmental monitoring data cluster; If the clustering results meet the preset conditions, multidimensional quality analysis is performed on the environmental monitoring data to be analyzed. The multidimensional quality analysis includes at least data stability analysis and data correlation analysis; the preset condition is that the clustering result of the environmental monitoring data to be analyzed has a similarity higher than a threshold compared with the clustering result of the standard sample data. Obtain the multidimensional quality analysis results of the environmental monitoring data to be analyzed; The multidimensional quality analysis results include at least: a first analysis result characterizing the stability of the environmental monitoring data to be analyzed, and a second analysis result characterizing the correlation of the environmental monitoring data to be analyzed.

2. The method of claim 1, wherein, The K-distance map is a distance map arranged in descending order. When an inflection point in the K-distance map does not meet a preset inflection point condition, a reference inflection point is determined using a sliding window algorithm, and the K-distance corresponding to the reference inflection point is used as the neighborhood radius of the environmental monitoring data to be analyzed. This includes: In the absence of obvious inflection points in the K-distance graph, the difference between any two adjacent environmental monitoring data points in the K-distance graph is calculated sequentially to obtain the difference set. Calculate the relative rate of change between any two adjacent difference values ​​in the difference set to obtain a set of relative rates of change; The first relative change rate in the set of relative change rates that is greater than a preset percentage is determined as the target relative change rate; The environmental monitoring data point with the smallest K-distance among the four environmental monitoring data points corresponding to the target relative change rate is taken as the reference inflection point, and the K-distance corresponding to the reference inflection point is taken as the neighborhood radius of the environmental monitoring data to be analyzed.

3. The method of claim 2, wherein, When the inflection point in the K-distance graph does not meet the preset inflection point conditions, a reference inflection point is determined using a sliding window algorithm, and the K-distance corresponding to the reference inflection point is used as the neighborhood radius of the environmental monitoring data to be analyzed. This further includes: When all relative change rates in the set of relative change rates are lower than a preset percentage, the set of relative change rates is determined to be normally distributed. The candidate set of reference inflection points in the normal distribution is determined according to the preset confidence interval, and the K distance corresponding to the first reference inflection point in the candidate set of reference inflection points is selected as the neighborhood radius of the environmental monitoring data to be analyzed.

4. The method of claim 1, wherein, Multidimensional quality analysis is performed on the environmental monitoring data to be analyzed, including: Data stability analysis is performed on the environmental monitoring data to be analyzed to obtain a first analysis result characterizing the stability of the environmental monitoring data to be analyzed. A data correlation analysis is performed on the environmental monitoring data to be analyzed to obtain a second analysis result that characterizes the correlation of the environmental monitoring data to be analyzed.

5. The method of claim 4, wherein, The step of performing data correlation analysis on the environmental monitoring data to be analyzed, and obtaining a second analysis result to characterize the correlation of the environmental monitoring data to be analyzed, includes: The second analysis result is obtained by comparing the environmental monitoring data to be analyzed with relevant data; the relevant data includes standard sample data corresponding to the environmental monitoring data to be analyzed, and relevant attribute data corresponding to the environmental monitoring data to be analyzed.

6. The method of claim 1, wherein, Also includes: The display results are generated based on the multidimensional quality analysis results; The displayed results are at least one of the following: multi-dimensional radar chart, analysis report, and preset output text.

7. An analysis device of environmental monitoring data, characterized in that include: The first acquisition unit is used to acquire environmental monitoring data to be analyzed and standard sample data with attributes consistent with the environmental monitoring data to be analyzed. The standard sample data is data stored in a standard sample database divided according to data attributes; the standard sample database is an automatic monitoring database of ecological and environmental pollution sources within the target area. The clustering analysis unit is used to call a density-based clustering algorithm to process the environmental monitoring data to be analyzed and the standard sample data to determine the K-distance map of the environmental monitoring data to be analyzed; the clustering algorithm is used to compare the environmental monitoring data to be analyzed with the standard sample data of its corresponding attributes to determine which sub-standard sample under the standard sample data the data in the environmental monitoring data to be analyzed belongs to. The inflection point determination unit is used to determine a reference inflection point using a sliding window algorithm when the inflection point in the K-distance map does not meet the preset inflection point conditions, and to use the K-distance corresponding to the reference inflection point as the neighborhood radius of the environmental monitoring data to be analyzed, thereby obtaining the clustering result of the environmental monitoring data to be analyzed; the clustering result includes at least one environmental monitoring data cluster; The quality analysis unit is used to perform multidimensional quality analysis on the environmental monitoring data to be analyzed when the clustering results meet preset conditions. The multidimensional quality analysis includes at least data stability analysis and data correlation analysis; the preset condition is that the clustering result of the environmental monitoring data to be analyzed has a similarity higher than a threshold compared with the clustering result of the standard sample data. The second acquisition unit is used to acquire the multidimensional quality analysis results of the environmental monitoring data to be analyzed. The multidimensional quality analysis results include at least: a first analysis result characterizing the stability of the environmental monitoring data to be analyzed, and a second analysis result characterizing the correlation of the environmental monitoring data to be analyzed.

8. An environment monitoring data analysis device, characterized by, comprising at least one processor and a memory connected with the processor, wherein: the memory is configured to store a computer program; the processor is configured to execute the computer program to enable the environment monitoring data analysis device to implement the environment monitoring data analysis method according to any one of claims 1 to 6.

9. A computer program product, characterised in that, computer readable instructions, which, when run on an electronic device, enable the electronic device to implement the environment monitoring data analysis method according to any one of claims 1 to 6.

10. A computer storage medium, characterized in that a storage medium carrying one or more computer programs, which, when executed by an electronic device, enable the electronic device to implement the environment monitoring data analysis method according to any one of claims 1 to 6.

Images