Air quality data quality control method based on time sequence data

Abstract

The invention provides an air quality data quality control method based on time series data, and relates to the technical field of air detection, and the method comprises the steps: obtaining multi-source atmosphere monitoring station data, constructing a spatial topological relation network of monitoring stations, and defining a dynamic hierarchical structure of atmosphere pollutant data under different spatial and temporal scales and a cross-hierarchical association rule of the dynamic hierarchical structure; carrying out anomaly detection and traceability analysis on the obtained monitoring data, and carrying out data correction and filling; performing credibility evaluation and grading on the corrected monitoring data by applying a space-time clustering algorithm and combining a site topological relation and a data association rule, and generating a data credibility grading result of a space-time dimension; when the data credibility is detected to be abnormal or the potential failure risk is detected, a graph neural network model integrated with atmospheric diffusion physical constraints is adopted; and generating a visual atmosphere data quality evaluation and control decision report, and automatically triggering an instrument calibration, maintenance or data review instruction for a specific site or region according to the risk level.

Description

Technical Field

[0001] The present invention relates to the field of air detection technology, and in particular to an air quality data quality control method based on time series data. Background Art

[0002] With the acceleration of industrialization and urbanization, air quality issues have become increasingly prominent. Urban multi-scale monitoring networks have been gradually established and continuously improved to monitor the concentration of pollutants in the atmosphere in real time. However, these monitoring networks face many data quality issues.

[0003] Traditional air quality monitoring relies primarily on fixed national air quality monitoring stations, which are sparsely distributed and fail to fully reflect air quality conditions in complex urban environments. In recent years, the introduction of micro-stations and mobile monitoring devices has given the monitoring network a multi-scale nature. However, different levels of monitoring equipment exhibit varying data accuracy, temporal resolution, and spatial coverage. For example, micro-station data may be significantly affected by the surrounding microenvironment and lack stability compared to national air quality monitoring stations. Mobile monitoring device data is primarily used for specific scenarios, such as traffic pollution monitoring. Effectively integrating this multi-source, heterogeneous data and ensuring its quality poses a challenge. Existing data processing methods often focus on single statistical analysis or simple calibration, ignoring the impact of atmospheric physical and chemical processes on changes in pollutant concentrations. This results in inaccurate identification of anomalous data and a lack of physical basis for data correction. Furthermore, spatiotemporal clustering analysis has been rarely used in air quality data processing, making it difficult to identify the temporal and spatial patterns of variation inherent in the data and effectively grading its credibility. Furthermore, existing technologies lack forward-looking predictions of data failure risks, often processing data after issues have already occurred, making it difficult to proactively prevent potential data failures. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the present invention provides an air quality data quality control method based on time series data.

[0005] An air quality data quality control method based on time series data includes the following steps: Preferably, the specific working steps of S1 are as follows: Establishing a spatial correlation matrix of monitoring sites ; The monitoring sites are divided into three data levels.

[0006] Preferably, the specific working steps of S2 are as follows: Multi-source atmospheric monitoring station data specifically includes Data, ozone, nitrogen dioxide and nitrogen oxides, using the aerosol hygroscopic growth model to analyze the original The data is corrected to obtain the corrected data; When the ratio of nitrogen dioxide to nitrogen oxides is greater than the first threshold, and When the ratio of the carbon monoxide to the carbon monoxide is less than the second threshold, it is determined that the sensor has failed; If the hourly concentration of ozone is greater than the third threshold and the concentration of nitrogen dioxide is less than the fourth threshold, it is determined to be a transmission pollution event.

[0007] Preferably, the specific working steps of S3 are as follows: Use dynamic time warping algorithm to perform time series clustering: If it shows bimodal characteristics of morning peak and evening peak, and the peak ratio is greater than 1.5, it is a traffic source mode; If it is characterized by continuous flat-peak fluctuations with a variance of less than 15% and a matching degree with the wind direction vector of less than 30°, it is an industrial emission mode; If it meets both spatiotemporal clustering and chemical mechanism, it is Level 1; When only a single clustering feature of time or space is met, it is classified as Level 2; Cases that do not meet any of the above characteristics are judged as Level 3.

[0008] Preferably, the step S3 further comprises the following steps: First determine within a certain time interval The change in concentration is obtained by subtracting the concentration at the start time from the concentration at the end time; Clarify the time length corresponding to the above concentration changes, that is, the length of time between the start time and the end time; calculate The rate of change of concentration in the vertical direction, that is, the change in concentration per unit height; Divide the concentration change by the product of the time interval and the vertical concentration gradient to obtain The actual sedimentation rate; Compare the calculated actual sedimentation rate with the theoretical sedimentation rate. If the deviation between the actual value and the theoretical value exceeds 30%, the data will be downgraded from Level 2 to Level 3. Measure the nitrogen dioxide concentration and ozone concentration in the current environment respectively, divide the nitrogen dioxide concentration by the ozone concentration to obtain their concentration ratio; Check whether the calculated concentration ratio is greater than 20, and measure the solar radiation intensity at this time to see if it is less than 200 watts per square meter; If the ratio of nitrogen dioxide to ozone concentrations is greater than 20 and the solar radiation intensity is less than 200 watts per square meter, the data is considered abnormal.

[0009] Preferably, said S4 comprises the following steps: Build a physically constrained graph neural network model. The input layer includes the site topology matrix and 72 hours of multi-parameter time series data. The site topology matrix reflects the spatial relationships between sites, while the multi-parameter time series data contains observations of multiple parameters at different times. Together, these form the basic information for the model input. Hidden layer: The model's hidden layer consists of a three-layer graph convolutional network coupled with LSTM units. The graph convolutional network mines spatial correlation information between sites, while the LSTM unit captures dynamic changes in time series. Data is processed sequentially through these three layers to gradually extract deep features. Output layer: After processing by the hidden layer, the model outputs the probability of data failure in the next 24 hours. This probability is between 0 and 1. The closer the probability is to 1, the greater the possibility of data failure; the closer it is to 0, the smaller the possibility of data failure. First, find the difference between the model's predicted value and the true value, then square this difference and calculate the average of all samples to get the first part of the loss; Considering each term in the atmospheric diffusion equation, we calculate the rate of change of pollutant concentration over time, the product of the wind speed vector and the concentration gradient, and the combination of the turbulent diffusion coefficient and the concentration gradient. We combine these terms and then find the deviation from the actual atmospheric diffusion situation. We square the deviations and calculate the average value to obtain the second part of the loss. The above two parts of loss are weighted and summed, where the weight coefficient is 0.6, and the value of the entire loss function is finally obtained.

[0010] Preferably, the S4 further comprises the following steps: Different countermeasures are taken based on the data failure probability calculated by S4. If the failure probability is greater than 0.7, the device calibration instruction is immediately triggered; If the failure probability is between 0.4 and 0.7, mark the data as Level 2 and shorten the sampling interval; When the failure probability is less than 0.4, it is sufficient to maintain regular monitoring frequency and operations.

[0011] Preferably, the S5 comprises the following steps: Generate a three-dimensional data quality heat map, mark the location of each site on the map, and use color markers to indicate different levels, with green representing Level 1, yellow representing Level 2, and red representing Level 3; Display the changing curve of data credibility in the past 24 hours in a rolling manner; Construct a correlation matrix between pollutants and meteorological parameters.

[0012] Preferably, it also includes establishing a quality control knowledge base update mechanism to automatically expand the S3 clustering pattern category when a new pollution source type is added.

[0013] Preferably, the method further includes analyzing which pollution sources mainly cause the abnormal data based on the clustering pattern proportion in step S3, as follows: To calculate maintenance priority, failure probability is used as a factor with a weight of 0.7, and centrality is used as a factor with a weight of 0.3. Find the maximum centrality value among all sites, multiply the failure probability by 0.7, divide the centrality by the maximum centrality, and then multiply by 0.3. Finally, add the two to get the maintenance priority; For each site pair, the corresponding element in the spatial correlation matrix is ​​multiplied by the absolute value of the concentration difference between the two sites to obtain the contribution value of the site pair in regional transmission. The contribution values ​​of all relevant site pairs are accumulated to obtain the total regional transmission contribution.

[0014] The present invention provides an air quality data quality control method based on time series data. It has the following beneficial effects: By constructing the spatial topological relationship of monitoring sites, the spatial correlation and influencing factors between different sites can be fully considered, and the spatiotemporal correlation and consistency of the data can be improved; anomaly detection and data correction based on atmospheric physical and chemical mechanisms can effectively eliminate outliers and errors in the data and improve data accuracy; the spatiotemporal clustering algorithm realizes data credibility grading, which helps to quickly identify the quality of data and provide a more targeted basis for subsequent data analysis and decision-making; the physically constrained graph neural network predicts the risk of data failure, which can provide early warning of potential data problems, take maintenance measures in time, and reduce the impact of data failure on air quality monitoring; the final generated visual quality control decision report and maintenance instructions can intuitively present the data quality status and quality control results, making it easier for managers to make decisions quickly and implement corresponding maintenance operations, thereby effectively improving the overall quality of air quality data in the city's multi-scale monitoring network and providing more powerful data support for urban air pollution prevention and control and environmental management. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a system flow chart of the present invention. DETAILED DESCRIPTION

[0016] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0017] like Figure 1 As shown, the present invention proposes an air quality data quality control method based on time series data, comprising the following steps: S1. Acquire data from multiple atmospheric monitoring stations, construct a spatial topological relationship network of monitoring stations, and define the dynamic hierarchical structure of atmospheric pollutant data at different spatiotemporal scales and its cross-hierarchical association rules; S2. Perform anomaly detection and traceability analysis on the acquired monitoring data, and perform data correction and filling; S3. Apply a spatiotemporal clustering algorithm, combining site topology relationships with data association rules, to assess and grade the credibility of the corrected monitoring data, generating a spatiotemporal data credibility grading result. S4. When data credibility anomalies or potential failure risks are detected, a graph neural network model incorporating atmospheric diffusion physics constraints is used to predict the future failure probability and risk level of data at the monitoring site or region, thereby performing data failure risk prediction. S5. Based on the data credibility grading results of S3 and the data failure risk prediction of S4, a visual atmospheric data quality assessment and control decision report is generated, and instrument calibration, maintenance or data review instructions for specific sites or areas are automatically triggered according to the risk level.

[0018] It should be noted that a time-series data-based air quality data quality control method is provided for application in complex urban multi-scale monitoring networks. Its core is to construct spatial topological relationships among monitoring sites, define dynamic data hierarchies and association rules, perform anomaly detection and data correction based on atmospheric physical and chemical mechanisms, implement data credibility grading using spatiotemporal clustering algorithms, and employ physically constrained graph neural networks to predict data failure risks. Finally, a visual quality control decision report is generated and maintenance instructions are triggered to comprehensively improve the accuracy and reliability of air quality data. This method targets data from different types of monitoring sites in an urban multi-scale monitoring network and uses a variety of technologies and algorithms to optimize the entire process from data collection to processing, analysis, and quality control.

[0019] It aims to address issues such as uneven air quality data quality, difficulty in predicting data failures, and limited quality control methods in urban multi-scale monitoring networks, providing more accurate and reliable data support for urban air quality management and decision-making. Existing technologies typically focus on data processing at a single monitoring site or employ simple statistical methods for data quality control, lacking systematic consideration of multi-scale monitoring networks and comprehensive quality control methods. This design, based on the spatial topological relationships of monitoring sites, integrates advanced technologies such as atmospheric physicochemical mechanisms, spatiotemporal clustering algorithms, and physically constrained graph neural networks to form a complete air quality data quality control system capable of more comprehensive and in-depth processing and analysis of air quality data. By constructing the spatial topological relationship of monitoring sites, the spatial correlation and influencing factors between different sites can be fully considered, and the spatiotemporal correlation and consistency of the data can be improved; anomaly detection and data correction based on atmospheric physical and chemical mechanisms can effectively eliminate outliers and errors in the data and improve data accuracy; the spatiotemporal clustering algorithm realizes data credibility grading, which helps to quickly identify the quality of data and provide a more targeted basis for subsequent data analysis and decision-making; the physically constrained graph neural network predicts the risk of data failure, which can provide early warning of potential data problems, take maintenance measures in time, and reduce the impact of data failure on air quality monitoring; the final generated visual quality control decision report and maintenance instructions can intuitively present the data quality status and quality control results, making it easier for managers to make decisions quickly and implement corresponding maintenance operations, thereby effectively improving the overall quality of air quality data in the city's multi-scale monitoring network and providing more powerful data support for urban air pollution prevention and control and environmental management.

[0020] As an optional embodiment: the specific working steps of S1 are as follows: Establishing a spatial correlation matrix of monitoring sites ; It should be noted that: ; is the Euclidean distance, h is the altitude, H is 100m and is the boundary layer size parameter. is the prevailing wind direction angle, 0.3 is the wind farm weight; The monitoring stations are divided into three data layers. It should be noted that the national monitoring station (L1) is used as the baseline layer, the micro station (L2) needs to be compared and calibrated with the L1 data every 6 hours, and the mobile monitoring equipment (L3) is only used for traffic pollution model verification; It should be noted that, based on claim 1, this claim 2 further clarifies the specific method for constructing the spatial topological relationship of the monitoring stations in step S1. By establishing a spatial correlation matrix of the monitoring stations, taking into account factors such as Euclidean distance, altitude, boundary layer scale parameters, prevailing wind direction angle, and wind field weight, the degree of correlation between the stations is calculated, thereby constructing a correlation matrix that can reflect the spatial topological relationship of the stations. At the same time, three levels of data are divided, including national control stations (L1), micro stations (L2), and mobile monitoring equipment (L3), and clear provisions are made for data comparison, correction, and application of stations at different levels; It should be noted that by establishing a spatial correlation matrix of monitoring sites, the spatial relationships between different sites in the urban multi-scale monitoring network can be quantitatively described, providing an important spatial information foundation for subsequent data processing and analysis. The division of three-level data hierarchies helps to classify and manage data from monitoring sites of different types, different precisions, and different uses, and to process them in a targeted manner, thereby improving the efficiency and accuracy of data quality control. Further refine and clarify the method of constructing the spatial topological relationship of monitoring sites and the specific plan for data hierarchical division, so as to provide more accurate spatial information and data basis for subsequent steps such as anomaly detection, data correction, credibility grading and failure risk prediction, and ensure the effectiveness and scientific nature of the entire quality control method.

[0021] When constructing the spatial topological relationship of monitoring stations, existing technologies often only consider a single distance factor or a simple spatial position relationship, and lack comprehensive consideration of multiple factors such as terrain and wind field, resulting in the constructed topological relationship being inaccurate and incomplete. This design introduces multiple factors such as Euclidean distance, altitude, boundary layer scale parameters, prevailing wind direction angle, and wind field weight to construct a spatial correlation matrix of monitoring stations that is more in line with actual conditions and can more realistically reflect the spatial mutual influence relationship between stations. At the same time, the division of data levels is also more detailed and reasonable, fully considering the characteristics and functions of different types of monitoring stations, and providing strong support for subsequent differentiated data processing and quality control.

[0022] It more accurately reflects the spatial topological relationship between monitoring sites, provides a more reliable spatial information basis for subsequent data processing and analysis, is conducive to improving the accuracy of anomaly detection and data correction, enhances the rationality of data credibility classification, and improves the reliability of data failure risk prediction, thereby further improving the performance and effect of the entire air quality data quality control method, making the quality control results more in line with the actual air quality conditions and the operating characteristics of the monitoring network, and providing stronger guarantees for the refined management of urban air quality and scientific decision-making.

[0023] As an optional embodiment: the specific working steps of S2 are as follows: Multi-source atmospheric monitoring station data specifically includes Data, ozone, nitrogen dioxide and nitrogen oxides, using the aerosol hygroscopic growth model to analyze the original The data is corrected to obtain the corrected Data; It should be noted that the correction formula is: ; Among them, α is a coefficient calibrated by the aerosol component; RH represents relative humidity, and RHref is the reference relative humidity, which is 40%. When the relative humidity RH exceeds 80%, liquid water content correction needs to be enabled.

[0024] When the ratio of nitrogen dioxide to nitrogen oxides is greater than a first threshold, and the ratio of PM2.5 to carbon monoxide is less than a second threshold, it is determined that the sensor has failed; If the hourly concentration of ozone is greater than the third threshold and the concentration of nitrogen dioxide is less than the fourth threshold, it is determined to be a transmission pollution event. It should be noted that in this embodiment, all thresholds are as follows: ) and nitrogen oxides ( ) of the ratio ( ) is greater than 0.8, and With carbon monoxide ( ) of the ratio ( ) is less than 0.02 micrograms per cubic meter When the sensor fails, it is determined to be faulty.

[0025] If ozone ( ) hourly concentration is greater than 200ppb, while nitrogen dioxide ( ) is less than 10ppb, it is determined to be a transmission pollution event; It should be noted that, in the method according to claim 1, step S2 is described in detail in claim 3. In terms of anomaly detection and data correction, on the one hand, the aerosol hygroscopic growth model is applied to On the one hand, the data is corrected, and on the other hand, anomaly detection is performed based on the chemical coupling relationship of pollutants. Specifically, the aerosol hygroscopic growth model is corrected. The data is calculated by considering the effect of relative humidity (RH) on the hygroscopic growth of aerosol particles and using the aerosol component calibration parameter α and the set reference relative humidity RHref to calculate the corrected The data is collected and liquid water content correction is enabled when the relative humidity exceeds 80% to eliminate the effect of humidity on the Interference with measurement results. When detecting anomalies based on the chemical coupling relationship of pollutants, it is stipulated that The ratio is greater than 0.8 and The ratio is less than When the sensor fails, Hourly concentration exceeds 200ppb and When the concentration is less than 10 ppb, it is determined to be a transmission pollution event.

[0026] α: A parameter calibrated by aerosol components. The determination of aerosol components can be obtained through laboratory analysis or on-site sampling and measurement, combined with relevant physical and chemical models. Different components of aerosol contribute differently to hygroscopic growth. By calibrating the α parameter, the hygroscopic characteristics of aerosols can be accurately reflected, thus The effect of humidity is taken into account in data correction. For example, for aerosols with different chemical compositions, such as sulfate, nitrate, and organic aerosols, their hygroscopic growth factors and corresponding α values ​​are different.

[0027] The reference relative humidity is set at 40%, which is a standard humidity condition determined based on long-term environmental monitoring experience and research on the hygroscopic growth characteristics of aerosols. At this humidity, the hygroscopic growth of aerosols is relatively stable. The reference humidity for data correction is used to compare and calculate the moisture absorption growth effect under different humidity conditions.

[0028] 、 、 、CO、 Concentration data: These pollutant concentration data are monitored and collected in real time by the corresponding sensors at the monitoring sites. Through the signal output and data conversion processing of the monitoring instruments, the concentration values ​​of different pollutants at the monitoring time point can be obtained for subsequent chemical coupling relationship analysis and anomaly detection.

[0029] This claim claims the application of aerosol hygroscopic growth model and pollutant chemical coupling relationship to The data is corrected and abnormal conditions are detected to improve the accuracy and reliability of the data and ensure that the monitoring data can truly reflect the air quality conditions.

[0030] Aerosol hygroscopic growth The influence of humidity and chemical reactions on the measurement results and the chemical coupling relationship followed by the chemical reactions and interactions between pollutants are considered, and corresponding data correction and anomaly detection methods are proposed to solve the problems of air quality data deviation and inaccurate anomaly judgment caused by humidity factors and chemical reactions in the existing technology, thereby improving the quality control level of air quality data.

[0031] Existing technology in When correcting data, usually only the measurement error of the instrument itself or the influence of simple environmental factors is considered. However, this design can more accurately correct the error caused by humidity changes by introducing the aerosol hygroscopic growth model. Data bias correction methods are more consistent with the physical and chemical properties of aerosols. In terms of anomaly detection, existing technologies mostly rely on thresholds for single pollutant concentrations or simple statistical rules. However, this design, based on the chemical coupling relationship between pollutants, comprehensively considers the interactions and reaction patterns between multiple pollutants, establishing more scientific and reasonable anomaly detection rules. This can more accurately identify anomalies such as sensor failure and transmission pollution events, effectively improving the accuracy and reliability of anomaly detection.

[0032] Calibration using the aerosol hygroscopic growth model Data, which can eliminate humidity Measurement interference, improve Data accuracy ensures that monitoring data more accurately reflects actual air quality conditions, providing more reliable data support for the prevention and control of atmospheric particulate matter pollution. Anomaly detection methods based on the chemical coupling relationship between pollutants can promptly and accurately detect anomalies such as sensor failures and transmission pollution events, avoiding misjudgments and incorrect decisions caused by data anomalies. This ensures the normal operation of the air quality monitoring system and the reliability of data, providing stronger support for urban air quality management and environmental regulation.

[0033] As an optional embodiment: the specific working steps of S3 are as follows: Use dynamic time warping algorithm to perform time series clustering: If it shows the bimodal characteristics of morning peak and evening peak, and the peak ratio is greater than 1.5, it is a traffic source mode; it should be noted that the morning peak is 07:00-09:00; the evening peak (17:00-19:00) If it is characterized by continuous flat-peak fluctuations with a variance of less than 15% and a matching degree with the wind direction vector of less than 30°, it is an industrial emission mode; At the same time, spatial DBSCAN clustering is also combined to identify regional pollution clusters. The cluster radius is equal to the difference between the area under the square root of π and the number of sites. If it meets both spatiotemporal clustering and chemical mechanism, it is Level 1; When only a single clustering feature of time or space is met, it is classified as Level 2; Cases that do not meet any of the above characteristics are judged as Level 3.

[0034] It should be noted that, based on claim 1, this claim 4 provides a detailed description of step S3. First, a dynamic time warping (DTW) algorithm is used for time series clustering to identify three typical pollution patterns: a traffic source pattern, an industrial emission pattern, and a regional transmission pattern. Characteristic descriptions of each pattern are provided. For example, the traffic source pattern exhibits bimodal characteristics with morning and evening peaks, with a peak ratio greater than 1.5; the industrial emission pattern exhibits continuous flat-peak fluctuations, with a variance less than 15%, and a matching degree with the wind direction vector less than 30°. At the same time, spatial DBSCAN clustering is combined to identify regional pollution clusters. Sites with similar spatiotemporal characteristics are clustered by calculating the cluster radius to form regional pollution clusters. Finally, a credibility grading rule is formulated, categorizing the data into three levels: Level 1, Level 2, and Level 3, depending on whether it conforms to both spatiotemporal clustering and chemical mechanisms, conforms to only a single temporal or spatial clustering feature, or conforms to none of the features.

[0035] The area of ​​a city can be calculated based on the geographical scope and boundaries of the monitoring area using Geographic Information System (GIS) software. For example, the area of ​​a city can be calculated based on the city's administrative boundaries or electronic map data of the actual monitoring area, which can be used to calculate the cluster radius.

[0036] It refers to the number of monitoring sites involved in the cluster analysis. Before performing spatial DBSCAN clustering, it is determined by counting the number of all valid sites in the monitoring network. It is used to calculate the cluster radius to determine which sites belong to the same regional pollution cluster in space.

[0037] The wind direction is obtained by real-time monitoring by the wind direction sensor at the monitoring site. The angle value is usually used to express the wind direction. It is used to analyze the transmission path and diffusion direction of pollutants. In the identification of industrial emission patterns, it is combined with the fluctuation of pollutant concentration at the site to determine whether there is an impact from industrial emission sources.

[0038] By combining temporal and spatial clustering, the credibility of air quality data can be graded. This can comprehensively consider the temporal and spatial variation characteristics of the data and the chemical mechanisms of pollutants, thereby more comprehensively and accurately assessing the reliability of the data and providing more valuable reference for subsequent data analysis and decision-making. The goal is to identify different types of pollution patterns and regional pollution clusters through effective spatiotemporal clustering methods, and to grade the credibility of data based on its spatiotemporal clustering characteristics and the degree of conformity with chemical mechanisms, so as to better understand the quality status and pollution sources of air quality data and provide a scientific basis for refined control and precise governance of air pollution. Compared to existing techniques that use only temporal or spatial clustering, this design combines the Dynamic Time Warping (DTW) algorithm with spatial DBSCAN clustering to more accurately capture the complex temporal and spatial variations in air quality data, effectively identifying different types of pollution patterns. Furthermore, clustering results are combined with pollutant chemical mechanisms for credibility grading, further enhancing the scientific and rationality of data quality assessments and providing a more reliable basis for subsequent atmospheric pollution analysis and decision-making. Using the DTW algorithm for time series clustering can effectively identify different types of pollution patterns, such as traffic sources, industrial emission sources, and regional transmission sources. This helps to gain a deeper understanding of the emission patterns and source characteristics of pollutants, providing a basis for developing targeted pollution prevention and control measures. Combining spatial DBSCAN clustering to identify regional pollution clusters can intuitively display the spatial distribution and transmission range of pollutants within a region, facilitating the identification of high-incidence areas of pollution and key prevention and control targets. The application of credibility grading rules can clearly delineate data of different quality levels, enabling managers to quickly understand the reliability of the data and adopt appropriate processing and analysis strategies for different levels of data, improving the scientific nature and accuracy of decision-making, thereby effectively improving air quality monitoring and management and promoting improvements in the atmospheric environment.

[0039] As an optional embodiment: S3 further includes the following steps: First determine within a certain time interval The change in concentration is obtained by subtracting the concentration at the start time from the concentration at the end time; Clarify the time length corresponding to the above concentration changes, that is, the length of time between the start time and the end time; calculate The rate of change of concentration in the vertical direction, that is, the change in concentration per unit height; Divide the concentration change by the product of the time interval and the vertical concentration gradient to obtain The actual sedimentation rate; Compare the calculated actual sedimentation rate with the theoretical sedimentation rate. If the deviation between the actual value and the theoretical value exceeds 30%, the data will be downgraded from Level 2 to Level 3. Measure the nitrogen dioxide concentration and ozone concentration in the current environment respectively, divide the nitrogen dioxide concentration by the ozone concentration to obtain their concentration ratio; Check whether the calculated concentration ratio is greater than 20, and measure the solar radiation intensity at this time to see if it is less than 200 watts per square meter; If the ratio of nitrogen dioxide to ozone concentrations is greater than 20 and the solar radiation intensity is less than 200 watts per square meter, the data is considered abnormal.

[0040] It should be noted that the processing of Level 2 data has been further supplemented. For data classified as Level 2, atmospheric chemical transport verification is started. On the one hand, the calculation The actual sedimentation rate is compared with the theoretical value. If the deviation from the theoretical value exceeds 30%, the data is downgraded to Level 3. Photochemical equilibrium, when The ratio is greater than 20 and the solar radiation is less than When , the data is judged to be abnormal; ΔC / Δt: indicates The rate of change of concentration over time can be determined by the time at which the monitoring station Calculated from concentration monitoring data to reflect The trend and speed of change over time are calculated One of the key parameters of actual sedimentation rate.

[0041] Δz: is the height interval in the vertical direction, in the calculation The deposition rate is usually determined based on the vertical detection level or sampling height range of the monitoring site, and is used to calculate the vertical deposition velocity of PM2.5 in combination with the concentration change rate, and then evaluate whether its actual deposition rate conforms to the laws of atmospheric chemical transport.

[0042] Wind speed vector u: It is obtained by real-time monitoring of the wind speed sensor at the monitoring station. It includes the magnitude and direction of wind speed. It is an important parameter in the atmospheric diffusion equation. It is used to describe the transmission speed and direction of pollutants in the atmosphere. It plays a key role in the diffusion and dilution of pollutants. Actual sedimentation rates and verification The influence of wind speed vector must be considered when calculating photochemical balance.

[0043] Solar radiation: The solar radiation intensity received in the atmosphere is monitored in real time by a solar radiation monitor. The intensity of solar radiation directly affects the rate and extent of photochemical reactions, especially in In the photochemical equilibrium system, solar radiation is The generation and The conversion of photochemical energy provides energy, the size of which is important for judging the state of photochemical equilibrium.

[0044] For Level 2 data that do not fully conform to the spatiotemporal clustering characteristics and chemical mechanisms, further atmospheric chemical transport verification will be conducted to more rigorously evaluate the reliability of the data, ensuring that only data that conform to the laws of atmospheric chemical transport can obtain a higher credibility level, thereby improving the strictness and accuracy of overall data quality control.

[0045] Further improve the data credibility grading system, conduct more in-depth verification and analysis of Level 2 data at the intermediate level to more accurately judge its quality status, avoid decision-making errors caused by data misjudgment, and improve the accuracy and reliability of air quality data quality control.

[0046] After the existing technology has graded the credibility of air quality data, it often lacks further verification and refinement of the intermediate level data, which may lead to the misuse of some data with questionable quality. This design introduces the atmospheric chemical transport verification method to conduct a secondary verification of Level 2 data, combined with Sedimentation rate and Key atmospheric chemical processes such as photochemical balance can further identify the reliability of data, effectively make up for the shortcomings of existing technologies, and make data credibility classification more rigorous and scientific.

[0047] Performing atmospheric chemical transmission verification on Level 2 data can more accurately identify potential anomalies in the data, further improving the accuracy and reliability of data credibility grading. This ensures that only data that conforms to atmospheric chemical transmission laws is classified as high-confidence, providing higher-quality data support for subsequent air quality analysis and decision-making. Furthermore, by downgrading data that does not meet requirements, potential problems in the monitoring system can be promptly identified, providing a basis for equipment maintenance and data quality improvement. This helps to improve the operational quality and data quality of the entire air quality monitoring network, and better serve urban air quality management.

[0048] As an optional embodiment: S4 includes the following steps: Build a physically constrained graph neural network model. The input layer includes the site topology matrix and 72 hours of multi-parameter time series data. The site topology matrix reflects the spatial relationships between sites, while the multi-parameter time series data contains observations of multiple parameters at different times. Together, these form the basic information for the model input. Hidden layer: The model's hidden layer consists of a three-layer graph convolutional network coupled with LSTM units. The graph convolutional network mines spatial correlation information between sites, while the LSTM unit captures dynamic changes in time series. Data is processed sequentially through these three layers to gradually extract deep features. Output layer: After processing by the hidden layer, the model outputs the probability of data failure in the next 24 hours. This probability is between 0 and 1. The closer the probability is to 1, the greater the possibility of data failure; the closer it is to 0, the smaller the possibility of data failure. First, find the difference between the model's predicted value and the true value, then square this difference and calculate the average of all samples to get the first part of the loss; Considering each term in the atmospheric diffusion equation, we calculate the rate of change of pollutant concentration over time, the product of the wind speed vector and the concentration gradient, and the combination of the turbulent diffusion coefficient and the concentration gradient. We combine these terms and then find the deviation from the actual atmospheric diffusion situation. We square the deviations and calculate the average value to obtain the second part of the loss. The weighted sum of the two losses above, with a weight coefficient of 0.6, ultimately yields the value of the entire loss function. It should be noted that this loss function measures the difference between the model's predictions and the actual situation. During model training, by optimizing this loss function, the model can be made more accurate in predicting the probability of data failure while also meeting the physical laws of atmospheric diffusion. It should be noted that the constructed physically constrained graph neural network model consists of an input layer, hidden layers, and an output layer. The input layer receives the site topology matrix G and 72 hours of multi-parameter time series data, which includes continuously monitored values ​​of various air quality-related parameters. The hidden layer consists of a three-layer graph convolutional network (GCN) coupled with LSTM units. The GCN effectively handles data associations and topological relationships between sites, while the LSTM unit excels at capturing long-term dependencies in time series data. The combination of the two fully exploits the complex characteristics of spatiotemporal data. The output layer predicts the probability of data failure Pfail for the next 24 hours, which ranges from [0 to 1]. Constraints of the atmospheric diffusion equation are embedded in the loss function. By adding a regularization term, the physical laws of the atmospheric diffusion process are incorporated into the model training process. λ is the regularization coefficient, and the wind speed vector u and the turbulent diffusion coefficient κ are key parameters in the atmospheric diffusion equation.

[0049] Parameter explanation: Regularization coefficient λ: The value is 0.6. It is a weight coefficient determined through a large number of experiments and model optimization. It is used to balance the relative importance between the data fitting term and the atmospheric diffusion equation constraint term in the loss function. During the model training process, by adjusting the value of λ, the model can fit the historical data while also meeting the physical laws described by the atmospheric diffusion equation, avoiding the occurrence of model overfitting or underfitting, and improving the accuracy and reliability of model predictions.

[0050] The turbulent diffusion coefficient, k, can be measured experimentally or estimated based on atmospheric boundary layer theory. It reflects the ability of turbulent motion in the atmosphere to disperse and dilute pollutants. The value of k varies under different meteorological and environmental conditions. In this design, it is incorporated into the loss function as a key parameter in the atmospheric diffusion equation to ensure that the model's predictions align with the actual physical processes of atmospheric diffusion.

[0051] Explanation: By building a physically constrained graph neural network model, combining site topology and multi-parameter time series data, leveraging the synergy of GCN and LSTM units to mine spatiotemporal data features, and introducing the atmospheric diffusion equation to constrain the optimization model training process, we can accurately predict the probability of data failure in the next 24 hours. Existing technologies for predicting air quality data failure risk typically rely solely on simple statistical models or machine learning algorithms, lacking consideration of physical laws. This results in low accuracy and reliability. This design introduces a physically constrained graph neural network model, embedding physical laws such as the atmospheric diffusion equation. This allows the model to simultaneously learn the spatiotemporal and physical characteristics of the data, thereby better capturing the changing patterns of data failure risk and improving the accuracy and scientific nature of predictions.

[0052] The physically constrained graph neural network model can make full use of the topological relationships and multi-parameter time series data of monitoring sites, explore the deep-level features and laws in the data, and combine the constraints of the atmospheric diffusion equation to ensure that the prediction results are consistent with physical reality, thereby more accurately predicting the probability of data failure in the next 24 hours, providing sufficient time and basis for preventive maintenance of monitoring equipment, effectively reducing the risk of data failure, ensuring the continuity and reliability of air quality monitoring data, and improving the operating efficiency and data quality assurance level of the entire monitoring system. It helps to promptly discover potential data problems and take corresponding solutions, improve the stability and reliability of urban air quality monitoring, and provide more timely and accurate data support for environmental management departments.

[0053] As an optional embodiment: S4 further includes the following steps: Different countermeasures are taken based on the data failure probability calculated by S4. If the failure probability is greater than 0.7, the device calibration instruction is immediately triggered. This indicates that the risk of data failure is very high. In this case, the device calibration instruction should be triggered immediately and the relevant equipment should be calibrated to ensure data accuracy. If the failure probability is between 0.4 and 0.7, the data is marked as Level 2 and the sampling interval is shortened; this indicates that there is a certain risk, but it has not yet reached the most serious level. At this time, the data is marked as Level 2 and the sampling interval is shortened, that is, the sampling frequency is increased to monitor the data status more closely.

[0054] When the failure probability is less than 0.4, maintain normal monitoring frequency and operations. This means that the possibility of data failure is low, so maintain normal monitoring frequency and operations.

[0055] When the proportion of failed sites in an area exceeds 30%, this indicates that there may be widespread problems in the area, and network-level diagnosis needs to be initiated to thoroughly identify the causes. At the same time, the topology matrix needs to be reconstructed, that is, the connection relationship and structure between sites need to be readjusted and optimized to adapt to the current monitoring situation and needs, thereby improving the reliability and effectiveness of the entire monitoring network.

[0056] This dynamic risk response mechanism adjusts the monitoring system's operating strategies and maintenance measures in real time based on the predicted probability of data failure, minimizing the impact of data failure risks on air quality monitoring and ensuring the reliability and effectiveness of monitoring data.

[0057] Establish a response mechanism that can dynamically adjust monitoring and maintenance strategies according to the degree of data failure risk, respond promptly to data failures of different risk levels, ensure the stable operation of the monitoring system and the continuous reliability of data quality, and avoid environmental management decision-making errors and public health risks caused by data failure.

[0058] Existing technologies for data quality control typically use fixed maintenance cycles or simple threshold judgment methods to address data failure. These methods lack the ability to dynamically adjust based on the risk of data failure, potentially leading to issues such as untimely or excessive maintenance. The dynamic risk response mechanism proposed in this design establishes different levels of response strategies based on continuous predictions of data failure probabilities. This enables refined management of monitoring and maintenance measures, improving the flexibility and effectiveness of addressing data failure risks.

[0059] Through a dynamic risk response mechanism, data failure risks can be promptly and accurately responded to, and appropriate maintenance measures can be taken based on the level of risk. This not only prevents the escalation of data failures due to untimely maintenance, but also rationally utilizes maintenance resources and reduces unnecessary maintenance work. When the proportion of regional failed sites is high, network-level diagnostics and topology matrix reconstruction are initiated to comprehensively investigate and resolve potential systemic issues in the monitoring network, safeguarding the stability and reliability of the entire monitoring network and ensuring continuous and accurate monitoring of air quality data. This provides stable and reliable data support for urban air quality management and decision-making, while also reducing the operating costs and maintenance workload of the monitoring system and improving operational efficiency and economic benefits.

[0060] As an optional embodiment: S5 includes the following steps: Generate a three-dimensional data quality heat map, mark the location of each site on the map, and use color markers to indicate different levels, with green representing Level 1, yellow representing Level 2, and red representing Level 3; The data credibility change curve in the past 24 hours is displayed in a rolling manner; you can intuitively see the dynamic changes of data credibility over time.

[0061] Construct a correlation matrix between pollutants and meteorological parameters; show the relationship between different pollutants and meteorological conditions.

[0062] Based on the clustering pattern proportions in step S3, analyze the pollution sources that mainly cause abnormal data, as follows: To calculate the maintenance priority, the failure probability is factored in with a weight of 0.7, and the centrality is factored in with a weight of 0.3. Centrality, i.e., the importance of a site in the network, is weighted 0.3 to reflect its relative importance to the maintenance priority.

[0063] Find the maximum centrality value among all sites, multiply the failure probability by 0.7, divide the centrality by the maximum centrality, and then multiply by 0.3. Finally, add the two together to get the maintenance priority; the higher this value, the more priority the site needs for maintenance.

[0064] For each site pair, the corresponding element in the spatial correlation matrix is ​​multiplied by the absolute value of the concentration difference between the two sites to obtain the contribution value of the site pair in regional transmission. The contribution values ​​of all relevant site pairs are accumulated to obtain the total regional transmission contribution.

[0065] By constructing three-dimensional data quality heat maps, automated quality control reports, and maintenance priority formulas, the complex air quality data quality control results are presented in an intuitive and visual manner, providing a scientific basis for maintenance decision-making. At the same time, by calculating the regional transmission contribution, the pollution transmission relationship between regions is further analyzed.

[0066] It aims to provide managers with an intuitive, comprehensive and easy-to-understand air quality data quality control decision support tool to help them quickly understand data quality status, identify major pollution sources, determine maintenance priorities, and analyze the contribution of pollution transmission between regions, so as to more effectively formulate air quality management strategies and measures and improve the scientificity and timeliness of decision-making.

[0067] Existing technologies typically present data quality control results in simple reports or charts, which lack intuitiveness and comprehensiveness, making it difficult to simultaneously display the temporal and spatial variation characteristics of data quality and multi-dimensional information. This design constructs a three-dimensional data quality heat map, integrating information from three dimensions: space, time, and parameters, making data quality status clear at a glance. The analysis of the dominant source of abnormal data and the maintenance priority formula in the automated quality control report can more deeply explore the information behind the data, providing a more targeted and scientific basis for decision-making. The calculation of regional transmission contribution further expands the scope of data quality control analysis, analyzes pollution transmission relationships at the regional level, and provides support for cross-regional joint prevention and control of air pollution. These are all features that existing technologies do not have.

[0068] The three-dimensional data quality heat map can intuitively display the spatiotemporal distribution characteristics and quality change trends of air quality data, allowing managers to quickly locate data quality issues and high-incidence areas of pollution, thereby improving decision-making efficiency. The analysis of the dominant source of abnormal data in the automated quality control report helps to gain a deeper understanding of the sources and causes of pollution, providing strong support for precise pollution control; the application of the maintenance priority formula can rationally arrange equipment maintenance work, improve maintenance efficiency, reduce maintenance costs, and ensure the normal operation of the monitoring system. The calculation of regional transmission contribution provides a quantitative basis for inter-regional pollution transmission research and joint prevention and control work, which is conducive to strengthening regional cooperation and jointly addressing air pollution problems, thereby effectively improving the overall level of urban air quality monitoring and management and promoting the continuous improvement of regional air quality.

[0069] As an optional embodiment, it also includes establishing a quality control knowledge base update mechanism, and automatically expanding the S3 clustering model category when a new pollution source type is added.

[0070] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for quality control of air quality data based on time series data, characterized in that: The following steps are involved: S1. Acquire data from multiple atmospheric monitoring stations, construct a spatial topological relationship network of monitoring stations, and define the dynamic hierarchical structure of atmospheric pollutant data at different spatiotemporal scales and its cross-hierarchical association rules; S2. Perform anomaly detection and traceability analysis on the acquired monitoring data, and perform data correction and filling; S3. Apply a spatiotemporal clustering algorithm, combining site topology relationships with data association rules, to assess and grade the credibility of the corrected monitoring data, generating a spatiotemporal data credibility grading result. S4. When data credibility anomalies or potential failure risks are detected, a graph neural network model incorporating atmospheric diffusion physics constraints is used to predict the future failure probability and risk level of data at the monitoring site or region, thereby performing data failure risk prediction. S5. Based on the data credibility grading results of S3 and the data failure risk prediction of S4, a visual atmospheric data quality assessment and control decision report is generated, and instrument calibration, maintenance or data review instructions for specific sites or areas are automatically triggered according to the risk level.

2. The air quality data quality control method based on time series data according to claim 1, characterized in that: The specific working steps of S1 are as follows: Establishing a spatial correlation matrix of monitoring sites ; The monitoring sites are divided into three data levels.

3. The air quality data quality control method based on time series data according to claim 1, characterized in that: The specific working steps of S2 are as follows: Multi-source atmospheric monitoring station data specifically includes Data, ozone, nitrogen dioxide and nitrogen oxides, using the aerosol hygroscopic growth model to analyze the original The data is corrected to obtain the corrected data; When the ratio of nitrogen dioxide to nitrogen oxides is greater than the first threshold, and When the ratio of the carbon monoxide to the carbon monoxide is less than the second threshold, it is determined that the sensor has failed; If the hourly concentration of ozone is greater than the third threshold and the concentration of nitrogen dioxide is less than the fourth threshold, it is determined to be a transmission pollution event.

4. The air quality data quality control method based on time series data according to claim 2, characterized in that: The specific working steps of S3 are as follows: Use dynamic time warping algorithm to perform time series clustering: If it shows bimodal characteristics of morning peak and evening peak, and the peak ratio is greater than 1.5, it is a traffic source mode; If it is characterized by continuous flat-peak fluctuations with a variance of less than 15% and a matching degree with the wind direction vector of less than 30°, it is an industrial emission mode; If it meets both spatiotemporal clustering and chemical mechanism, it is Level 1; When only a single clustering feature of time or space is met, it is classified as Level 2; Cases that do not meet any of the above characteristics are judged as Level 3.

5. The air quality data quality control method based on time series data according to claim 4, characterized in that: The S3 further comprises the following steps: First determine within a certain time interval The change in concentration is obtained by subtracting the concentration at the start time from the concentration at the end time; Clarify the time length corresponding to the above concentration changes, that is, the length of time between the start time and the end time; calculate The rate of change of concentration in the vertical direction, that is, the change in concentration per unit height; Divide the concentration change by the product of the time interval and the vertical concentration gradient to obtain The actual sedimentation rate; Compare the calculated actual sedimentation rate with the theoretical sedimentation rate. If the deviation between the actual value and the theoretical value exceeds 30%, the data will be downgraded from Level 2 to Level 3. Measure the nitrogen dioxide concentration and ozone concentration in the current environment respectively, divide the nitrogen dioxide concentration by the ozone concentration to obtain their concentration ratio; Check whether the calculated concentration ratio is greater than 20, and measure the solar radiation intensity at this time to see if it is less than 200 watts per square meter; If the ratio of nitrogen dioxide to ozone concentrations is greater than 20 and the solar radiation intensity is less than 200 watts per square meter, the data is considered abnormal.

6. The air quality data quality control method based on time series data according to claim 4, characterized in that: The S4 comprises the following steps: Build a physically constrained graph neural network model. The input layer includes the site topology matrix and 72 hours of multi-parameter time series data. The site topology matrix reflects the spatial relationships between sites, while the multi-parameter time series data contains observations of multiple parameters at different times. Together, these form the basic information for the model input. Hidden layer: The model's hidden layer consists of a three-layer graph convolutional network coupled with LSTM units. The graph convolutional network mines spatial correlation information between sites, while the LSTM unit captures dynamic changes in time series. Data is processed sequentially through these three layers to gradually extract deep features. Output layer: After processing by the hidden layer, the model outputs the probability of data failure in the next 24 hours. This probability is between 0 and 1. The closer the probability is to 1, the greater the possibility of data failure; the closer it is to 0, the smaller the possibility of data failure. First, find the difference between the model's predicted value and the true value, then square this difference and calculate the average of all samples to get the first part of the loss; Considering each term in the atmospheric diffusion equation, we calculate the rate of change of pollutant concentration over time, the product of the wind speed vector and the concentration gradient, and the combination of the turbulent diffusion coefficient and the concentration gradient. We combine these terms and then find the deviation from the actual atmospheric diffusion situation. We square the deviations and calculate the average value to obtain the second part of the loss. The above two parts of loss are weighted and summed, where the weight coefficient is 0.6, and the value of the entire loss function is finally obtained.

7. The air quality data quality control method based on time series data according to claim 6, characterized in that: The S4 further comprises the following steps: Different countermeasures are taken based on the data failure probability calculated by S4. If the failure probability is greater than 0.7, the device calibration instruction is immediately triggered; If the failure probability is between 0.4 and 0.7, mark the data as Level 2 and shorten the sampling interval; When the failure probability is less than 0.4, it is sufficient to maintain regular monitoring frequency and operations.

8. The air quality data quality control method based on time series data according to claim 1, characterized in that: The S5 comprises the following steps: Generate a three-dimensional data quality heat map, mark the location of each site on the map, and use color markers to indicate different levels, with green representing Level 1, yellow representing Level 2, and red representing Level 3; Display the changing curve of data credibility in the past 24 hours in a rolling manner; Construct a correlation matrix between pollutants and meteorological parameters.

9. The air quality data quality control method based on time series data according to claim 1, characterized in that: It also includes establishing a quality control knowledge base update mechanism to automatically expand the S3 clustering pattern categories when new pollution source types are added.

10. The air quality data quality control method based on time series data according to claim 8, characterized in that: It also includes analyzing the pollution sources that mainly cause abnormal data based on the proportion of cluster patterns in step S3, as follows: To calculate maintenance priority, failure probability is used as a factor with a weight of 0.7, and centrality is used as a factor with a weight of 0.

3. Find the maximum centrality value among all sites, multiply the failure probability by 0.7, divide the centrality by the maximum centrality, and then multiply by 0.

3. Finally, add the two to get the maintenance priority; For each site pair, the corresponding element in the spatial correlation matrix is ​​multiplied by the absolute value of the concentration difference between the two sites to obtain the contribution value of the site pair in regional transmission. The contribution values ​​of all relevant site pairs are accumulated to obtain the total regional transmission contribution.

Images