Atmospheric pollution time series data feature fragment mining method and system based on form

Abstract

The invention discloses a form-based atmospheric pollution time series data feature fragment mining method and system. According to the method, clustering analysis is carried out by extracting time sequence sub-fragments (u-fragments), so that typical modes in the air pollution process are recognized. Specifically, the method proposes a discrete feature representation method to capture representative local morphological features in atmospheric pollution time series data, and at the same time, uses the similarity between extended Euclidean distance degree quantum sequences, and identifies sub-fragments with pollution identification degree through sub-sequence clustering. Based on the method, the invention further constructs a form-based atmospheric pollution time series data feature fragment mining system. The system can mine pollution fragments with identification degree from atmospheric pollution time sequence data, overcomes the limitation that a traditional statistics and frequency domain analysis method is weak in local feature capturing capability, and can be applied to the fields of urban pollution mode recognition, pollution traceability auxiliary decision making, environmental governance optimization and the like.

Description

Technical Field

[0001] The present invention relates to the field of atmospheric pollution data analysis, and in particular to a method and system for analyzing atmospheric pollution processes by means of morphological feature mining. Background Art

[0002] With the acceleration of urbanization and the continued growth of industrial emissions, air quality indicators, represented by pollutants such as PM2.5, NO2, and O3, have shown significant temporal fluctuations, characterized by complex properties such as high-frequency fluctuations, sudden changes, and spatial heterogeneity. Analysis of this type of pollution data requires not only identifying overall trends but also deeply exploring key morphological fragments in local evolution processes to assist in pollution source tracing, anomaly warnings, and remediation responses. Current atmospheric pollution data analysis primarily relies on traditional statistical methods and machine learning algorithms. Statistical methods such as moving averages and exponential smoothing can reflect the overall trends of pollutants but struggle to capture local morphological characteristics. While methods based on Fourier transforms or wavelet analysis can extract frequency domain features, they are insufficiently capable of characterizing the temporal morphological characteristics of pollution processes. In recent years, some studies have attempted to apply time series clustering methods (such as the k-Means algorithm based on dynamic time warping (DTW)) to the analysis of atmospheric pollution data. However, due to the high noise and high dimensionality of atmospheric pollution data, traditional algorithms face the following problems when processing it: First, existing methods find it difficult to effectively identify key morphological segments in the pollution process, such as typical patterns such as "rapid rise", "slow decline" or "multi-peak fluctuations"; second, current methods lack in-depth exploration of the correlation between pollution morphological characteristics and geographic spatial factors, which limits their application in pollution tracing and governance decision-making.

[0003] In response to the above problems, the present invention proposes a method and system for mining characteristic fragments of atmospheric pollution time series data based on morphology. This method focuses on the local morphology in the pollution evolution process, overcomes the limitations of traditional statistical and frequency domain analysis methods in their weak ability to capture local features, and can effectively extract representative key pollution morphological fragments such as "rapid rise", "slow decline" and "multi-peak fluctuation" from atmospheric pollution time series data. In order to further improve the analysis and application capabilities, an interactive visualization system was developed in combination with this method to realize the linkage analysis of pollution morphology and spatiotemporal elements. Starting from the essence of morphology, this system makes up for the shortcomings of traditional methods in local feature extraction and spatiotemporal correlation analysis of pollution processes, and can effectively improve the intelligence level and practical value of atmospheric pollution data analysis. Summary of the Invention

[0004] The present invention aims to provide a morphology-based method and system for mining characteristic segments of atmospheric pollution time series data. This method uses the morphology-based method to identify typical pollution morphology sub-segments and, based on these sub-segments, perform pollution feature pattern recognition, thereby addressing the problems raised in the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for mining characteristic segments of atmospheric pollution time series data based on morphology, the solution comprising the following steps:

[0006] S1. Obtain air quality monitoring data, including PM 2.5 、PM 10 , CO, NO2, O3 and SO2 six types of pollutants;

[0007] S2. Use the sliding window technology to segment the time series data of each type of pollutant, extract all subsequences and form a set of candidate subsequences, where the window length is sLen and the step length is sl;

[0008] S3. Use the discrete feature representation method DFR to represent each subsequence and extract its local morphological features. This step is the key to realizing the morphological recognition of the pollution process. It is suitable for the identification and mining of key segments such as the "rapid pollution rise segment", "violent pollution fluctuation segment" and "rapid pollution fall segment" commonly seen in atmospheric pollution data. By using the DFR method to represent the original pollutant time series data, the identifiability of local changes in the evolution of pollution events can be enhanced. For example, define the sequence in the sliding window as , fix sl to 1 to illustrate the specific processing steps of DFR:

[0009] A1. Initialize two lists and , which are used to store the intermediate variables and the final discrete results obtained during the window sliding process; where w represents the width of the sliding window, which is set according to the fluctuation degree of the pollutant data;

[0010] A2. Calculate the global mean (gmean) and local mean (lmean) of S:

[0011] (1)

[0012] (2)

[0013] Among them, gmean represents the average value of the pollutant time series S over the entire time range, that is, the overall concentration level of the pollutant in a given observation period, which is used to reflect the pollution baseline formed under long-term emissions, regional transmission or meteorological stability conditions. By comparing the gmean of different cities, stations or monitoring periods, it can be used to analyze regional air quality differences, cross-regional pollution impacts and changes in governance effectiveness. And lmean represents the mean of the local subsequence in the sliding window, that is, the local equilibrium level of pollutant concentration in a certain period of time. Compared with the long-term trend reflected by gmean, lmean emphasizes the responsiveness to short-term dynamic changes, and is especially suitable for capturing periodic fluctuations and sudden rises and falls in pollutant concentrations. When When , it means that the window is a local pollution rising area; when , indicating that the window is a region of local pollution decline. This comparison establishes a directional sensitivity mechanism that can be used to identify "pollution outbreak segments" and "inflection points," enhancing the method's ability to analyze pollution evolution trends.

[0014] A3. Q-based sorting index update :

[0015] (3)

[0016] in, Returns the index of the current subsequence Q in ascending order, Returns the index of the current subsequence Q in descending order. The relative ranking structure of pollutant concentrations is crucial for identifying typical temporal patterns such as "single-point peak pollution" and "delayed pollution growth," and is a core feature in characterizing pollution evolution. The adopted relative ordering mechanism does not rely on the absolute values ​​of pollutant concentrations, but only retains the relative magnitude relationship between time points. This improves the stability and generalization of the method under different sampling frequencies, pollution source intensities, and monitoring areas, laying the foundation for cross-regional pollution pattern mining and analysis.

[0017] A4. Calculation :

[0018] (4)

[0019] ,in (5)

[0020] in, express The value of the i-th element in , Used to find The position index in Q, m is the sliding window through The number of times we finally get is the DFR discrete representation result.

[0021] S4. After the DFR discrete representation, the candidate subsequences are filtered using random masking techniques to obtain a candidate set of u-shapelets. At the same time, the extended Euclidean distance (EED) is used to measure the similarity between sequences. The EED is calculated as follows:

[0022] (6)

[0023] (7)

[0024] in, and are the i-th data point of the time series TX and TY respectively, n is the length of the time series, and are the mean and standard deviation of the time series T, respectively. Traditional time series similarity metrics require z-normalization of the data, requiring the data to be approximately normally distributed. However, air pollution data often exhibit non-normal characteristics such as right-skewed distribution, multimodal distribution, and sudden peaks. The EED method does not require a strict normal distribution assumption and can more accurately preserve the morphological characteristics of the original data, which is particularly important for identifying sudden pollution events.

[0025] S5. Based on the gap value and the classification criteria , select the subsequence with the largest gap value in the candidate u-shapelet set as the most contamination-identifying subsequence u-shapelet in the current round, and divide the original dataset D into two subsets: a time series set with similar morphology to the fragment and the remaining time series set .Will As a separate cluster and extracted from the data set, and continue from Repeat the above operation until the iteration termination condition is met. The u-shapelets obtained in the whole process are sequence segments with pollution identification, that is, the characteristic segments of the mined atmospheric pollution time series data. Among them, the calculation method of the gap function is:

[0026] (8)

[0027] in and Represent the target sequence fragment S and the data set respectively and The average value of the distance between them is:

[0028] , ; and Represent the target sequence fragment S and the data set respectively and The standard deviation of the distance between them is: , ; Represents the distance between the sequence fragment S and all sequences in the data set D. The larger the gap value, the higher the quality of the subsequence and the stronger the discrimination.

[0029] To better implement the above method, we propose a morphology-based feature segment mining system for atmospheric pollution time series data. The system includes an air quality data module and a feature segment mining module. The air quality data module provides the data required by the feature segment mining module, which processes the atmospheric pollution time series data and returns the resulting u-shapelets, which are pollution-discerning subsequences.

[0030] The air quality data module includes: a data comparison unit, a data map visualization unit, a data retrieval and display unit, and a data crawler unit. The data comparison unit displays the changes in AQI and pollutant concentrations in different cities over the same time period; the data map visualization unit helps users observe the air quality distribution and pollutant concentration trends across the entire region; the data retrieval and display unit uses a table format to comprehensively display air quality data for multiple cities and related information on various pollutants; and the data crawler unit provides real-time air quality data display capabilities.

[0031] The feature fragment mining module includes: a mining method calling unit, a mining result management unit, and a mining result visualization unit. Among them, the mining method calling unit is used to select the corresponding air quality data and call the morphology-based atmospheric pollution time series data feature fragment method for processing based on the time period, subsequence length, and pollutant type information input by the user. It also provides the functions of saving mining results and downloading time series data sets. The mining result management unit is used to insert, delete, and query mining results. The mining result visualization unit is used to display the feature fragment mining results stored in the database or obtained when calling the morphology-based atmospheric pollution time series data feature fragment method. It uses Echarts to realize various visualization charts to help users analyze the distribution of pollutants.

[0032] Compared with the prior art, the present invention has the following significant advantages:

[0033] (1) Outstanding feature representation capability: The time series feature fragments extracted by the morphology-based atmospheric pollution time series data feature fragment mining method can more accurately represent the key morphological features in atmospheric pollution data;

[0034] (2) Superior computational performance: Using an optimized similarity measurement algorithm, it can effectively evaluate the similarity of time series while ensuring the accuracy of feature extraction, making it more suitable for processing atmospheric pollution time series data;

[0035] (3) Significant application value: The extracted feature fragments can directly serve practical application scenarios such as regional pollution feature analysis, pollution source fingerprint library construction, and governance effect evaluation. Compared with existing technologies, they have stronger engineering practicality and business interpretability;

[0036] (4) System integration innovation: The morphology-based atmospheric pollution time series data feature segment mining method is deeply integrated with the air quality analysis system, building a complete technical chain from multi-source data collection, feature mining to visualization display, and realizing the process and systematization of atmospheric pollution feature analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Schematic diagram of a flow chart of a method for mining characteristic segments of atmospheric pollution time series data based on morphology according to the present invention;

[0038] Figure 2 This is a structural diagram of a morphology-based atmospheric pollution time series data feature segment mining system of the present invention.

[0039] Figure 3 These are the u-shapelets extracted from the time series data of six air pollutants in this paper.

[0040] Figure 4 This is the processing result of the PM2.5 dataset by the present invention.

[0041] Figure 5 This is a visual display of the results of the present invention's treatment of PM10. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0043] Implementation example: Figure 1 As shown, the present invention provides a technical solution:

[0044] S1. Get the content containing PM 2.5 、PM 10 Time series monitoring data of air quality of six pollutants: CO, NO2, O3 and SO2;

[0045] S2. Use the sliding window technology to segment the time series data of each type of pollutant, extract all subsequences and form a set of candidate subsequences, where the window length is sLen and the step length is sl;

[0046] S3. Use the discrete feature representation method (DFR) to represent each subsequence and extract its morphological features. For example, define the sequence in the sliding window as , fix sl to 1, the specific steps are as follows:

[0047] (1) Initialize two lists and , which are used to store the intermediate variables and the final discrete results obtained during the window sliding process; where w represents the width of the sliding window, which is set according to the fluctuation degree of the pollutant data;

[0048] (2) Calculate the global mean (gmean) and local mean (lmean) of S:

[0049] (1)

[0050] (2)

[0051] Among them, gmean represents the average value of the pollutant time series S over the entire time range, that is, the overall concentration level of the pollutant in a given observation period, which is used to reflect the pollution baseline formed under long-term emissions, regional transmission or meteorological stability conditions. By comparing the gmean of different cities, stations or monitoring periods, it can be used to analyze regional air quality differences, cross-regional pollution impacts and changes in governance effectiveness. And lmean represents the mean of the local subsequence in the sliding window, that is, the local equilibrium level of pollutant concentration in a certain period of time. Compared with the long-term trend reflected by gmean, lmean emphasizes the responsiveness to short-term dynamic changes, and is especially suitable for capturing periodic fluctuations and sudden rises and falls in pollutant concentrations. When When , it means that the window is a local pollution rising area; when , indicating that the window is a region of local pollution decline. This comparison establishes a directional sensitivity mechanism that can be used to identify "pollution outbreak segments" and "inflection points," enhancing the method's ability to analyze pollution evolution trends.

[0052] (3) Q-based sorting index update :

[0053] (3)

[0054] in, Returns the index of the current subsequence Q in ascending order, Returns the index of the current subsequence Q in descending order. The relative ranking structure of pollutant concentrations is crucial for identifying typical temporal patterns such as "single-point peak pollution" and "delayed pollution growth," and is a core feature in characterizing pollution evolution. The adopted relative ordering mechanism does not rely on the absolute values ​​of pollutant concentrations, but only retains the relative magnitude relationship between time points. This improves the stability and generalization of the method under different sampling frequencies, pollution source intensities, and monitoring areas, laying the foundation for cross-regional pollution pattern mining and analysis.

[0055] (4) Calculation :

[0056] (4)

[0057] ,in (5)

[0058] in, express The value of the i-th element in , Used to find The position index in Q, m is the sliding window through The final result is is the DFR discrete representation result.

[0059] S4. In the candidate subsequence set, a random masking technique is used to filter out redundant subsequences to obtain a u-shapelet candidate set. At the same time, the extended Euclidean distance (EED) is used to measure the similarity between sequences. The EED calculation formula is as follows:

[0060] (6)

[0061] in, and are the i-th data point of the time series TX and TY respectively, n is the length of the time series, and are the mean and standard deviation of the time series T, respectively. Traditional time series similarity metrics require z-normalization of the data, requiring the data to be approximately normally distributed. However, air pollution data often exhibit non-normal characteristics such as right-skewed distribution, multimodal distribution, and sudden peaks. The EED method does not require a strict normal distribution assumption and can more accurately preserve the morphological characteristics of the original data, which is particularly important for identifying sudden pollution events.

[0062] S5. Based on the gap value and the classification criteria , select the subsequence with the largest gap value in the candidate u-shapelet set as the most contamination-identifying subsequence u-shapelet in the current round, and divide the original dataset D into two subsets: a time series set with similar morphology to the fragment and the remaining time series set .Will As a separate cluster and extracted from the data set, and continue from Repeat the above operation until the iteration termination condition is met. The u-shapelets obtained in the whole process are sequence segments with pollution identification, that is, the characteristic segments of the mined atmospheric pollution time series data.

[0063] The system includes:

[0064] Air quality data module and u-shapelets clustering method module;

[0065] Among them, the air quality data module is used to provide the data required by the u-shapelets clustering method module. The air quality data module includes: a data comparison unit, a data map visualization unit, a data retrieval and display unit, and a data crawler unit. Among them, the data comparison unit is used to display the changes in AQI and pollutant concentrations in different cities within the same time period; the data map visualization unit is used to help users observe the air quality distribution and pollutant concentration change trends in the entire area; the data retrieval and display unit uses a table format to comprehensively display the air quality data of multiple cities and related information on multiple pollutants; the data crawler unit is used to provide real-time air quality data display function;

[0066] Among them, the u-shapelets clustering method module is used to cluster the air quality data and return the clustering results. The u-shapelets clustering method module includes: a clustering algorithm calling unit, a clustering result management unit and a clustering result visualization unit. Among them, the clustering algorithm calling unit is used to select the corresponding air quality data and call the improved u-shapelets algorithm for processing based on the time period, subsequence length and pollutant type information input by the user, and provides the functions of saving the clustering results and downloading the time series data set. The clustering result management unit is used to insert, delete and query the clustering results. The clustering result visualization unit is used to display the clustering results stored in the database or obtained when calling the improved u-shapelets algorithm, and realizes a variety of visual charts through Echarts to help users analyze the distribution of pollutants.

[0067] Example

[0068] The experiment uses air quality data from 76 cities across China in 2024 as an example. The experimental steps are as follows: 1. Obtain air quality data and perform data preprocessing; 2. Use the method to process each pollutant dataset; 3. Visualize the clustering results.

[0069] The experimental process is as follows:

[0070] 1. Obtain air quality data. Crawl the air quality data for the target city and use the local mean filling method to fill in missing values ​​in the data. Ultimately, obtain the air quality data for 76 cities across China in 2024.

[0071] 2. Use the method to process each pollutant dataset. When using the improved u-shapelets method to process air quality data, the number of categories and subsequence length must be considered. To avoid overly granular data segmentation, the number of categories is set to 3, and the subsequence length ratio is selected to be 15%, resulting in a subsequence length of 54. Figure 3 These are the u-shapelets extracted from the time series data of six air pollutants by this method.

[0072] like Figure 4 Figure 2 shows the results of processing the PM2.5 dataset. It can be seen that after the improved u-shapelets method is used, the time series within each cluster show higher similarity in morphological characteristics (such as fluctuation trend, peak position, and change amplitude), while the time series between different clusters show significant differences.

[0073] 3. Visual display of clustering results. Figure 5 This visualization shows the results of using this method to process PM10. Users can view clustering results in charts and compare air quality data between cities using line graphs. The brown sequence segments are the most recognizable pollution feature segments, u-shapelets.

[0074] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

[0075] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A method for mining characteristic segments of atmospheric pollution time series data based on morphology, characterized in that: The following steps are involved: S1. Get the PM 2.5 、PM 10 Time series monitoring data of air quality of six pollutants: CO, NO2, O3 and SO2; S2. Use the sliding window technology to segment the time series data of each type of pollutant, extract all subsequences and form a set of candidate subsequences, where the window length is sLen and the step length is sl; S3. Use Discrete Feature Representation (DFR) to represent each subsequence and extract its morphological features. S4. In the candidate subsequence set, a random masking technique is used to filter redundant subsequences to obtain a u-shapelet candidate set; S5. In the u-shapelet candidate set, subsequences with high pollution recognition are selected as u-shapelets; each u-shapelet represents a typical pattern of a class of pollutant time series.

2. The method for mining characteristic fragments of atmospheric pollution time series data based on morphological features (DFR) according to claim 1, characterized in that: The specific steps of the DFR method in S3 include: A1. Initialize two lists and , which are used to store the intermediate variables and the final discrete results obtained during the window sliding process; where w represents the width of the sliding window, which is set according to the fluctuation degree of the pollutant data; A2. Calculate the global mean (gmean) and local mean (lmean) of S: (1) (2) Among them, gmean represents the average value of the pollutant time series S over the entire time range, that is, the overall concentration level of the pollutant in a given observation period, which is used to reflect the pollution baseline formed under long-term emissions, regional transmission or meteorological stability conditions. By comparing the gmean of different cities, stations or monitoring periods, it can be used to analyze regional air quality differences, cross-regional pollution impacts and changes in governance effectiveness. Lmean represents the mean of the local subsequence within the sliding window, that is, the local equilibrium level of pollutant concentration within a certain period of time. Compared with the long-term trend reflected by gmean, lmean emphasizes the responsiveness to short-term dynamic changes and is particularly suitable for capturing periodic fluctuations and sudden increases and decreases in pollutant concentrations. A3. Q-based sorting index update : (3) in, Returns the ascending order index of pollutant concentrations in Q, highlighting the low concentration interval characteristics; Return the descending index of pollutant concentrations in Q to enhance morphological identification during high-concentration periods; The ranking structure of pollutant concentrations is crucial for identifying patterns such as "single-point peak pollution" and "delayed pollution growth" and is the core feature of characterizing temporal morphology. A4. Calculation : (4) ,in (5) in, express The value of the i-th element in , Used to find The position index in Q, m is the sliding window through The number of times we finally get is the DFR discrete representation result.

3. The method according to claim 1, characterized in that The steps for selecting the most contamination-discriminative subsequence u-shapelet in S5 are as follows: Based on the gap value, according to the classification criteria , select the subsequence with the largest gap value in the candidate u-shapelet set as the most contamination-identifying subsequence u-shapelet in the current round, and divide the original dataset D into two subsets: a time series set with similar morphology to the fragment and the remaining time series set ;Will As a separate cluster and extracted from the data set, and continue from Repeat the above operation until the iteration termination condition is met; the u-shapelets obtained in the whole process are sequence segments with pollution identification, that is, the characteristic segments of the mined atmospheric pollution time series data; the gap value calculation formula is as follows: (6) in and Represent the target sequence fragment S and the data set respectively and The average value of the distance between them is: , ; and Represent the target sequence fragment S and the data set respectively and The standard deviation of the distance between them is: , ; Represents the distance between the sequence segment S and all sequences in the dataset D.

4. A morphology-based system for mining characteristic segments of atmospheric pollution time series data, characterized by: The system includes an air quality data module and a feature segment mining module. The air quality data module is used to provide the data required by the feature segment mining module; the feature segment mining module is used to process the atmospheric pollution time series data and return the obtained subsequence u-shapelets with pollution identification.

5. The method according to claim 5, characterized in that: The air quality data module includes: a data comparison unit, a data map visualization unit, a data retrieval and display unit, and a data crawler unit. The data comparison unit is used to display the changes in AQI and pollutant concentrations in different cities within the same time period; the data map visualization unit is used to help users observe the air quality distribution and pollutant concentration change trends in the entire area; the data retrieval and display unit uses a table format to comprehensively display the air quality data of multiple cities and related information on various pollutants; the data crawler unit is used to provide real-time air quality data display functions.

6. A method according to claim 5, characterized in that: The feature fragment mining module includes: a mining method calling unit, a mining result management unit and a mining result visualization unit. Among them, the mining method calling unit is used to select the corresponding air quality data and call the morphology-based atmospheric pollution time series data feature fragment method for processing according to the time period, subsequence length and pollutant type information input by the user, and at the same time provides the functions of saving the mining results and downloading the time series data set; the mining result management unit is used to insert, delete and query the mining results; the mining result visualization unit is used to display the feature fragment mining results stored in the database or obtained when calling the morphology-based atmospheric pollution time series data feature fragment method, and realizes a variety of visualization charts through Echarts to help users analyze the distribution of pollutants.

Images