Air quality prediction method and system

Abstract

The present invention discloses an air quality prediction method and system. The method includes obtaining an air quality data set; constructing a correlation segmentation and reorganization sorting interactive convolution transformer model; the model includes a CNN convolution neural network model for extracting local features of input data; two Transformer models for extracting global dependency features of input data; a fully connected layer for outputting air quality prediction results based on local features and global dependency features; inputting data in the air quality data set into the correlation segmentation and reorganization sorting interactive convolution transformer model to obtain the air quality prediction results output by the correlation segmentation and reorganization sorting interactive convolution transformer model. The solution provided by the present invention performs better in long-term time series prediction, and has higher accuracy and generalization ability. At the same time, the response efficiency to data mutation values ​​is improved through the dynamic feature capture mechanism, and the dynamic interaction relationship of multivariate features is integrated, which significantly improves the prediction performance.

Description

Technical Field

[0001] The present invention relates to the technical field of environmental monitoring, and in particular to an air quality prediction method and system. Background Art

[0002] At present, air quality prediction is an important part of air pollution prevention and control, and has important guiding significance for scientific decision-making and environmental governance. To address this problem, academia and industry have adopted a variety of methods, mainly including physical models, statistical models, and deep learning methods that have developed rapidly in recent years.

[0003] Physical models describe and simulate chemical, diffusion, deposition and dynamic processes in the atmosphere through mathematical and physical equations. They are based on physical principles, have strong applicability and are easy to interpret. Common physical models include Gaussian models (such as ISC, AERMOD, ADMS) and integrated air quality simulation systems (such as CMAQ, CAMx, WRF-CHEM, NAQPMS). These models can effectively simulate the interaction of multiple pollutants, especially in complex environments. However, physical models have limitations such as complex models, high data requirements, and lack of accurate description of nonlinear chemical reactions, and are insufficient in the treatment of urban and regional chemical reactions.

[0004] Statistical models predict air quality by statistically analyzing numerical data and establishing the relationship between material content and meteorological conditions. Common statistical models include concurrent regression models, stepwise regression models, multivariate regression models, cluster analysis models, and BP neural network models. For example, Cobourn used a nonlinear regression model to predict the daily maximum value of regional PM2.5, and Paschalidou et al. used artificial neural networks (such as MLP and RBF) to predict PM10 concentration. In addition, support vector regression (SVR), hidden Markov model, AdaBoost, and ensemble learning methods are also used in air quality prediction. However, statistical models are insufficient in dealing with nonlinear and complex spatiotemporal relationships, and are sensitive to data quality, which limits the prediction accuracy and real-time performance.

[0005] With the growth of data scale and the improvement of computing power, deep learning has gradually shown superior performance in air quality prediction. Deep learning methods can process complex data and capture nonlinear relationships, and are important tools for predicting complex environments. For example, the Informer model based on long-sequence time series prediction significantly improves the inference speed of long-sequence prediction; the framework combining 3D convolutional neural network and LSTM model can effectively capture the characteristics of PM2.5 data. In addition, graph neural network (GNN) also performs well in air quality prediction. In particular, the IVLSTM-MCMR model proposed by Fang et al. and the multi-model combination scheme of Wen et al. significantly improve the prediction accuracy. However, current deep learning methods still have shortcomings in long-term prediction and sensitivity to mutation values. For example, Feng et al. developed a new air quality prediction model called CRINet, which showed strong performance in predicting air quality time series. However, its framework mainly reflects the excellent performance in short-term prediction, and its performance in long-term prediction is still unsatisfactory. Summary of the invention

[0006] In order to solve the technical problems that the existing air quality prediction model has great deficiencies in long-term time series prediction and sensitivity to sudden changes, the embodiments of the present invention provide an air quality prediction method and system.

[0007] The technical solution of the embodiment of the present invention is achieved as follows:

[0008] An embodiment of the present invention provides an air quality prediction method, which includes: obtaining an air quality data set; constructing a correlation segmentation and reorganization sorting interactive convolution transformer model; the correlation segmentation and reorganization sorting interactive convolution transformer model is a CTT structure, including a CNN convolution neural network model, two Transformer models and a fully connected layer; the CNN convolution neural network model is used to extract local features of input data; the Transformer model is used to extract global dependency features of input data; the fully connected layer is used to output air quality prediction results according to the local features and the global dependency features; data in the air quality data set is input into the correlation segmentation and reorganization sorting interactive convolution transformer model to obtain the air quality prediction results output by the correlation segmentation and reorganization sorting interactive convolution transformer model.

[0009] In one embodiment, the CNN convolutional neural network model includes an Attention-split layer, an MBConv module, a CRINet model, a CRI-Attention layer and a connection layer; and the Transformer model includes a CRI-Attention layer.

[0010] In one embodiment, the Attention-split layer is used to divide the input data into subsequences with high correlation and subsequences with low correlation according to the correlation level of the input data, input the subsequences with high correlation into the MBConv module, and input the subsequences with low correlation into the CRINet model.

[0011] In one embodiment, the Attention-split layer is specifically used to calculate the attention score of each feature in the input data relative to the variable target using the self-attention mechanism; sort the features of the input data in descending order of the attention scores; obtain the median value in the sorting result; divide the features before the median value in the sorting result into subsequences with high correlation; divide the features after the median value in the sorting result into subsequences with low correlation; input the subsequences with high correlation into the MBConv module, and input the subsequences with low correlation into the CRINet model.

[0012] In one embodiment, the MBConv module is used to extract key features from the highly correlated subsequences and enhance the expressive power of the correlation segmentation and reorganization sorting interactive convolution transformer model for highly correlated variables.

[0013] In one embodiment, the CRINet model is used to perform feature learning on the subsequences with low correlation, and enhance the modeling capability of the correlation segmentation and reorganization sorting interactive convolutional transformer model for complex relationships.

[0014] In one embodiment, the CRI-Attention layer is used to convert input data into queries and keys using a CRINet model with a step size of k, capture local context information in the input data based on the converted queries and keys, and enhance the modeling ability of the correlation segmentation and reorganization sorting interactive convolutional transformer model for spatiotemporal dependencies.

[0015] In one embodiment, the connection layer is used to perform a first fusion of the features output by the MBConv module and the features output by the CRINet model to obtain a first fusion feature; and perform a second fusion of the first fusion feature and the features output by the CRI-Attention layer to obtain a second fusion feature.

[0016] An embodiment of the present invention further provides an air quality prediction system, comprising: a processor and a memory for storing a computer program that can be run on the processor; wherein the processor executes the steps of the above-mentioned method when running the computer program.

[0017] An embodiment of the present invention further provides a storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0018] This embodiment has the following beneficial effects:

[0019] 1. Efficient feature segmentation and extraction

[0020] Correlation segmentation and modular processing: Through the Attention-Split layer, the data is divided into two sub-sequences of high correlation and low correlation according to the correlation between the variable and the target variable, and the CRINet and MBConv modules are used for feature learning respectively. This segmentation and modular processing method greatly improves the efficiency and pertinence of feature extraction.

[0021] Technical Effect: When capturing the complex characteristics of multivariate time series, the model can effectively distinguish key information from secondary information, optimize computational efficiency and improve prediction accuracy.

[0022] 2. Improved Attention Mechanism

[0023] CRI-Attention layer: CRINet convolution is used to replace the traditional convolution with a stride of 1, so that the model can fully capture local context information when generating queries (Q) and keys (K).

[0024] Technical effect: It enhances the model's ability to understand the context of spatiotemporal data, solves the limitations of traditional Transformer in local context sensitivity, and greatly improves the ability to identify extreme values ​​and outliers.

[0025] 3. Innovative CTT structure

[0026] Combination of CNN and Transformer: CRIformer uses a CTT structure consisting of a convolutional neural network (CNN) and dual Transformers, combining the ability to capture local features with the ability to model long-term spatiotemporal dependencies.

[0027] Technical effect: It realizes unified modeling of short-term dynamic trends and long-term dependencies, which is particularly suitable for processing multivariate time series characteristics in air quality data and improves the prediction stability and generalization ability of the model.

[0028] 4. Lightweight and high efficiency

[0029] Introduction of MBConv module: Optimizes computational efficiency through depth-wise separable convolution and bottleneck structure, reduces the number of parameters and computation, while retaining powerful feature extraction capabilities.

[0030] Technical effect: The complexity is significantly reduced while maintaining the model performance, making it suitable for resource-constrained computing environments.

[0031] 5. Enhanced robustness and generalization

[0032] Multi-module collaboration: The Attention-Split layer is combined with the CRI-Attention layer to provide higher sensitivity to outliers and mutations that are common in air quality data.

[0033] Technical effect: When faced with external factors such as extreme weather and emergencies, the model can effectively capture key changes and generate stable forecast results.

[0034] 6. Excellent long-term prediction capability

[0035] Optimization of long-term spatiotemporal dependency modeling: Through the improvement of the CRINet model and the synergy of dual Transformers, the model solves the common problem of cumulative error amplification in long-term time series prediction.

[0036] Technical effect: Significantly improved the accuracy and reliability of the model in long-term prediction tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a structural diagram of the existing CRINet model in an embodiment of the present invention;

[0038] Figure 2 Schematic diagram of the flow of the air quality prediction method according to an embodiment of the present invention;

[0039] Figure 3 Schematic diagram of the principle of the CRIformer model of an embodiment of the present invention;

[0040] Figure 4 This is a schematic diagram of the principle of the Attention-Split layer in an embodiment of the present invention;

[0041] Figure 5 Schematic diagram of the principle of the MBConv module according to an embodiment of the present invention;

[0042] Figure 6 A comparison diagram of the conventional Transformer framework using ordinary convolution to generate Q and K and the CRIformer model of the present invention using the CRI-Attention layer to generate Q and K; wherein, Figure 6 (a) Schematic diagram of using ordinary convolution to generate Q and K in the traditional Transformer framework; Figure 6 (b) is a schematic diagram of the CRIformer model according to an embodiment of the present invention using the CRI-Attention layer to generate Q and K;

[0043] Figure 7 1 is a diagram showing the internal structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] Before introducing the solution of this embodiment, the existing CRINet model is first introduced.

[0045] CRINet: is the abbreviation of Correlation-split and Reorganization-sort Interaction Convolution Networks. It is a deep learning model designed for time series prediction. It groups multivariate data by correlation through "correlation splitting", optimizes feature arrangement to highlight important variables through "reorganization sorting", and uses interactive convolutional networks to model the dynamic interaction between variables to capture local and global features in spatiotemporal data. The CRINet model performs well in processing complex nonlinear features and multivariate interactions, and is particularly suitable for fields such as air quality prediction.

[0046] The existing CRINet model is superior in improving the performance of processing time series problems, mainly in accurately capturing data trends and patterns within a short time range. The structural diagram of the CRINet model is shown in the figure below. Figure 1 shown. Figure 1 In the paper, the first layer of the CRINet model divides the input data into two subsequences through a related segmentation strategy, and then performs convolutional learning. In the second layer of the model, multiple CRI-Blocks are combined into a binary tree structure, and there is a reorganization and sorting operation layer between each CRI-Block layer to better integrate information. Within the CRI-Block, the input data is split again and convolutional learning is performed to capture more advanced feature representations. Finally, by reordering the minimum subsequences inside each CRI-Block and adding them to the initial sequence, the CRINet model is able to generate the final prediction output. This new network structure that combines the related segmentation strategy and the reorganization and sorting operation enables the CRINet model to more effectively capture relevant information in the data and show better performance in prediction tasks.

[0047] However, when faced with long-term prediction tasks, the performance of the CRINet model has not yet reached a fully satisfactory level, which involves a series of complex challenges and limitations. In long-term prediction, time series data may be affected by more variable factors, usually covering multiple variables, some of which have a significant impact on PM2.5, such as wind direction and humidity. Therefore, the CRINet model may face the problem of insufficient model memory capacity, making it difficult to effectively capture these long-term dependencies, resulting in a decrease in the accuracy of the prediction results. In addition, long-term prediction tasks may also be interfered by more external factors, such as emergencies, policy changes, or natural disasters, and the uncertainty of these factors may affect the robustness and generalization ability of the model. In terms of long-term prediction, the accumulated errors of the data may also gradually amplify over time, leading to instability in the prediction.

[0048] At the same time, air quality data is characterized by multiple outliers due to the influence of many factors, including but not limited to holidays, human activities, industrial production emissions, and extreme weather conditions. As these diverse factors are intertwined, monitoring and predicting air quality becomes challenging. Data fluctuations and outliers may appear in a short period of time, which will make it difficult for the CRINet model to accurately predict.

[0049] Based on this, this embodiment proposes an air quality prediction method based on the CRINet model, and makes up for the shortcomings of the existing CRINet model in air quality prediction, especially the limitations in long-term modeling capabilities and sensitivity to data mutation values, by proposing a correlation-split and reorganization-sort interaction convolution transformer model (also known as the CRIformer model). The Transformer model of this embodiment not only captures the local trend pattern of multivariate air quality time series data, but also improves the modeling capabilities of long-term and spatial dependencies by designing a CRI-Attention layer and combining a hybrid CRINet, CNN and Attention mechanism. In addition, compared with the CRINet model that is only good at short-term modeling, the CRIformer model performs better in long-term prediction and has higher accuracy and generalization ability. At the same time, the dynamic feature capture mechanism improves the response efficiency to data mutation values, and integrates the dynamic interaction relationship of multivariate features, significantly improving the prediction performance. Experiments show that the CRIformer model of this embodiment is superior to existing methods in terms of accuracy, real-time and adaptability, and provides a more efficient and comprehensive solution for air quality prediction.

[0050] The present invention will be described in further detail below with reference to the accompanying drawings and embodiments.

[0051] The embodiment of the present invention provides an air quality prediction method, such as Figure 2 As shown, the method includes:

[0052] Step S101: Acquire an air quality data set;

[0053] Step S102: constructing a correlation segmentation and reorganization sorting interactive convolution transformer model; the correlation segmentation and reorganization sorting interactive convolution transformer model is a CTT structure, including a CNN convolution neural network model, two Transformer models and a fully connected layer; the CNN convolution neural network model is used to extract local features of input data; the Transformer model is used to extract global dependency features of input data; the fully connected layer is used to output air quality prediction results according to the local features and the global dependency features;

[0054] Step S103: inputting the data in the air quality data set into the correlation segmentation and reorganization sorting interactive convolution transformer model to obtain the air quality prediction result output by the correlation segmentation and reorganization sorting interactive convolution transformer model.

[0055] This embodiment proposes a CRIformer model. This model aims to make up for the shortcomings of the CRINet model in terms of long time series and its lack of sensitivity to mutation values. The CRIformer model proposed in this embodiment achieves significant improvements in air quality prediction tasks by combining the CRINet and Transformer architectures. Its design solves the shortcomings of existing models in long time series prediction and extreme value capture, while greatly improving the model's ability to understand and process complex time series data.

[0056] Transformer: is a deep learning model based on the attention mechanism. It replaces the traditional recurrent neural network (RNN) with the structure of Self-Attention and Feed-Forward Network, solving the problem of capturing dependencies in long sequence data modeling. The core of Transformer lies in the Attention Mechanism, which can dynamically assign weights to different positions to efficiently capture global dependencies while supporting parallel computing. Due to its powerful modeling capabilities and flexibility, Transformer is widely used in natural language processing (NLP), time series analysis, computer vision and other fields.

[0057] The CRIformer model proposes the CRI-Attention layer and captures the dynamic, temporal and nonlinear characteristics of multivariate air quality time series data by mixing CRINet, CNN and Attention mechanisms. The model can learn local trend patterns and long-term spatiotemporal dependencies of multivariate air quality related time series data (such as PM2.5, wind speed, temperature, etc.). The results show that the proposed CRIformer model has good prediction performance and generalization ability. Experiments show that the method is effective.

[0058] Specifically, see Figure 3 , Figure 3 The schematic diagram of the principle of the CRIformer model of the embodiment of the present invention. The CRIformer model of this embodiment uses the CTT (C: CNN, T: Transformer) structure, and simultaneously utilizes the powerful capabilities of convolutional neural networks (CNN) and Transformers. Based on the MBConv block and relative self-attention, translation equivariance, input adaptive weighting, and global receptive field are integrated together. Figure 3 It can be seen that this embodiment also adds an attention mechanism to structure C, and combines the CRI-Attention layer and the convolutional neural network to enhance the feature learning ability of the model.

[0059] As can be seen from the structure, the input data enters the Attention-split layer and the CRI-Attention layer at the same time. The Attention-split layer splits the input data into two groups of sub-data sequences with different correlations. The sub-sequences with high correlation use CRINet for feature learning, and the sub-sequences with low correlation use the MBConv module for feature extraction learning. After that, the results of the Attention-split layer and the CRI-Attention layer are combined, and then input into the two CRI-Attention modules for output, and finally the prediction results are obtained through the fully connected layer (FC).

[0060] Several important parts of the CRIformer model of this embodiment are introduced in detail below.

[0061] Attention-Split Layer

[0062] In the Transformer model, the self-attention mechanism is the core component of the model, which allows the model to establish direct dependencies between different positions in the same sequence. This mechanism is suitable for processing long sequences because it can reveal the complex interactions and dependencies between elements in the sequence. Specifically, in self-attention, each sequence position not only pays attention to its own information, but also adjusts its expression based on the information of other positions. The core of the self-attention mechanism is to assign weights to each element in the sequence by calculating the attention weight matrix A, evaluate the similarity between elements by calculating the dot product between the query and the key, and then normalize these scores through the softmax function to obtain the attention weight of each element to other elements. As shown in formula (1):

[0063] (1)

[0064] Among them, Q, K, and V represent query vector (Query), key vector (Key), and value vector (Value), respectively. d Represents the dimensions of the query vector and key vector.

[0065] Therefore, in the Attention-Split layer, the attention score of each feature in the data set relative to the variable target is first calculated. These scores are obtained by the softmax function, which is usually measured based on the scaled dot product, which evaluates the alignment or correlation between the feature vector and the target vector. This alignment shows the predictability of the change in the feature to the change in the predicted target. By calculating the attention score, this embodiment can quantify the correlation between each feature and the target variable, and use the score to sort all the features in the data set. In other words, this sorting is based on the potential impact of each feature on the accuracy of the model predicting the target variable, thereby determining the priority of each feature. After the attention score is calculated, the features are sorted from high to low according to the score, and the features are divided into two groups based on the median of the calculated attention score. Features with attention scores higher than the median constitute the upper half, and this part of the feature subsequence is considered to have higher predictive value and have a greater influence on the prediction results of the target variable or provide more key information. Features with attention scores lower than the median constitute the lower half, and the predictive value of these feature subsequences is relatively low. This hierarchical approach allows features of different importance to be processed separately or analyzed collectively, thereby optimizing the establishment of subsequent prediction models and improving computational efficiency and prediction effectiveness. Figure 4 shown.

[0066] MBConv Module

[0067] MBConv: It is the abbreviation of Moving Backward Residual Bottleneck Convolution, which means "reverse moving residual bottleneck convolution". It is an efficient deep learning convolution module. Its main feature is to introduce residual paths through moving backward connections (Moving Backward) to alleviate the gradient vanishing problem, and combine the bottleneck structure (Bottleneck) to reduce computational complexity. MBConv usually includes operations such as dilated convolution, depthwise separable convolution, and projected convolution. It can reduce the number of parameters and computation while maintaining model accuracy. Therefore, it is widely used in lightweight neural network structures (such as MobileNetV2, EfficientNet) to improve the efficiency of inference on resource-constrained devices.

[0068] The MBConv module makes full use of the Depthwise Separable Convolution and Bottleneck structures to reduce the number of parameters and the amount of computation, making the network model lighter, so that better performance can be achieved with limited computing resources. Among them, the depthwise separable convolution decomposes the traditional convolution operation into two steps: first, depthwise convolution is performed to apply convolution independently on each input channel; then point convolution is performed, that is, the output channels of the depthwise convolution are mixed in the convolution kernel used. This method greatly reduces the number of multiplication operations required in the calculation process, thereby reducing the computational requirements of the model and improving the computational efficiency. This modular network design not only improves computational efficiency, but also reduces the complexity of the model without sacrificing network performance.

[0069] In addition, the MBConv module incorporates a bottleneck structure, characterized by its initial expansion layer, deep convolution, and subsequent linear projection layer that compresses the expanded feature map. This design improves the efficiency and effectiveness of feature extraction by increasing the depth of the network and learning more complex features with fewer computing resources. In the bottleneck structure, the addition of the linear projection layer maintains the consistency of the network channel size, allowing the network to achieve seamless integration of features through operations such as addition or multiplication. This technology further enhances the network's ability to capture and synthesize key features, making it more efficient and effective when processing various data. The principle diagram of the MBConv module is shown below. Figure 5 shown. Figure 5 A visual representation of the module’s complex structure and operational workflow is provided, offering insight into the rationale behind its functional enhancements and design choices, which are critical to achieving superior performance even under tight computational constraints.

[0070] CRI-Attention Layer

[0071] Traditional Transformer frameworks usually use convolutions with a stride of 1 to transform inputs into queries and keys. Patterns in time series can change significantly over time due to various events, such as holidays and extreme weather. Therefore, whether an observation is an anomaly, a change point, or part of a pattern is highly dependent on its surrounding environment. However, in the Transformer's self-attention layer, the similarity between the query and the key is calculated based on their point-wise values, without fully utilizing local context (such as shape). Figure 6 (a). The query key matching is agnostic to the local context, which may confuse the self-attention module as to whether the observed value is an anomaly, a change point, or part of a pattern, and bring potential optimization issues.

[0072] To overcome the above limitations, see Figure 6 (b) The CRIformer model uses a CRINet model with a stride of k to replace the traditional convolution with a stride of 1 to convert input data into queries and keys. This design allows queries and keys to capture local contextual information more effectively, thereby improving the accuracy of predictions. As a special form of point convolution, the CRINet model aims to capture spatial hierarchical structures by aggregating information from a larger receptive field without losing resolution. This addresses common limitations in standard convolution operations and provides an improved way to handle complex input data, thereby improving the performance and efficiency of the overall model. By adjusting the stride to k, these convolutions help to aggregate spatial features more extensively and effectively at each layer of the network. Converting input data into queries and keys through CRINet convolutions effectively enhances the model's attention mechanism. In neural networks, queries and keys are vectors used in the attention mechanism to calculate the alignment scores between different parts of the input data, allowing the model to focus on the most relevant information for prediction. By effectively converting input data into queries and keys, the CRINet model helps the model capture richer contextual clues, achieve a deep understanding and precise expression of the input data, and thus significantly improve the model's prediction accuracy. Therefore, using the CRINet model in the CRIformer model to calculate queries and keys not only optimizes the extraction of local and extended context information, but also significantly improves the prediction accuracy and efficiency of the model in complex analysis tasks.

[0073] The innovations and improvements of the CRI-Attention module can be summarized into the following key points:

[0074] 1. Local context capture based on CRINet convolution

[0075] Traditional Transformer self-attention mechanisms usually ignore local contextual information and rely solely on the dot product calculation between the query and the key, which may lead to insufficient understanding of local patterns in long sequences.

[0076] Improvement: The CRI-Attention module explicitly captures local context information in the computation of queries and keys by replacing the traditional stride 1 convolution with CRINet convolution with stride k.

[0077] Effect: This convolution method can aggregate a wider range of spatial features while retaining resolution, significantly improving the model's ability to perceive local patterns, thereby optimizing prediction accuracy.

[0078] 2. Step size setting improves feature capture efficiency

[0079] The introduction of CRINet convolution with a step size of k solves the problem of small receptive field in traditional convolution.

[0080] Innovation: The increase in step size enables the convolution operation to capture spatial hierarchical structure information in a larger range without increasing the computational complexity.

[0081] Results: Compared with the traditional convolution with a step size of 1, the improved method reduces the redundant calculation of fine-grained features and improves the utilization efficiency of contextual information.

[0082] 3. Optimized representation of queries and keys

[0083] The queries and keys generated by processing the input data through CRINet convolution not only carry the basic features of the input, but also integrate richer contextual information.

[0084] Mechanism: The high-dimensional features generated by convolution are used as the input of queries and keys, so that the attention distribution can focus on important input areas more accurately.

[0085] Advantages: Compared with queries and keys generated directly using linear transformations, this method is more expressive in capturing complex relationships and hierarchical features between data.

[0086] 4. Balance between global receptive field and local features

[0087] The CRI-Attention module overcomes the defect of the traditional attention mechanism that relies too heavily on global features by combining local context and global information.

[0088] Technical highlights: By capturing local context through CRINet convolution and combining it with the global receptive field of Transformer, the module can more comprehensively understand the overall structure and detailed characteristics of the input data.

[0089] Improvement effect: This innovation greatly improves the model's ability to identify abnormal patterns and change points in time series, and enhances the robustness of the model.

[0090] 5. Enhance prediction accuracy and generalization ability

[0091] By introducing local context information, the CRI-Attention module not only improves the prediction accuracy, but also enhances the generalization ability of the model in complex and changing time series data.

[0092] Specific performance: For outliers caused by emergencies or external interference, the CRI-Attention module can more flexibly adjust the attention distribution, thereby improving prediction stability and accuracy.

[0093] The CRI-Attention module captures long-sequence dependencies while introducing deep attention to local context. This CRINet convolution-based design not only optimizes the traditional Transformer's self-attention mechanism, but also significantly improves the performance of the time series prediction model in changing scenarios, providing a more efficient solution to the long and short time series prediction problem.

[0094] The CRIformer model in this embodiment is a new model that combines the convolutional neural network (CNN) and the transformer (Transformer) architecture, and is designed for multivariate time series prediction. Through the "correlation segmentation" and "reorganization sorting" mechanisms, CRIformer can efficiently screen and sort key features, thereby highlighting the correlation between variables; using interactive convolutional layers to capture local features of spatiotemporal data, and then combining the Transformer structure to model long-term dependencies and complex nonlinear relationships. This model excels in capturing dynamic change patterns, improving prediction accuracy and generalization capabilities, and is particularly suitable for areas such as air quality prediction.

[0095] The CRIformer model proposed in this embodiment demonstrates excellent technical effects and performance advantages in air quality prediction through innovative architecture design. Its core advantages are reflected in the high efficiency of feature extraction, the improvement of attention mechanism, lightweight design, and strong robustness and generalization capabilities. Experimental results show that the CRIformer model can generate high-precision prediction results in a variety of complex scenarios, providing a reliable and efficient solution for long-term and complex multivariate time series analysis tasks.

[0096] Before the CRIformer model of this embodiment predicts the air quality data set, it can be trained, verified and tested with the data set. Here, the data set used in the experiment can use the Beijing PM2.5 data set and the Beijing multi-point air quality data set. In the Beijing PM2.5 data set, there are 8 variables, namely: PM2.5 concentration value, dew point, temperature, air pressure, wind direction, wind speed, snowfall, and rainfall. The time range is from January 1, 2010 to December 31, 2014, with a total of 43,824 data. The data set is divided into a training set, a verification set and a test set. Among them, the data from January 1, 2010 to December 31, 2012 is used as a training set; the data from January 1, 2013 to December 31, 2013 is used as a verification set. Finally, the remaining data (January 1, 2014 to December 31, 2014) is used as a test set.

[0097] There are 10 variables in the Beijing multi-point air quality dataset, such as PM2.5, PM10, NO2, CO, O3, SO2, temperature, precipitation, pressure, wind speed, etc. The time range of this dataset is 420,768 records from 12 monitoring stations in Beijing, including Olympic Sports Center, Changping, Dingling, Dongsi, Guanyuan, Gucheng, Huairou, Agricultural Exhibition Hall, Shunyi, Temple of Heaven, Wanliu, and Wanshou West Palace, from March 1, 2013 to February 28, 2017. The Beijing multi-point air data set is divided into a training set (80%) with a total of 336,614 data, a validation set (10%) with a total of 42,076 data, and a test set (10%) with a total of 42,076 data.

[0098] The CRIformer model proposed in this embodiment first uses the attention segmentation module to divide the input data into two groups of subsequences according to different correlation levels. The subsequences with low correlation use correlation splitting and reorganization sorting interactive convolution (CRINet) for feature learning, and the subsequences with high correlation use moving reverse residual bottleneck convolution (MBConv) for feature extraction learning. This method enhances the ability of feature extraction. Then, correlation splitting and reorganization sorting interactive convolution attention is used to solve the problem that Transformer ignores the contextual relationship when calculating Query and Key, thereby improving the accuracy of air quality extreme value prediction. CRIformer uses a convolutional neural network (CNN) and a dual Transformer to form a CTT structure to extract internal dependencies and time features from air quality data for prediction.

[0099] Compared with the existing technology, it has the following differences:

[0100] (1) Innovative feature segmentation and learning mechanism

[0101] CRIformer first divides the input data into two groups of subsequences, low-correlation and high-correlation, according to their correlation level through the attention segmentation module. For low-correlation subsequences, Correlation Splitting and Reorganization Inter-convolution (CRINet) is used for feature learning to enhance the modeling ability of complex relationships. For high-correlation subsequences, Moving Inverse Residual Bottleneck Convolution (MBConv) is used to extract key features, which enhances the model's ability to express highly correlated variables.

[0102] Difference: Compared with existing technologies (such as traditional Transformer and CRINet), CRIformer can specifically process data with different correlation levels, improving the pertinence and efficiency of feature extraction.

[0103] (2) Improved Transformer Attention Mechanism

[0104] CRIformer introduces the interactive convolutional attention of correlation segmentation and reorganization sorting, which effectively solves the problem that the traditional Transformer ignores the contextual relationship when calculating the query and key. By combining convolution with the attention mechanism, the modeling ability of spatiotemporal dependencies is enhanced, and the accuracy of air quality extreme value prediction is significantly improved.

[0105] Difference: Traditional Transformer has the disadvantage of missing contextual information, while CRIformer's improved attention mechanism fills this deficiency, making it more accurate in processing time series data.

[0106] (3) Innovative applications of CTT structure

[0107] The core structure of CRIformer is a CTT structure that combines a convolutional neural network (CNN) with a dual Transformer, which can simultaneously extract local features (through CNN) and global dependencies (through Transformer) of data. This structure establishes a strong connection between feature extraction and time series prediction, and is particularly suitable for processing multivariate time series characteristics in air quality data.

[0108] Difference: Compared with models that only use CNN or Transformer, CRIformer's CTT structure fully utilizes the advantages of both and has significant improvements in modeling long and short time series features.

[0109] (4) Enhance the ability to predict extreme air quality values

[0110] CRIformer proposes an optimized mechanism for extreme events (such as pollution peaks) in air quality prediction. Through attention modules and feature learning methods, it significantly improves the prediction accuracy of extreme data points.

[0111] Difference: Compared with existing models, CRIformer performs better in capturing mutation features and extreme events in data.

[0112] (5) Adaptability and efficiency

[0113] While enhancing prediction accuracy, CRIformer optimizes the model's computational efficiency and adaptability, making it suitable for prediction scenarios in different regions and with different pollution characteristics.

[0114] Difference: Traditional models often have the problem of insufficient adaptability when dealing with multiple regions or multiple variables, while CRIformer provides a more generalizable solution through dynamic segmentation and joint learning mechanisms.

[0115] In order to implement the method of an embodiment of the present invention, an embodiment of the present invention also provides an air quality prediction system, including: a processor and a memory for storing a computer program that can be run on the processor; wherein, when the processor is used to run the computer program, the steps of the above-mentioned method are executed.

[0116] The above-mentioned system provided in this embodiment belongs to the same concept as the above-mentioned method embodiment. The specific implementation process thereof is detailed in the method embodiment and will not be repeated here.

[0117] In order to implement the method of the embodiment of the present invention, the embodiment of the present invention also provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of the above method.

[0118] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiment of the present invention, the embodiment of the present invention further provides an electronic device (computer device). Specifically, in one embodiment, the computer device may be a terminal, and its internal structure diagram may be as follows: Figure 7As shown. The computer device includes a processor A01, a network interface A02, a display screen A04, an input device A05 and a memory (not shown in the figure) connected through a system bus. Among them, the processor A01 of the computer device is used to provide computing and control capabilities. The memory of the computer device includes an internal memory A03 and a non-volatile storage medium A06. The non-volatile storage medium A06 stores an operating system B01 and a computer program B02. The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 in the non-volatile storage medium A06. The network interface A02 of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor A01, the method of any one of the above embodiments is implemented. The display screen A04 of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device A05 of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0119] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0120] The device provided by the embodiment of the present invention includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, the method of any one of the above embodiments is implemented.

[0121] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0122] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0123] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0124] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0125] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0126] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0127] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0128] It can be understood that the memory of the embodiment of the present invention can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAMbus random access memory (DRRAM, Direct Rambus Random Access Memory).The memories described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memories.

[0129] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0130] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. An air quality prediction method, characterized in that: The method comprises: Get air quality dataset; Constructing a correlation segmentation and reorganization sorting interactive convolution transformer model; the correlation segmentation and reorganization sorting interactive convolution transformer model is a CTT structure, including a CNN convolution neural network model, two Transformer models and a fully connected layer; the CNN convolution neural network model is used to extract local features of input data; the Transformer model is used to extract global dependency features of input data; the fully connected layer is used to output air quality prediction results according to the local features and the global dependency features; Inputting the data in the air quality data set into the correlation segmentation and reorganization sorting interactive convolution transformer model to obtain the air quality prediction result output by the correlation segmentation and reorganization sorting interactive convolution transformer model; The CNN convolutional neural network model includes an Attention-split layer, an MBConv module, a CRINet model, a CRI-Attention layer and a connection layer; the Transformer model includes a CRI-Attention layer; the Attention-split layer is used to divide the input data into subsequences with high correlation and subsequences with low correlation according to the correlation level of the input data, input the subsequences with high correlation into the MBConv module, and input the subsequences with low correlation into the CRINet model; the CRI-Attention layer is used to convert the input data into queries and keys using a CRINet model with a step size of k, capture local context information in the input data based on the converted queries and keys, and enhance the modeling ability of the correlation segmentation and reorganization sorting interactive convolutional transformer model for spatiotemporal dependencies.

2. The air quality prediction method according to claim 1, characterized in that: The Attention-split layer is specifically used to calculate the attention score of each feature in the input data relative to the variable target by using the self-attention mechanism; and sort the features of the input data in descending order of the attention scores; Obtain the median value in the sorting result; divide the features before the median value in the sorting result into subsequences with high correlation; divide the features after the median value in the sorting result into subsequences with low correlation; input the subsequences with high correlation into the MBConv module, and input the subsequences with low correlation into the CRINet model.

3. The air quality prediction method according to claim 1, characterized in that: The MBConv module is used to extract key features from the highly correlated subsequences and enhance the expressive power of the correlation segmentation and reorganization sorting interactive convolution transformer model for highly correlated variables.

4. The air quality prediction method according to claim 1, characterized in that: The CRINet model is used to perform feature learning on the subsequences with low correlation, and enhance the modeling capability of the correlation segmentation and reorganization sorting interactive convolutional transformer model for complex relationships.

5. The air quality prediction method according to claim 1, characterized in that: The connection layer is used to perform a first fusion of the features output by the MBConv module and the features output by the CRINet model to obtain a first fusion feature; The first fused feature is fused for a second time with the feature output by the CRI-Attention layer to obtain a second fused feature.

6. An air quality prediction system, characterized in that: include: A processor and a memory for storing a computer program that can be run on the processor; wherein the processor, when used to run the computer program, executes the steps of the method according to any one of claims 1 to 5.

7. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images