Method, device and equipment for predicting concentration of inhalable particles based on meteorological factors

Abstract

The invention discloses a meteorological factor-based inhalable particle concentration prediction method, device and equipment, and the method comprises the steps: decomposing a time domain signal of original meteorological factor data into a frequency domain signal, and obtaining a frequency domain feature representation; independently embedding a time sequence of each meteorological factor in the frequency domain feature representation into a token to obtain a global feature representation of each meteorological factor; performing feature interaction on the global feature representation of each meteorological factor through a residual attention mechanism of a DF-Net network, capturing complex association among the meteorological factors, and obtaining feature representation after interaction; performing feature selection on the interacted feature representations through a self-adaptive gating unit of the DF-Net network, and dynamically adjusting feature importance of each meteorological factor to obtain optimized feature representations; and summarizing and mapping the optimized feature representation through a full connection layer of the DF-Net network to generate a predicted value of the inhalable particle concentration. According to the invention, the accuracy of inhalable particle concentration prediction can be improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device and equipment for predicting the concentration of inhalable particulate matter based on meteorological factors. Background Art

[0002] In the field of prediction of inhalable particulate matter concentration, existing technologies are not effective in dealing with inhalable particulate matter under complex meteorological conditions (e.g. ) There are many deficiencies in the concentration prediction.

[0003] Traditional prediction methods are mostly based on machine learning and traditional regression models. For example, some studies use the support vector regression (SVR) model to predict features through wavelet transform and partial least squares method. daily concentration. However, this method is highly dependent on data quality, and noise and outliers in the data will significantly affect the prediction accuracy. Moreover, the feature fusion process is complex, and feature selection and fusion methods are difficult to be universal in different regions and meteorological conditions, which increases the limitations of model application.

[0004] There is also a method that combines the grey Markov model and the land use regression model to try to predict the However, the grey Markov model has high requirements on the stability of data, while the actual meteorological data and The concentration data is affected by many factors and often presents non-stationary and non-linear characteristics, which makes it difficult for the model to accurately capture the real trend of the data. At the same time, the land use regression model is insufficient in considering the dynamic changes of meteorological factors and cannot timely reflect the real-time impact of meteorological conditions on concentration.

[0005] Although linear regression models and random forest models have been used in hierarchical classification prediction, they are not suitable for dealing with complex meteorological data and The ability of the random forest model is limited when it comes to the relationship between meteorological factors and concentration. The linear regression model assumes that there is a linear relationship between variables, which is inconsistent with the actual situation. Meteorological factors and concentrations often present a complex nonlinear relationship. Although the random forest model can handle nonlinear problems to a certain extent, it is prone to overfitting when faced with high-dimensional, multivariate meteorological data, resulting in poor generalization of the model.

[0006] With the development of deep learning, some deep learning-based models have been applied to Concentration prediction. For example, a model combining the long short-term memory network (LSTM) and the convolutional neural network (CNN) uses the time series prediction ability of LSTM and the feature extraction ability of CNN to process air pollution data. However, although LSTM can solve the gradient vanishing problem to a certain extent when processing long time series data, its ability to capture long-distance dependencies still needs to be improved. When the time series is too long, early important information may be gradually lost during the propagation process, affecting the accurate prediction of long-term trends. When processing time series data, CNN focuses more on the extraction of local features and does not adequately mine the global dependencies between different time steps.

[0007] The Transformer model and its variants have advantages in processing long sequence data, and can better extract global and local information through the multi-head attention mechanism. However, when processing meteorological data, there is still a problem of insufficient modeling of spatiotemporal dynamic changes. For example, some sudden changes in meteorological data, such as extreme weather events such as strong winds and precipitation in a short period of time, these models may not be able to capture their impact on concentration in a timely and accurate manner. Moreover, some models based on the Transformer improvement also have defects in capturing basic sequence representations and depicting multivariate correlations, which makes it impossible to fully explore the relationship between meteorological factors and The complex relationship between concentrations.

[0008] In summary, the existing inhalable particulate matter concentration prediction technology has problems such as high data quality requirements, difficulty in handling nonlinear and non-stationary data, and inability to effectively capture long-distance dependencies and spatiotemporal dynamic changes when processing data under complex meteorological conditions, which affects the accuracy of inhalable particulate matter concentration prediction. Summary of the invention

[0009] The embodiments of the present invention provide a method, device and equipment for predicting the concentration of inhalable particulate matter based on meteorological factors, which can improve the accuracy of the prediction of the concentration of inhalable particulate matter.

[0010] An embodiment of the present invention provides a method for predicting the concentration of inhalable particulate matter based on meteorological factors, comprising: The original meteorological factor data is standardized, and the time domain signal is decomposed into the frequency domain signal through Fourier transform to obtain the frequency domain feature representation; The time series of each meteorological factor in the obtained frequency domain feature representation is independently embedded as a token to obtain the global feature representation of each meteorological factor; For the global feature representation of each meteorological factor, the residual attention mechanism of the DF-Net network is used to perform feature interaction, capture the complex correlation between meteorological factors, and obtain the feature representation after interaction; The obtained interactive feature representation is subjected to feature selection through the adaptive gating unit of the DF-Net network, and the feature importance of each meteorological factor is dynamically adjusted to obtain the optimized feature representation; The optimized feature representation is summarized and mapped through the fully connected layer of the DF-Net network to generate the predicted value of the inhalable particulate matter concentration.

[0011] As an improvement of the above scheme, the original meteorological factor data is standardized, and the time domain signal is decomposed into a frequency domain signal by Fourier transform to obtain a frequency domain feature representation, including: The concentration time series of inhalable particulate matter in the original meteorological factor data is standardized to obtain a standardized concentration time series of inhalable particulate matter; Performing Fourier transform on the standardized time series of the concentration of inhalable particulate matter, decomposing the time domain signal into a frequency domain signal, and obtaining a frequency domain representation of the concentration of inhalable particulate matter; The frequency domain representation of the inhalable particulate matter concentration is decomposed into multiple frequency components, each frequency component corresponds to a sine wave of a different frequency, and a simplified frequency domain signal is obtained as a frequency domain feature representation.

[0012] As an improvement of the above scheme, the time series of each meteorological factor in the obtained frequency domain feature representation is independently embedded as a token to obtain a global feature representation of each meteorological factor, including: For each meteorological factor in the frequency domain feature representation, its entire time series is considered as a whole; The time series of each meteorological factor is embedded as a whole, and it is embedded into tokens independently, and finally the global feature representation of each meteorological factor is obtained.

[0013] As an improvement of the above scheme, the global feature representation of each meteorological factor is obtained, and feature interaction is performed through the residual attention mechanism of the DF-Net network to capture the complex correlation between meteorological factors and obtain the feature representation after interaction, including: The global feature representation of each meteorological factor is input into the residual attention mechanism of the DF-Net network; The residual attention mechanism of the DF-Net network is used to perform feature interaction processing on the global feature representation of each input meteorological factor, capture the complex correlation between meteorological factors, and obtain the interactive feature representation.

[0014] As an improvement of the above scheme, the obtained interactive feature representation is subjected to feature selection through the adaptive gating unit of the DF-Net network, and the feature importance of each meteorological factor is dynamically adjusted to obtain an optimized feature representation, including: Input the interacted feature representation into the adaptive gating unit of the DF-Net network; The adaptive gating unit of the DF-Net network is used to select the interactive feature representation and dynamically adjust the feature importance of each meteorological factor. Based on feature selection and importance adjustment operations, an optimized feature representation is obtained.

[0015] As an improvement to the above scheme, the optimized feature representation is summarized and mapped through the fully connected layer of the DF-Net network to generate a predicted value of the inhalable particulate matter concentration, including: Inputting the optimized feature representation into the fully connected layer of the DF-Net network; The multi-layer structure of the fully connected layer of the DF-Net network is used to summarize the optimized feature representation of the input to integrate the optimized feature information of each meteorological factor; Through the linear transformation of the fully connected layer, the summarized feature representation is mapped to the output space to obtain the predicted value of the inhalable particulate matter concentration.

[0016] Another embodiment of the present invention provides a device for predicting the concentration of inhalable particulate matter based on meteorological factors, comprising: The preprocessing module is used to standardize the original meteorological factor data and decompose the time domain signal into the frequency domain signal through Fourier transform to obtain the frequency domain feature representation; An embedding module is used to embed the time series of each meteorological factor in the obtained frequency domain feature representation into tokens independently to obtain a global feature representation of each meteorological factor; The interaction module is used to obtain the global feature representation of each meteorological factor, perform feature interaction through the residual attention mechanism of the DF-Net network, capture the complex correlation between meteorological factors, and obtain the feature representation after interaction; The adjustment module is used to select features of the obtained interactive feature representation through the adaptive gating unit of the DF-Net network, dynamically adjust the feature importance of each meteorological factor, and obtain the optimized feature representation; The generation module is used to summarize and map the optimized feature representation through the fully connected layer of the DF-Net network to generate the predicted value of the inhalable particulate matter concentration.

[0017] As an improvement of the above solution, the preprocessing module is specifically used for: The concentration time series of inhalable particulate matter in the original meteorological factor data is standardized to obtain a standardized concentration time series of inhalable particulate matter; Performing Fourier transform on the standardized time series of the concentration of inhalable particulate matter, decomposing the time domain signal into a frequency domain signal, and obtaining a frequency domain representation of the concentration of inhalable particulate matter; The frequency domain representation of the inhalable particulate matter concentration is decomposed into multiple frequency components, each frequency component corresponds to a sine wave of a different frequency, and a simplified frequency domain signal is obtained as a frequency domain feature representation.

[0018] As an improvement of the above solution, the embedded module is specifically used for: For each meteorological factor in the frequency domain feature representation, its entire time series is considered as a whole; The time series of each meteorological factor is embedded as a whole, and it is embedded into tokens independently, and finally the global feature representation of each meteorological factor is obtained.

[0019] Another embodiment of the present invention provides an inhalable particle concentration prediction device based on meteorological factors, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, the inhalable particle concentration prediction method based on meteorological factors described in the above-mentioned embodiment of the invention is implemented.

[0020] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: Firstly, the original meteorological factor data is standardized, and the time domain signal is decomposed into frequency domain signal by Fourier transform to obtain more representative frequency domain feature representation, effectively eliminate redundant information and extract key features; secondly, the time series of each meteorological factor in the frequency domain feature representation is independently embedded as a token to obtain the global feature representation of each meteorological factor, avoiding the problems of noise and information loss in traditional multi-factor mixed modeling; then, the residual attention mechanism of the DF-Net network is used to perform feature interaction on the global feature representation of each meteorological factor to capture the complex correlation between meteorological factors, and at the same time, the jump connection is used to enhance the gradient fluidity and solve the gradient disappearance problem in the deep network; then, the adaptive gating unit of the DF-Net network is used to select the feature representation after interaction, dynamically adjust the feature importance of each meteorological factor, suppress the propagation of invalid information, and improve the accuracy and robustness of feature extraction; finally, the optimized feature representation is summarized and mapped through the fully connected layer of the DF-Net network to generate the predicted value of the inhalable particulate matter concentration. From the above analysis, it can be seen that the embodiments of the present invention can efficiently process multivariate time series data under complex meteorological conditions, accurately capture nonlinear characteristics, and significantly improve the accuracy of inhalable particulate matter concentration prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1It is a flow chart of a method for predicting the concentration of inhalable particulate matter based on meteorological factors provided by one embodiment of the present invention; Figure 2 is a schematic diagram of a DF-Net model provided by an embodiment of the present invention; Figure 3 It is a schematic diagram of the DF-Net architecture provided by an embodiment of the present invention; Figure 4 is the preprocessed data provided by an embodiment of the present invention. Schematic diagram of concentration curve; Figure 5 is a schematic diagram of the importance of meteorological factor characteristics provided by an embodiment of the present invention; Figure 6 It is a schematic diagram comparing the prediction values ​​of the DF-Net model provided by one embodiment of the present invention and other models; Figure 7 It is a schematic diagram of the structure of a device for predicting the concentration of inhalable particulate matter based on meteorological factors provided by one embodiment of the present invention; Figure 8 It is a schematic diagram of the structure of an inhalable particulate matter concentration prediction device based on meteorological factors provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] See also Figure 1 , is a flow chart of a method for predicting the concentration of inhalable particulate matter based on meteorological factors provided by an embodiment of the present invention. The method for predicting the concentration of inhalable particulate matter based on meteorological factors includes: The original meteorological factor data is standardized, and the time domain signal is decomposed into the frequency domain signal through Fourier transform to obtain the frequency domain feature representation; The time series of each meteorological factor in the obtained frequency domain feature representation is independently embedded as a token to obtain the global feature representation of each meteorological factor; For the global feature representation of each meteorological factor, the residual attention mechanism of the DF-Net network is used to perform feature interaction, capture the complex correlation between meteorological factors, and obtain the feature representation after interaction; The obtained interactive feature representation is subjected to feature selection through the adaptive gating unit of the DF-Net network, and the feature importance of each meteorological factor is dynamically adjusted to obtain the optimized feature representation; The optimized feature representation is summarized and mapped through the fully connected layer of the DF-Net network to generate the predicted value of the inhalable particulate matter concentration.

[0024] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: Firstly, the original meteorological factor data is standardized, and the time domain signal is decomposed into frequency domain signal by Fourier transform to obtain more representative frequency domain feature representation, effectively eliminate redundant information and extract key features; secondly, the time series of each meteorological factor in the frequency domain feature representation is independently embedded as a token to obtain the global feature representation of each meteorological factor, avoiding the problems of noise and information loss in traditional multi-factor mixed modeling; then, the residual attention mechanism of the DF-Net network is used to perform feature interaction on the global feature representation of each meteorological factor to capture the complex correlation between meteorological factors, and at the same time, the jump connection is used to enhance the gradient fluidity and solve the gradient disappearance problem in the deep network; then, the adaptive gating unit of the DF-Net network is used to select the feature representation after interaction, dynamically adjust the feature importance of each meteorological factor, suppress the propagation of invalid information, and improve the accuracy and robustness of feature extraction; finally, the optimized feature representation is summarized and mapped through the fully connected layer of the DF-Net network to generate the predicted value of the inhalable particulate matter concentration. From the above analysis, it can be seen that the embodiments of the present invention can efficiently process multivariate time series data under complex meteorological conditions, accurately capture nonlinear characteristics, and significantly improve the accuracy of inhalable particulate matter concentration prediction.

[0025] As an example, the method of normalizing the original meteorological factor data and decomposing the time domain signal into the frequency domain signal by Fourier transform to obtain the frequency domain feature representation includes: The concentration time series of inhalable particulate matter in the original meteorological factor data is standardized to obtain a standardized concentration time series of inhalable particulate matter; Performing Fourier transform on the standardized time series of the concentration of inhalable particulate matter, decomposing the time domain signal into a frequency domain signal, and obtaining a frequency domain representation of the concentration of inhalable particulate matter; The frequency domain representation of the inhalable particulate matter concentration is decomposed into multiple frequency components, each frequency component corresponds to a sine wave of a different frequency, and a simplified frequency domain signal is obtained as a frequency domain feature representation.

[0026] In this embodiment, the time domain signal is decomposed into a frequency domain signal by standardizing and Fourier transforming the original meteorological factor data, thereby obtaining a frequency domain feature representation. This process first standardizes the time series of the inhalable particle concentration in the original meteorological factor data to eliminate the dimensional influence and numerical differences of different meteorological factors, so that the data is comparable. Then, the standardized time series of the inhalable particle concentration is Fourier transformed to convert the time domain signal into a frequency domain signal, extract the frequency component and amplitude information of the signal, and obtain the frequency domain representation of the inhalable particle concentration. Finally, the frequency domain representation of the inhalable particle concentration is decomposed into multiple frequency components, each frequency component corresponds to a sine wave of different frequencies, and a simplified frequency domain signal is obtained as a frequency domain feature representation. This embodiment can effectively extract the frequency characteristics of the inhalable particle concentration, provide more abundant information for subsequent predictions, and thus improve the accuracy and reliability of the prediction.

[0027] Specifically, the concentration time series of inhalable particulate matter in the original meteorological factor data usually has dimension differences and value fluctuations due to factors such as measurement units and measurement environment. Common standardization methods can be used, such as z-score standardization (formula is ,in is the original data value, is the mean of the data, is the standard deviation of the data) to process it. By calculating the mean and standard deviation of the time series, each data point is converted according to the above formula to obtain the standardized time series of inhalable particulate matter concentration, making the data distribution more concentrated and facilitating subsequent processing. Fourier transform: For the standardized time series of inhalable particulate matter concentration, Fourier transform is used to convert it from time domain to frequency domain. In practical applications, the fast Fourier transform (FFT) algorithm can be used to improve computational efficiency. FFT is an algorithm for efficiently calculating discrete Fourier transform (DFT). It reduces the computational complexity of DFT from Reduce to ( is the number of data points). After FFT processing, the time series is decomposed into a superposition of sine waves of different frequencies. Each sine wave has a corresponding frequency, amplitude and phase, and the frequency domain representation of the inhalable particle concentration is obtained, so that the distribution of the data at different frequencies can be clearly seen. Decompose the frequency domain representation: decompose the frequency domain representation of the inhalable particle concentration into multiple frequency components. Each frequency component corresponds to a sine wave of a specific frequency, which can be divided according to methods such as frequency range or amplitude threshold. For example, an amplitude threshold is set, and the frequency components with amplitudes lower than the threshold are regarded as noise and removed, retaining the main frequency components with larger amplitudes. After screening and processing, a simplified frequency domain signal is obtained. This simplified signal removes most of the redundant information and only retains the key frequency features that are closely related to the changes in the inhalable particle concentration. It is used as a frequency domain feature representation to provide a more representative input for subsequent models.

[0028] For example, in an air quality monitoring scenario in an industrial city, the environmental protection department continuously records the amount of inhalable particulate matter (such as ) to form the time series of inhalable particulate matter concentration in the original meteorological factor data. These data may be affected by many factors such as different seasons, different industrial production activities and meteorological conditions, and the numerical range and fluctuation of the data vary greatly. First, these original inhalable particulate matter concentration time series are standardized. For example, a monitoring station records The concentration data is converted into a standardized time series by calculating the mean and standard deviation according to the z-score standardization formula, so that the data from different monitoring sites can be compared on the same scale. Then, the standardized time series is subjected to fast Fourier transform. The FFT algorithm is used to convert the concentration data within a week into a standardized time series. The concentration time series was converted to the frequency domain, and several obvious frequency peaks were found. Further analysis showed that one of the frequencies corresponds to the weekly working day-rest day cycle, which may be related to the difference in industrial production activities on working days and rest days; the other frequency corresponds to the diurnal variation cycle, which may be related to factors such as the day and night temperature difference and human activity patterns. Finally, the frequency domain representation is decomposed into multiple frequency components. An amplitude threshold is set to remove frequency components with smaller amplitudes, and the main frequency components corresponding to the working day-rest day cycle and the diurnal variation cycle are retained to obtain a simplified frequency domain signal. This simplified frequency domain signal is input into the inhalable particulate matter concentration prediction model based on meteorological factors as a frequency domain feature representation. Through the analysis and prediction of the model, the environmental protection department can understand in advance the concentration of inhalable particulate matter in different time periods. The concentration trend can be monitored to develop more targeted pollution prevention and control measures, such as strengthening emission supervision of industrial enterprises during specific time periods on weekdays and appropriately increasing road sweeping and watering operations at night to reduce dust.

[0029] As an example, the time series of each meteorological factor in the obtained frequency domain feature representation is independently embedded as a token to obtain a global feature representation of each meteorological factor, including: For each meteorological factor in the frequency domain feature representation, its entire time series is considered as a whole; The time series of each meteorological factor is embedded as a whole, and it is embedded into tokens independently, and finally the global feature representation of each meteorological factor is obtained.

[0030] In this embodiment, the global feature representation of each meteorological factor is obtained by independently embedding the time series of each meteorological factor in the frequency domain feature representation. This process first regards the time series of each meteorological factor in the frequency domain feature representation as a whole, and then embeds the time series of each meteorological factor as a whole, and embeds it independently as a token. This independent embedding method enables the model to model the global characteristics of different meteorological factors separately, avoiding the noise and information loss problems caused by the mixing of multiple factors in traditional models. In this way, the model can more effectively capture the global characteristics of each meteorological factor, thereby improving the accuracy and reliability of the prediction.

[0031] Specifically, the meteorological factor time series is considered as a whole: after obtaining the frequency domain feature representation, each meteorological factor is traversed. This requires a clear understanding and parsing ability of the data structure of the frequency domain feature representation, and the ability to accurately identify the time series data corresponding to each meteorological factor. For example, if the frequency domain feature representation is stored in the form of a matrix, different columns may represent the time series of different meteorological factors. Through indexing and data reading operations, the time series corresponding to each column is completely extracted and regarded as an indivisible whole, ensuring that subsequent operations can process the information of the meteorological factor from a global perspective. The embedding operation obtains tokens and global feature representations: The embedding operation is performed on the time series of each meteorological factor as a whole, which is usually implemented using an embedding layer. The embedding layer can map the input time series data into a low-dimensional and more expressive vector space. In practical applications, the weight matrix of the embedding layer is learned through model training. Assuming that the input meteorological factor time series data is a series of discrete values, the embedding layer will find the corresponding vector representation in the weight matrix based on these values, thereby converting the time series into a set of continuous vectors, that is, independently embedded as tokens. In this way, each meteorological factor is converted into a unique token that contains the global information of its time series, and finally a global feature representation of each meteorological factor is obtained.

[0032] Take the air quality monitoring scenario of a city on the edge of the desert as an example. The concentration of inhalable particulate matter in the city is affected by multiple meteorological factors, such as wind speed, wind direction, humidity, temperature, etc. After obtaining the frequency domain feature representation containing these meteorological factors, each meteorological factor is processed. For the meteorological factor of wind speed, its time series data records the changes in wind speed at different times. The entire time series of wind speed is extracted and processed as a whole. In the model, the wind speed time series is embedded through the embedding layer. Assuming that the weight matrix of the embedding layer has been determined in the previous training, each data point in the wind speed time series will be mapped to a vector according to this weight matrix. These vectors are combined to form a token representing the wind speed, that is, the global feature representation of the wind speed. This token contains global information such as the change trend and fluctuation of the wind speed at different time points. Similarly, similar operations are performed on other meteorological factors such as wind direction, humidity, and temperature. In this way, each meteorological factor exists in the form of an independent token, and the model can analyze them separately. For example, the impact of changes in wind speed in a day on the concentration of inhalable particulate matter can be studied separately without being interfered by other meteorological factors, providing a more accurate basis for the subsequent prediction of the concentration of inhalable particulate matter.

[0033] As an example, the obtained global feature representation of each meteorological factor is subjected to feature interaction through the residual attention mechanism of the DF-Net network to capture the complex associations between meteorological factors and obtain the feature representation after interaction, including: The global feature representation of each meteorological factor is input into the residual attention mechanism of the DF-Net network; The residual attention mechanism of the DF-Net network is used to perform feature interaction processing on the global feature representation of each input meteorological factor, capture the complex correlation between meteorological factors, and obtain the interactive feature representation.

[0034] In this embodiment, feature interaction is performed on the global feature representation of each meteorological factor to capture the complex associations between meteorological factors, thereby obtaining the feature representation after interaction. This process first inputs the global feature representation of each meteorological factor into the residual attention mechanism of the DF-Net network, and then uses the residual attention mechanism to perform feature interaction processing on the input feature representation. The residual attention mechanism can effectively solve the gradient vanishing problem in deep neural networks, while improving computational efficiency, so that the model can better capture the complex associations between meteorological factors. In this way, the model can more accurately understand the interactions between different meteorological factors, thereby improving the accuracy and reliability of the prediction.

[0035] Specifically, input the global feature representation: after obtaining the global feature representation of each meteorological factor, these feature representations are input into the residual attention mechanism in order according to the input specification of the DF-Net network. Assuming that the global feature representation of each meteorological factor is stored in the form of a tensor, these tensors will be combined into a batch of data in a certain order, and then input as a whole into the network layer where the residual attention mechanism is located. In this process, it is necessary to ensure that the dimension and shape of the data meet the requirements of the residual attention mechanism for subsequent processing. For example, if the residual attention mechanism expects the input tensor dimension to be [batch size, sequence length, feature dimension], then the global feature representation of each meteorological factor needs to be adjusted and combined accordingly. Feature interaction processing: The residual attention mechanism of the DF-Net network mainly realizes feature interaction and captures complex associations through multi-head attention mechanism and jump connection. The multi-head attention mechanism projects the input feature representation into multiple low-dimensional subspaces, each subspace corresponding to a "head". In each "head", the query (Query), key (Key) and value (Value) matrices are calculated respectively, and the formula is used Calculate the attention score, where They are query and key matrices for the layers, is the dimension of the key matrix. Then the Softmax function is used to normalize the attention scores to obtain the weight distribution, which is then multiplied by the value matrix to obtain the output of each "head". Finally, the outputs of multiple "heads" are concatenated to obtain the result of multi-head attention. At the same time, the residual attention mechanism also introduces skip connections. When calculating the attention score, the attention score of this layer is added to the attention score of the previous layer (the formula is ), which allows the gradient to propagate more smoothly, avoids the gradient vanishing problem, and also helps capture long-distance dependencies. Through these operations, feature interaction processing of the global feature representation of meteorological factors is realized, the complex associations between meteorological factors are captured, and finally the feature representation after interaction is obtained.

[0036] For example, the concentration of inhalable particulate matter during winter heating is affected by a variety of meteorological factors. The meteorological monitoring station continuously collects meteorological data such as wind speed, temperature, humidity, and air pressure, and obtains the global feature representation of each meteorological factor through preliminary processing. These global feature representations are input into the residual attention mechanism of the DF-Net network. In this process, different "heads" of the multi-head attention mechanism focus on different aspects of information. For example, one "head" may focus more on the relationship between wind speed and temperature, and find that when the wind speed is low and the temperature drops sharply, it may be due to insufficient combustion of the heating boiler, resulting in increased inhalable particulate matter emissions; another "head" focuses on the relationship between humidity and air pressure, and finds that when the humidity is high and the air pressure is low, it is not conducive to the diffusion of pollutants, which will increase the concentration of inhalable particulate matter. At the same time, the jump connection enables the model to remember the relationship between meteorological factors at previous moments. Combining this information, the residual attention mechanism captures the complex interaction between various meteorological factors. The final interactive feature representation contains more comprehensive and accurate information reflecting the impact of meteorological factors on the concentration of inhalable particulate matter, providing key support for the subsequent accurate prediction of the concentration of inhalable particulate matter.

[0037] As an example, the obtained interactive feature representation is subjected to feature selection by the adaptive gating unit of the DF-Net network, and the feature importance of each meteorological factor is dynamically adjusted to obtain an optimized feature representation, including: Input the interacted feature representation into the adaptive gating unit of the DF-Net network; The adaptive gating unit of the DF-Net network is used to select the interactive feature representation and dynamically adjust the feature importance of each meteorological factor. Based on feature selection and importance adjustment operations, an optimized feature representation is obtained.

[0038] In this embodiment, it is considered that under complex meteorological conditions, different meteorological factors have different degrees of influence on the concentration of inhalable particulate matter, and there is a problem that some redundant or invalid information interferes with the prediction accuracy. By inputting the interactive feature representation into the adaptive gating unit of the DF-Net network, using its gating mechanism, according to the actual effect of the meteorological factor characteristics, targeted screening of effective information and suppression of the transmission of useless information. This technical solution can dynamically adjust the importance of each meteorological factor feature, optimize the feature combination of the model input, and focus the model on key features, thereby improving the accuracy and robustness of feature extraction, so as to generate a more accurate prediction value of the concentration of inhalable particulate matter, thereby improving the performance of the prediction model in complex meteorological environments.

[0039] Specifically, first, the interactive feature representation obtained after processing by the DF-Net network residual attention mechanism is sorted and transmitted according to the input requirements of the adaptive gating unit. If the interactive feature representation is a multi-dimensional tensor, it is necessary to ensure that its dimension, data type, etc. are consistent with the input format expected by the adaptive gating unit. For example, if the input requirement of the adaptive gating unit is in the form of a tensor of [batch size, number of features, feature dimension], the interactive feature representation needs to be reshaped or formatted accordingly so that it can smoothly enter the adaptive gating unit for subsequent processing. The adaptive gating unit (GLU) divides the input interactive feature representation into two parts for processing. One part is directly transmitted through linear transformation, and the other part generates a gating value through the Sigmoid activation function. Specifically, for the input feature , GLU will split it into and , After linear transformation , After Sigmoid activation function Get the gate value ,in, is the weight matrix, is the bias term. Then, the result of the linear transformation is multiplied element by element with the gate value, that is, , in order to dynamically adjust the importance of each meteorological factor feature in subsequent processing. In this way, for meteorological factor features that have a greater impact on the concentration of inhalable particulate matter, the gating value will be close to 1, so that it can retain more information; while for features with less impact or redundant features, the gating value will be close to 0, suppressing their information transmission, thereby achieving feature selection and importance adjustment. After the above feature selection and importance adjustment operations, the result output by the adaptive gating unit is the optimized feature representation. This optimized feature representation removes most of the invalid or interfering information, highlights the meteorological factor features that are important for the prediction of inhalable particulate matter concentration, and provides better quality input data for the subsequent fully connected layer processing, which helps to improve the accuracy of the prediction.

[0040] As an example, the optimized feature representation obtained is summarized and mapped through the fully connected layer of the DF-Net network to generate a predicted value of the inhalable particulate matter concentration, including: Inputting the optimized feature representation into the fully connected layer of the DF-Net network; The multi-layer structure of the fully connected layer of the DF-Net network is used to summarize the optimized feature representation of the input to integrate the optimized feature information of each meteorological factor; Through the linear transformation of the fully connected layer, the summarized feature representation is mapped to the output space to obtain the predicted value of the inhalable particulate matter concentration.

[0041] In the present embodiment, after a series of processing of the original meteorological factor data, including standardization, Fourier transform, feature embedding, feature interaction and feature selection, an optimized feature representation is obtained. However, these features are scattered and abstract, and it is difficult to be directly used for the prediction of the concentration of inhalable particles. Therefore, the DF-Net network fully connected layer is used to first summarize and integrate these scattered optimized features, and then map them to the actual numerical space of the concentration of inhalable particles through linear transformation, so as to convert the complex meteorological factor features into intuitive and practical reference inhalable particle concentration prediction values. Its technical effect is to make full use of the effective feature information obtained by the previous processing, and through the summary and mapping of the fully connected layer, the accurate conversion from complex meteorological factor features to specific prediction values ​​is realized, and the accuracy and practicality of the prediction of the concentration of inhalable particles are improved.

[0042] Specifically, after completing the feature selection and importance adjustment of the interactive feature representation through the adaptive gating unit of the DF-Net network, the optimized feature representation is obtained. At this time, the optimized feature representation is passed according to the input requirements of the fully connected layer of the DF-Net network. Specifically, if the optimized feature representation is a multi-dimensional tensor, ensure that its dimension and data type match the input specification of the fully connected layer. For example, if the fully connected layer expects a two-dimensional tensor with a shape of [number of samples, number of features], the optimized feature representation needs to be dimensionally adjusted and format converted accordingly to ensure that the data can smoothly enter the fully connected layer for subsequent processing.

[0043] Summarize optimized feature information: The fully connected layer of the DF-Net network has a multi-layer structure. When the optimized feature representation is input to the fully connected layer, the neurons in each layer are connected to all neurons in the previous layer. For the input optimized feature representation, the first layer of neurons will perform a weighted summation of its various feature dimensions, that is, each neuron linearly combines the input features according to its own weight. Assume that the input feature is , the weight of the neuron in the first layer is , the bias is , then the output of the neuron for: In this way, the first layer of neurons makes a preliminary summary of the input optimization features. As the data is transmitted layer by layer in the multi-layer structure, each layer will perform a similar weighted summation operation on the output of the previous layer, and continuously integrate the optimized feature information of each meteorological factor. The calculation results of each layer will contain more comprehensive information, so that the correlation and interaction between different meteorological factors can be fully explored and reflected in this process, and finally a comprehensive summary of the optimized feature information of each meteorological factor can be achieved.

[0044] Mapping to the output space to get the predicted value: After being summarized by the multi-layer structure of the fully connected layer, the summarized feature representation is obtained. The last step of the fully connected layer is to map this summarized feature representation to the output space through a linear transformation. Specifically, the last layer of neurons will linearly combine the summarized feature representation according to their own weights and biases. Assume that the summarized feature representation is , the weight of the last layer for , the bias is , then through the linear transformation , the summarized feature representation is mapped to the numerical range of inhalable particle concentration to obtain the predicted value of inhalable particle concentration. This predicted value is obtained based on the previous processing of meteorological factors and the summary and mapping of the fully connected layer, which can reflect the inhalable particle concentration under the combined effect of various meteorological factors.

[0045] In order to facilitate understanding of the above embodiments, the relevant technical process is specifically described by way of example: 1.1 Data Source The meteorological elements of the meteorological data used in this embodiment include relative humidity (RHU), wind speed (WS), wind direction (WD), air pressure (PRS), temperature (TEM), and horizontal visibility (VIS).

[0046] 1.2 Data Preprocessing because Concentration data has the characteristic of cyclical fluctuations over time. The Fourier transform method is used to decompose the time domain signal into the frequency domain signal. The main function is to decompose a complex time domain signal into simple sine waves of different frequencies. These sine waves have different frequencies, amplitudes and phases, making feature analysis and extraction more intuitive and efficient. The concentration value is obtained by Fourier transform to obtain its frequency domain component, and x(t) is the original The concentration time series is first subjected to Fourier transform to obtain its frequency domain representation X(f). The mathematical expression of Fourier transform is:

[0047] In the formula, It's the frequency. is a complex exponential function that projects the signal to frequency On, among which is an imaginary unit. Decompose into multiple frequency domain components . It can be expressed as: (2) In the formula, is the simplified frequency domain signal, including frequency components. is the Dirac Delta function, which represents the frequency Only in The value is 1 when , otherwise it is 0.

[0048] Next, the simplified frequency domain signal Perform inverse Fourier transform to obtain the reconstructed time domain signal. The mathematical expression of inverse Fourier transform is:

[0049] In the formula, is the reconstructed time domain signal, Used to convert a signal from frequency to time domain. Each frequency component is multiplied by the corresponding complex exponential function , and then sum it up.

[0050] 1.3 Correlation analysis of meteorological characteristic factors The SHAP (SHapley Additive ex Planations) method was used to evaluate the effects of different meteorological characteristic factors on the SHAP is a tool for explaining the output of machine learning models, based on the concept of Shapley value in game theory. When a meteorological characteristic factor is added or removed from the model, its impact on the prediction result is judged. This impact is averaged over all possible feature combinations to obtain the SHAP value of the feature. In this way, the SHAP method can provide the additive contribution of each feature to the model prediction, ensuring consistency and fairness of the explanation.

[0051] First, the meteorological characteristic data and The concentration data is standardized, and then an XGBoost regression model is trained using the standardized data. The model is gradually built on a series of decision trees, and the model error is iteratively optimized for prediction. Concentration. The XGBoost model is iteratively constructed using the following formula:

[0052] In the formula, is the predicted value of the ith sample, is the kth decision tree, is the space of all possible trees. Then, the SHAP value is calculated to quantify the marginal contribution of each feature to the prediction result. , the value is calculated as follows:

[0053] In the formula, is the set of all meteorological characteristics, For the collection Remove meteorological features The subset of meteorological features selected from the subsequent set, Only a subset is used The predicted values ​​of the trained model, In the subset Add meteorological characteristics to the The model prediction value after . Indicates meteorological characteristics The weight of all possible meteorological features.

[0054] The positive and negative signs of the SHAP value indicate the direction of the impact of the meteorological feature on the forecast result. A positive SHAP value indicates that an increase in the feature will lead to an increase in the forecast value, while a negative SHAP value indicates that an increase in the feature will lead to a decrease in the forecast value. The features are then sorted according to their average absolute SHAP value (i.e., the average of the absolute values ​​of the SHAP values ​​of the meteorological feature in all samples). The larger the average absolute SHAP value, the more important the feature.

[0055] 1.4 DF-Net Architecture 1.4.1 Model Architecture The traditional Transformer assigns the same token value to each variable at the same timestamp, which will affect the extraction of correlation between variables and introduce noise. In addition, the Self-Attention and Embedding structures will also disrupt the time series information, which will also affect the prediction. For this reason, the DF-Net model architecture proposed in this embodiment will input each meteorological factor into the model as a separate whole. The model architecture diagram is shown in the figure below. Figure 2 shown.

[0056] This model regards the entire time series of a single meteorological factor as a token, aggregates the global representation of the sequence, and can pay more attention to multiple meteorological factors, so that the model can better capture the changes in meteorological factors caused by seasonal climate. And the use of the residual attention mechanism (ResidualAttentionLayer) has better performance in gradient descent and convergence, is more robust to hyperparameters during training, and can better pay attention to the relationship between multiple meteorological factors and The correlation between the concentrations.

[0057] Secondly, in order to reduce the transmission of invalid information and reduce the number of parameters, the gated linear unit (GLU) was introduced to further improve the model's expressiveness and computational efficiency. The introduction of GLU enables the model to selectively transmit valid information through the gating mechanism, avoiding unnecessary noise interference, thereby enhancing the accuracy of feature extraction. Combined with the residual attention mechanism, DF-Net can not only better capture the complex correlations between meteorological factors, but also effectively reduce computational overhead in the time series prediction of multiple meteorological factors.

[0058] In general, this architectural design not only retains the flexibility of the Transformer model, but also significantly improves the model's prediction accuracy and training stability through optimization for time series data.

[0059] The core working principle of DF-Net is to achieve the goal of Efficient modeling of concentration prediction. First, DF-Net embeds the time series of each meteorological factor independently as a token, so that the model can separately model the global characteristics of different factors, avoiding the noise and information loss problems caused by the mixing of multiple factors in traditional models. Secondly, through the residual attention mechanism, the model can effectively capture the complex correlations between meteorological factors in deep networks, and use jump connections to enhance gradient fluidity, solve the gradient vanishing problem in deep networks, and improve computational efficiency. In order to further optimize the feature selection and information transfer process, the model introduces an adaptive gating unit (GLU), which dynamically selects important features through a gating mechanism and suppresses the propagation of invalid information, thereby improving the accuracy and robustness of feature extraction. Combined with a pure encoder architecture, DF-Net focuses on the representation learning of multivariate time series and realizes the modeling of long-distance dependencies through deeply stacked encoder layers. Overall, the network maintains the flexibility of the Transformer model while targeting The time series characteristics of concentration prediction are effectively optimized, showing excellent performance and adaptability from feature extraction to generation of prediction results.

[0060] DF-Net's working process: DF-Net's working process includes three main stages: data preprocessing, feature extraction and interaction, and prediction generation and optimization. In the data preprocessing stage, the original meteorological factor data is standardized and Fourier transformed to decompose the time domain signal into the frequency domain signal to extract more representative features and eliminate redundant information. Subsequently, the time series of each meteorological factor is independently embedded as a token as the model input, and the residual attention layer is used to achieve efficient interaction between meteorological factors and capture their influence on the weather. The complex influence relationship of PM10 concentration. The residual attention layer balances the globality and locality of the features through skip connections and multi-head attention mechanisms, while avoiding the gradient vanishing problem in model training. Then, after further processing by the adaptive gating unit (GLU), the model dynamically adjusts the feature importance of each meteorological factor to optimize the information selection process and suppress noise interference. In the output stage, the features after gating adjustment are summarized and mapped through a fully connected network to finally generate a predicted value of PM10 concentration. In addition, the model evaluates the marginal contribution of meteorological factors through the SHAP analysis method, and uses this explanatory mechanism to further optimize the model structure and parameter configuration. The overall working process fully reflects the efficiency and accuracy of DF-Net for multivariate time series modeling under complex meteorological conditions, which is High-precision prediction of concentration provides full-process support.

[0061] 1.4.2 Pure Encoder Architecture The model adopts a pure encoder architecture (Encoder-Only), focusing on the representation learning and adaptive correlation capabilities of multivariate sequences. The time series of each meteorological factor is first tokenized to describe the attributes of the variable, applied to mutual interactions through residual attention, and processed separately by GLU to represent the sequence. The specific architecture is as follows Figure 3 shown.

[0062] For a multidimensional time series with a time length of T and a number of variables of N ,use , represents all meteorological factors at the same time, and Represents the complete observation data of a certain meteorological factor. Perform feature mapping on each x′n separately to obtain Characteristic representation of meteorological factors , where each All of them capture the changing pattern of meteorological factor i in the time series. These feature representations will enter the Transformer modules of each layer, firstly realize the information interaction between variables through the residual self-attention mechanism, then capture and select effective features through the gating mechanism, and then the fully connected layer will further map the features to the output space. The calculation process of the above expression can be expressed as follows:

[0063] In the formula, Represents the prediction results of each meteorological factor. Since neurons can be naturally encoded through the order between time points, the model does not need to introduce additional position encoding (PositionEmbedding).

[0064] 1.4.3 Residual Attention Mechanism In the original Transformer model structure, each layer contains a residual structure, which is to put layer normalization (LayerNorm, LN) after each sub-layer processing. The hourly concentration prediction of is characterized by long time and high complexity. This embodiment uses the residual attention mechanism as above. Figure 3 As shown in Figure 5, applying the residual structure to the attention layer can effectively solve the problems of gradient diffusion and network degradation in deep neural networks.

[0065] Compared with the attention mechanism in the traditional Transformer model, as follows:

[0066] The residual attention mechanism adds a residual structure when calculating the attention scores of all heads in each layer, which is to add the attention score of the current layer to the attention score of the previous layer. In addition, adding skip connections (SkipConnections) when calculating the attention matrix does not increase the amount of calculation exponentially, so its calculation efficiency is guaranteed. The formula for calculating the attention matrix of the nth layer is as follows:

[0067] In the formula, by allowing the attention of the first layer to pass directly to the last layer, the risk of gradient disappearance is minimized. The accuracy of concentration prediction is improved. In general, this mechanism adds the softmax attention score Prev of the previous layer with the shape of (heads,_from_seq_len,to_seq_len) to the current multi-head attention module. The overall formula is as follows: headi=RAttention(QwiQ,kwik,vwiv,previ) (9) Where Previ has the shape of (from_seq_len, to_seq_len), which corresponds to headi. The added skip connection and multi-head attention mechanism can capture long-distance dependencies, making the feature channels of all meteorological factors in a relatively uniform distribution, which can effectively improve Accuracy of the concentration prediction task.

[0068] 1.4.4 Adaptive Gating Unit GLU selectively transmits information through a gating mechanism. It divides the input features into two parts, one part is directly transmitted through a linear transformation, and the other part is generated through a Sigmoid activation function to generate a gating value. This mechanism enables the model to dynamically select which features should be retained and which should be suppressed, thereby making more effective use of information. It can be used to replace the feedforward network (FFN) layer in the Transformer structure.

[0069] FFN usually includes two linear transformations and an activation function ReLU in structure. The input first expands the dimension through a linear layer, then is activated by ReLU, and then restored to its original size through another linear layer. Its calculation formula is as follows: FFN (x)= ReLU (W1.x+b1).W2+b2 (10) In the formula, W1 and W2 are weight matrices, and b1 and b2 are bias terms. The gating mechanism of GLU is used to control the transmission and retention of information, so that useful information can be better retained in the deep layer. In DF-Net, after the residual attention mechanism, an adaptive gating unit is connected to each meteorological factor. Each gating unit generates a gating value gi according to the representation of the variable, which is used to adjust the importance of the variable in the next step of processing. The calculation formula is: gi =σ( W .ResidualAttentionOutput i + bg ) (11) Where Wg and bg are the weight and bias parameters of the gated unit, and σ is the Sigmoid activation function. After the gated unit, the gate control gi is multiplied element by element with the representation of the corresponding variable to obtain the adjusted output: Output i = gi .ResidualAttentionOutput i (12) Through the above formula, the contribution of each variable to the final prediction can be dynamically adjusted. Subsequently, the representations of all meteorological factors adjusted by the gating mechanism are aggregated to form the final feature representation. The final feature representation is input to the output layer and passes through a multi-layer fully connected network (MLP) to generate a predicted value for PM10 concentration.

[0070] By introducing an adaptive gating mechanism, the model can perform more refined control between different meteorological factors. The gating unit adaptively adjusts the influence of each factor based on the representation generated by the self-attention mechanism, thereby more effectively capturing and utilizing the complex relationships between meteorological factors. When processing high-dimensional, multivariate time series data, the model is allowed to dynamically adjust the importance of each meteorological factor while retaining the global information between different features. This approach can better help the model avoid treating all variables equally when dealing with multivariate prediction tasks, thereby improving the accuracy and robustness of the prediction.

[0071] 2 Results and analysis 2.1 Data Preprocessing In order to observe more intuitively After removing the abnormal number and normalizing and Fourier transforming, the decomposed frequency domain components are added to obtain a relatively smooth Concentration curve. The sine and cosine frequency domain components are used to synthesize Concentration values ​​such as Figure 4 shown.

[0072] 2.2 Data Characteristics After eliminating the abnormal data caused by equipment failure, the SHAP method is used to calculate the Marginal contributions in concentration prediction, such as Figure 5 shown.

[0073] Each point in the figure represents the SHAP value of a meteorological factor, and the characteristic value is represented by colors from blue to red from low to high. The larger the SHAP value, the greater the contribution of the feature to the prediction result. Figure 5 As shown, humidity and temperature are the factors affecting The main factor of concentration, among which the SHAP value of temperature, is mainly concentrated in the negative direction, indicating that the higher the temperature The lower the concentration, the better. Humidity has a positive effect, and increased humidity helps An increase in concentration, while an increase in temperature may reduce Wind speed and visibility show obvious negative effects, especially when the wind speed is high or the visibility is high. The concentration of pollutants has dropped significantly, indicating that these factors have a positive effect on the dilution and dispersion of pollutants in the atmosphere. In addition, recent studies have demonstrated that meteorological changes such as temperature and wind speed have a greater impact on the dilution and dispersion of pollutants than human activities. The observed trend at the horizontal level has a more significant effect.

[0074] 2.3 Model Training During the training process of this model, the number of encoder layers is set to 5, the hidden dimension of the model and the dimension size of the feedforward neural network are set to 512, the training batch size is set to 32, and the number of training iterations is set to 10. The initial learning rate is set to 0.0001. The adjustment of the learning rate is related to the performance of the model. When the loss of the validation set does not improve within several epochs, the learning rate is reduced. And combined with the optimizer Adam, during the training process, the optimizer dynamically adjusts the learning rate according to the set strategy to better find the global or local optimal solution, and performs denormalization after training.

[0075] The model input sequence length and output sequence length are both 48, the input dimension and output dimension are both 7, and each time step of the time series data includes relative humidity (RHU), wind speed (WS), wind direction (WD), air pressure (PRS), temperature (TEM), horizontal visibility (VIS) and PM10 mass concentration. The model moving average window size is set to 25, and the activation function defaults to GELU (GaussianErrorLinearUnit) to reduce the impact of noise and fluctuations. And the distillation mechanism is used to reduce the computational complexity and training time of the encoder.

[0076] 2.4 Model Performance Since PM10 concentration is affected by a variety of meteorological factors, a multi-input single-output model is used for model training. And with the popularity of Transformer in recent years, it and its variant models are increasingly being used in the prediction of time series data. There are many existing technologies that have applied the Transformer architecture to the prediction of PM2.5 concentration. This example compares the Transformer model and its variants Informer, Flowformer, Flashformer, and Reformer mentioned above, which are popular in the time series field, with this model. In comparison, this model can achieve better results. Its performance is shown in Table 1 below: Table 1 Comparison of prediction accuracy and fitting performance of different models

[0077] Among them, MAE stands for mean absolute error, which is the average of the absolute differences between the predicted value and the true value. MAPE and MSPE stand for mean absolute percentage error and mean square percentage error, respectively. One is the relative percentage of the prediction error and the other is the square average of the percentage error. R2 reflects the goodness of fit of the model. The value range is [0,1]. The closer to 1, the better the model captures and predicts the given variable. Changes in levels.

[0078] The results show that the R2 and MAE of the DF-Net model are 0.973 and 0.363 respectively. Compared with the other five models, the MAE can be reduced by 23.8%~518.2%, an average reduction of 281.12%, and the R2 can be increased by 3.2%~10.9%, an average increase of 7.72%. The MAPE and MSPE of the DF-Net model are 0.098 and 0.041 respectively. Compared with the other models, the MAPE is reduced by 3.3%~16.7%, an average reduction of 7.9%, and the MSPE is reduced by 1.2%~9%, an average reduction of 5.22%. It proves that the DF-Net model has a better ability to capture the characteristics of data under complex meteorological conditions.

[0079] In order to more intuitively reflect the effects of these models on the test set, this example selects a 45-hour test period, and its specific performance is as follows: Figure 6 As shown in the figure, it can be clearly seen that DF-Net has the highest fit with the actual value, and its fluctuation trend is closest to the true value curve, with only slight deviations, mainly reflected in small local fluctuations. The fluctuations of traditional Transformer, Informer and Reformer are more irregular, and the deviations from the true value are obvious at the peaks and troughs.

[0080] Among them, the reason for the poor performance of Transformer may be the lack of special design for time series features. It was originally designed for natural language processing tasks. The characteristics of time series and language sequences are different. The Transformer model will lack effective capture of time series with complex periodicity or trends due to over-reliance on global information. Although the Informer model reduces the computational complexity due to its unique sparse attention mechanism, it may also cause the model to ignore some important time points, especially when there are frequent or small fluctuations in the time series data, sparse attention may not be enough to accurately fit these details. In addition, the generative decoder of Informer can improve efficiency, but it does not make predictions step by step, but directly generates the entire prediction sequence, which also leads to insufficient grasp of details. The local sensitive hashing (LSH Attention) unique to Reformer reduces the amount of calculation by clustering similar sequence elements, but when the data fluctuations are more complex, LSH may not be able to accurately cluster similar time points, resulting in the model failing to capture key long-distance dependencies.

[0081] The Flowformer and Flashformer models have a worse fit than the model proposed in this embodiment. The reason may be that Flowformer is a flow-based (Flow-BasedAttention) model, which optimizes the self-attention mechanism and can allocate attention with lower computational complexity, especially when processing long sequence tasks. It has higher computational efficiency, but at the same time its local feature capture ability is average. Flashformer focuses on improving efficiency through FlashAttention, but when dealing with noisy data or high-frequency changes, it may cause the model to ignore important fluctuation points or make wrong predictions.

[0082] In summary, this embodiment uses the Fourier transform method to convert The time domain signals of concentration and meteorological observation data are decomposed into frequency domain signals, thereby extracting potential information and significantly improving the feature extraction accuracy of the model. Based on the SHAP analysis method, the effects of meteorological factors such as temperature, humidity, air pressure, wind speed, wind direction and visibility on the The proposed DF-Net model uses these meteorological factors as input variables, which can effectively improve the prediction performance of the model. The DF-Net model embeds each meteorological factor as a token and introduces the residual attention mechanism and GLU gating unit to enhance the model's understanding of the characteristics of the meteorological factors and their relationship with the meteorological factors. The ability to capture concentration correlation can effectively reduce noise interference.

[0083] See also Figure 7 , is a schematic diagram of a structure of a device for predicting the concentration of inhalable particles based on meteorological factors provided by an embodiment of the present invention. The device for predicting the concentration of inhalable particles based on meteorological factors comprises: The preprocessing module 10 is used to perform standardization processing on the original meteorological factor data and decompose the time domain signal into the frequency domain signal through Fourier transform to obtain the frequency domain feature representation; An embedding module 11 is used to embed the time series of each meteorological factor in the obtained frequency domain feature representation into a token independently to obtain a global feature representation of each meteorological factor; The interaction module 12 is used to perform feature interaction on the obtained global feature representation of each meteorological factor through the residual attention mechanism of the DF-Net network, so as to capture the complex correlation between meteorological factors and obtain the feature representation after interaction; The adjustment module 13 is used to select features of the obtained interactive feature representation through the adaptive gating unit of the DF-Net network, dynamically adjust the feature importance of each meteorological factor, and obtain an optimized feature representation; The generation module 14 is used to summarize and map the obtained optimized feature representation through the fully connected layer of the DF-Net network to generate a predicted value of the inhalable particulate matter concentration.

[0084] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: Firstly, the original meteorological factor data is standardized, and the time domain signal is decomposed into frequency domain signal by Fourier transform to obtain more representative frequency domain feature representation, effectively eliminate redundant information and extract key features; secondly, the time series of each meteorological factor in the frequency domain feature representation is independently embedded as a token to obtain the global feature representation of each meteorological factor, avoiding the problems of noise and information loss in traditional multi-factor mixed modeling; then, the residual attention mechanism of the DF-Net network is used to perform feature interaction on the global feature representation of each meteorological factor to capture the complex correlation between meteorological factors, and at the same time, the jump connection is used to enhance the gradient fluidity and solve the gradient disappearance problem in the deep network; then, the adaptive gating unit of the DF-Net network is used to select the feature representation after interaction, dynamically adjust the feature importance of each meteorological factor, suppress the propagation of invalid information, and improve the accuracy and robustness of feature extraction; finally, the optimized feature representation is summarized and mapped through the fully connected layer of the DF-Net network to generate the predicted value of the inhalable particulate matter concentration. From the above analysis, it can be seen that the embodiments of the present invention can efficiently process multivariate time series data under complex meteorological conditions, accurately capture nonlinear characteristics, and significantly improve the accuracy of inhalable particulate matter concentration prediction.

[0085] As an example, the preprocessing module is specifically used for: The concentration time series of inhalable particulate matter in the original meteorological factor data is standardized to obtain a standardized concentration time series of inhalable particulate matter; Performing Fourier transform on the standardized time series of the concentration of inhalable particulate matter, decomposing the time domain signal into a frequency domain signal, and obtaining a frequency domain representation of the concentration of inhalable particulate matter; The frequency domain representation of the inhalable particulate matter concentration is decomposed into multiple frequency components, each frequency component corresponds to a sine wave of a different frequency, and a simplified frequency domain signal is obtained as a frequency domain feature representation.

[0086] As an example, the embedded module is specifically used for: For each meteorological factor in the frequency domain feature representation, its entire time series is considered as a whole; The time series of each meteorological factor is embedded as a whole, and it is embedded into tokens independently, and finally the global feature representation of each meteorological factor is obtained.

[0087] As an example, the interaction module is specifically used for: The global feature representation of each meteorological factor is input into the residual attention mechanism of the DF-Net network; The residual attention mechanism of the DF-Net network is used to perform feature interaction processing on the global feature representation of each input meteorological factor, capture the complex correlation between meteorological factors, and obtain the interactive feature representation.

[0088] As an example, the adjustment module is specifically used for: Input the interacted feature representation into the adaptive gating unit of the DF-Net network; The adaptive gating unit of the DF-Net network is used to select the interactive feature representation and dynamically adjust the feature importance of each meteorological factor. Based on feature selection and importance adjustment operations, an optimized feature representation is obtained.

[0089] As an example, the generating module is specifically used for: Inputting the optimized feature representation into the fully connected layer of the DF-Net network; The multi-layer structure of the fully connected layer of the DF-Net network is used to summarize the optimized feature representation of the input to integrate the optimized feature information of each meteorological factor; Through the linear transformation of the fully connected layer, the summarized feature representation is mapped to the output space to obtain the predicted value of the inhalable particulate matter concentration.

[0090] See also Figure 8 , is a schematic diagram of an inhalable particle concentration prediction device based on meteorological factors provided by an embodiment of the present invention. The inhalable particle concentration prediction device based on meteorological factors of this embodiment includes: a processor 100, a memory 101, and a computer program stored in the memory 101 and executable on the processor 100, such as an inhalable particle concentration prediction program based on meteorological factors. When the processor 100 executes the computer program, the steps in the above-mentioned embodiments of the inhalable particle concentration prediction method based on meteorological factors are implemented. Alternatively, when the processor 100 executes the computer program, the functions of each module / unit in the above-mentioned device embodiments are implemented.

[0091] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the inhalable particulate matter concentration prediction device based on meteorological factors.

[0092] The inhalable particle concentration prediction device based on meteorological factors can be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The inhalable particle concentration prediction device based on meteorological factors may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the schematic diagram is only an example of an inhalable particle concentration prediction device based on meteorological factors, and does not constitute a limitation on the inhalable particle concentration prediction device based on meteorological factors. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the inhalable particle concentration prediction device based on meteorological factors may also include input and output devices, network access devices, buses, etc.

[0093] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the inhalable particle concentration prediction device based on meteorological factors, and uses various interfaces and lines to connect various parts of the inhalable particle concentration prediction device based on meteorological factors.

[0094] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the inhalable particulate matter concentration prediction device based on meteorological factors by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0095] Wherein, if the module / unit integrated in the inhalable particulate matter concentration prediction device based on meteorological factors is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media does not include electrical carrier signals and telecommunication signals.

[0096] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0097] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for predicting the concentration of inhalable particulate matter based on meteorological factors, characterized in that: include: The original meteorological factor data is standardized, and the time domain signal is decomposed into the frequency domain signal through Fourier transform to obtain the frequency domain feature representation; The time series of each meteorological factor in the obtained frequency domain feature representation is independently embedded as a token to obtain the global feature representation of each meteorological factor; For the global feature representation of each meteorological factor, the residual attention mechanism of the DF-Net network is used to perform feature interaction, capture the complex correlation between meteorological factors, and obtain the feature representation after interaction; The obtained interactive feature representation is subjected to feature selection through the adaptive gating unit of the DF-Net network, and the feature importance of each meteorological factor is dynamically adjusted to obtain the optimized feature representation; The optimized feature representation is summarized and mapped through the fully connected layer of the DF-Net network to generate the predicted value of the inhalable particulate matter concentration.

2. The method for predicting the concentration of inhalable particulate matter based on meteorological factors according to claim 1, characterized in that: The original meteorological factor data is standardized and the time domain signal is decomposed into a frequency domain signal by Fourier transform to obtain a frequency domain feature representation, including: The concentration time series of inhalable particulate matter in the original meteorological factor data is standardized to obtain a standardized concentration time series of inhalable particulate matter; Performing Fourier transform on the standardized time series of the concentration of inhalable particulate matter, decomposing the time domain signal into a frequency domain signal, and obtaining a frequency domain representation of the concentration of inhalable particulate matter; The frequency domain representation of the inhalable particulate matter concentration is decomposed into multiple frequency components, each frequency component corresponds to a sine wave of a different frequency, and a simplified frequency domain signal is obtained as a frequency domain feature representation.

3. The method for predicting the concentration of inhalable particulate matter based on meteorological factors according to claim 1, characterized in that: The time series of each meteorological factor in the obtained frequency domain feature representation is independently embedded as a token to obtain a global feature representation of each meteorological factor, including: For each meteorological factor in the frequency domain feature representation, its entire time series is considered as a whole; The time series of each meteorological factor is embedded as a whole, and it is embedded into tokens independently, and finally the global feature representation of each meteorological factor is obtained.

4. The method for predicting the concentration of inhalable particulate matter based on meteorological factors according to claim 1, characterized in that: The global feature representation of each meteorological factor is obtained by performing feature interaction through the residual attention mechanism of the DF-Net network to capture the complex associations between meteorological factors and obtain the feature representation after interaction, including: The global feature representation of each meteorological factor is input into the residual attention mechanism of the DF-Net network; The residual attention mechanism of the DF-Net network is used to perform feature interaction processing on the global feature representation of each input meteorological factor, capture the complex correlation between meteorological factors, and obtain the interactive feature representation.

5. The method for predicting the concentration of inhalable particulate matter based on meteorological factors according to claim 1, characterized in that: The obtained interactive feature representation is subjected to feature selection through the adaptive gating unit of the DF-Net network, and the feature importance of each meteorological factor is dynamically adjusted to obtain an optimized feature representation, including: Input the interacted feature representation into the adaptive gating unit of the DF-Net network; The adaptive gating unit of the DF-Net network is used to select the interactive feature representation and dynamically adjust the feature importance of each meteorological factor. Based on feature selection and importance adjustment operations, an optimized feature representation is obtained.

6. The method for predicting the concentration of inhalable particulate matter based on meteorological factors according to claim 5, characterized in that: The optimized feature representation is summarized and mapped through the fully connected layer of the DF-Net network to generate a predicted value of the inhalable particulate matter concentration, including: Inputting the optimized feature representation into the fully connected layer of the DF-Net network; The multi-layer structure of the fully connected layer of the DF-Net network is used to summarize the optimized feature representation of the input to integrate the optimized feature information of each meteorological factor; Through the linear transformation of the fully connected layer, the summarized feature representation is mapped to the output space to obtain the predicted value of the inhalable particulate matter concentration.

7. A device for predicting the concentration of inhalable particulate matter based on meteorological factors, characterized in that: include: The preprocessing module is used to standardize the original meteorological factor data and decompose the time domain signal into the frequency domain signal through Fourier transform to obtain the frequency domain feature representation; An embedding module is used to embed the time series of each meteorological factor in the obtained frequency domain feature representation into tokens independently to obtain a global feature representation of each meteorological factor; The interaction module is used to obtain the global feature representation of each meteorological factor, perform feature interaction through the residual attention mechanism of the DF-Net network, capture the complex correlation between meteorological factors, and obtain the feature representation after interaction; The adjustment module is used to select features from the interactive feature representation obtained by the adaptive gating unit of the DF-Net network, dynamically adjust the feature importance of each meteorological factor, and obtain the optimized feature representation; The generation module is used to summarize and map the optimized feature representation through the fully connected layer of the DF-Net network to generate the predicted value of the inhalable particulate matter concentration.

8. The device for predicting the concentration of inhalable particulate matter based on meteorological factors according to claim 7, characterized in that: The preprocessing module is specifically used for: The concentration time series of inhalable particulate matter in the original meteorological factor data is standardized to obtain a standardized concentration time series of inhalable particulate matter; Performing Fourier transform on the standardized time series of the concentration of inhalable particulate matter, decomposing the time domain signal into a frequency domain signal, and obtaining a frequency domain representation of the concentration of inhalable particulate matter; The frequency domain representation of the inhalable particulate matter concentration is decomposed into multiple frequency components, each frequency component corresponds to a sine wave of a different frequency, and a simplified frequency domain signal is obtained as a frequency domain feature representation.

9. The device for predicting the concentration of inhalable particulate matter based on meteorological factors according to claim 7, characterized in that: The embedded module is specifically used for: For each meteorological factor in the frequency domain feature representation, its entire time series is considered as a whole; The time series of each meteorological factor is embedded as a whole, and it is embedded into tokens independently, and finally the global feature representation of each meteorological factor is obtained.

10. A device for predicting the concentration of inhalable particulate matter based on meteorological factors, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the method for predicting the concentration of inhalable particulate matter based on meteorological factors according to any one of claims 1 to 6 is implemented.

Images