Carbon dioxide concentration prediction method based on multi-scale attention mechanism and residual learning
By employing a multi-scale attention mechanism and residual learning, the problems of single-scale prediction and inherent bias in carbon dioxide concentration forecasting are solved, achieving high-precision and robust carbon dioxide concentration forecasting suitable for meteorological data processing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 云南省大气探测技术保障中心
- Filing Date
- 2025-12-24
- Publication Date
- 2026-04-21
AI Technical Summary
Existing carbon dioxide concentration prediction methods have limited ability to handle nonlinearity, nonstationarity, and complex spatiotemporal dependence. They are unable to capture complex patterns at multiple time scales simultaneously and lack the ability to focus on key time points, resulting in inherent bias problems.
We employ a multi-scale attention mechanism and residual learning approach, using a pyramid multi-scale processing module to capture features at different time scales in parallel. We then combine scale fusion and time-step attention mechanisms to adaptively weight and fuse features, and introduce a residual learning framework to correct model biases, constructing a hybrid neural network for prediction.
It improves the accuracy and interpretability of carbon dioxide concentration prediction, enhances the robustness and generalization performance of the model, reduces systematic errors, and provides high-precision rolling prediction that meets actual business needs.
Smart Images
Figure CN121902579A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of meteorological data prediction technology, specifically, it relates to a carbon dioxide concentration prediction method based on multi-scale attention mechanism and residual learning. Background Technology
[0002] As global climate change becomes increasingly severe, accurate monitoring and forecasting of atmospheric carbon dioxide concentrations have become crucial. Accurate concentration forecasts help analyze carbon source and sink distribution, assess the effectiveness of emission reduction policies, and provide decision support for environmental governance.
[0003] Currently, carbon dioxide concentration prediction methods are mainly divided into two categories: one is based on traditional time series analysis or machine learning models (such as ARIMA, SVM, and random forest), and the other is based on deep learning sequence models (such as LSTM and GRU). Traditional methods have limited ability to handle the nonlinearity, non-stationarity, and complex spatiotemporal dependencies of carbon dioxide concentration sequences. While existing deep learning models have improved their ability to capture long-term dependencies, they still face the following technical bottlenecks: Single-scale problem: Carbon dioxide concentration changes are influenced by factors on multiple time scales (such as intraday photosynthesis, morning and evening traffic peaks, and seasonal vegetation changes). Single-scale models struggle to capture these complex patterns, ranging from hourly to seasonal variations, leading to inaccurate predictions of certain fluctuations (such as midday concentration troughs and stable nighttime accumulation).
[0004] Insufficient feature utilization: The model usually treats features of all historical time steps equally, lacking the ability to focus on key time points (such as concentration inflection points), which affects the interpretability and prediction accuracy of the model.
[0005] Inherent bias problem: Single neural network models are prone to inherent systematic biases during training, which limits further improvement in model prediction performance. Summary of the Invention
[0006] To overcome the problems existing in the background technology, the present invention provides a carbon dioxide concentration prediction method based on multi-scale attention mechanism and residual learning, which can realize multi-scale feature adaptive fusion, focus on key time information and self-correct system bias, thereby improving the accuracy of carbon dioxide concentration.
[0007] To achieve the above objectives, the present invention is implemented through the following technical solution: The carbon dioxide concentration prediction method based on multi-scale attention mechanism and residual learning includes the following steps: S1, Data Processing and Sequence Construction Historical values of carbon dioxide concentration and related historical meteorological data were collected to obtain the raw dataset. The raw data was cleaned and linearly interpolated. Principal component analysis was used to reduce the dimensionality of the high-dimensional meteorological features. Principal components with a cumulative variance contribution rate that reached a predetermined threshold were retained. The dimensionality-reduced features were concatenated with the normalized historical carbon dioxide concentration values to form the total input matrix of the model. S2, construct and train a multi-scale attention master prediction model; The main prediction module includes the following components in sequence. (1) Pyramid Multi-Scale Processing Module The pyramid multi-scale processing module is constructed by setting up the original scale branch and four downsampling scale branches in parallel. By processing multiple time scales in parallel, this method simultaneously captures intraday patterns, short-term fluctuations, and ultra-short-term abrupt changes in carbon dioxide concentration, overcoming the shortcomings of single-scale models in fully modeling complex time patterns; thus obtaining a multi-scale feature set. ; (2) Scale fusion attention module Global average pooling is performed on the five scales described in the above steps to obtain summary vectors for each of the five scales; after concatenating all summary vectors, they are passed through a fully connected layer and... Activation function, calculate attention weights at each scale: ,in This is the weight matrix. For bias vectors, and ; Features at each scale Multiply by its corresponding weight Then, sum them up to obtain the fused feature map: ; (3) Time step attention module The unnormalized importance score for each time step is calculated using a fully connected layer: , in, It is a feature of the t-th time step. This is the weight vector; The scores at all time steps are normalized to attention weights using the Softmax function: This represents the summation of the unnormalized importance scores of all time steps used, where S is the total number of time steps, which is 24. Calculate the weighted context vector: The importance of each time step in the fused feature sequence is evaluated. The context vector c is a weighted summary of the fused feature sequence in the time dimension, used to evaluate the importance of different time steps.
[0008] (4) Deep feature extraction and output Fusion features The input is fed into a GRU network for deep feature transformation to obtain high-level features. ; Transfer the attention context vector at each time step The final state of deep GRU The vectors are concatenated; the concatenated vectors are then passed through a progressively compressed network consisting of fully connected layers and Dropout layers, ultimately outputting the preliminary predictions of the main model. .
[0009] S3, Construct and train the residual correction model; S4, Integrated Prediction.
[0010] Furthermore, step S3 specifically involves: Calculate the residuals of the main model on the training set: ,in This represents the actual concentration value. The initial predictions of the main model on the training set; Input features of the main model at each step Flattened into vectors, these vectors serve as input to the residual model. Extreme gradient boosting trees are used as the residual model. The extreme gradient boosting trees are trained to learn the residual from the input features. The mapping relationship is used to obtain the residual prediction value. .
[0011] Furthermore, in step S4, The final predicted carbon dioxide concentration is obtained by adding the initial prediction from the master model to the correction value from the residual model. .
[0012] Furthermore, in step S1, the collected data includes temperature, humidity, wind speed, and air pressure; the cumulative variance contribution rate threshold cum_variance is set to 0.8-0.95. Furthermore, in step S2, in the pyramid multi-scale processing module, Input sequence The inputs are fed in parallel into four processing branches with different receptive fields, where B is the batch size, S is the time step, and F is the feature dimension. Set a set of scaling factors Calculate the pooling size for each scale ,in Indicates rounding down; Perform one-dimensional average pooling on the input sequence: Upsample the pooled sequence back to its original length S: The upsampled sequence is input into a gated recurrent unit to extract sequence features at this scale: Meanwhile, the original input sequence, i.e., the input sequence constructed in step 1, is also processed by a separate GRU branch to obtain the original scale features. Finally, a multi-scale feature set is obtained. .
[0013] The beneficial effects of this invention are: Multi-scale feature capture capability: Through the pyramid multi-scale processing module, this invention can analyze concentration sequences in parallel from different time scales (such as 24 hours, 12 hours, 8 hours, etc.), thereby simultaneously capturing intraday patterns, short-term fluctuations and ultra-short-term mutations, overcoming the shortcomings of single-scale models in modeling complex time patterns.
[0014] Adaptive Feature Focusing Capability: Through a dual mechanism of scale-based fusion attention and time-step attention, the model can automatically assess the importance of features at different scales and information from different historical moments to the current prediction, achieving adaptive weighted fusion. This enhances the efficiency of utilizing key information and improves the model's prediction accuracy and interpretability.
[0015] Systematic bias correction capability: An innovative residual learning framework is introduced, utilizing the powerful nonlinear fitting capability of the XGBoost model to learn and correct the inherent prediction bias of the main neural network. This hybrid architecture of "main model prediction + residual model correction" can effectively reduce the systematic error of prediction, making the final result closer to the true value.
[0016] Robustness and Generalization: The multi-scale structure and attention mechanism make the model more robust to noise and irregular fluctuations in the input data. At the same time, the combination of deep learning's sequence modeling capabilities and tree model's residual correction capabilities results in better generalization performance under different seasons and weather conditions.
[0017] Highly practical for engineering applications: The method of this invention has a clear process, a high degree of modularity, and is easy to implement and deploy. The setting of an input step size of 24 hours and an output step size of 1 hour perfectly meets the actual business needs for high-precision rolling prediction of concentration in the next hour. Attached Figure Description
[0018] Figure 1 This is a schematic diagram of the overall architecture of the hybrid neural network model provided in an embodiment of the present invention; Figure 2 This is a flowchart of the present invention. Detailed Implementation
[0019] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the technical solutions of this invention will be described in detail below. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are all within the scope of protection of this invention.
[0020] To illustrate the present invention more clearly, the following embodiments will be described in detail.
[0021] Example 1 The carbon dioxide concentration prediction method based on multi-scale attention mechanism and residual learning includes the following steps: Step 1: Collect a raw dataset containing historical carbon dioxide concentration values and relevant meteorological elements (such as temperature, humidity, wind speed, and air pressure). Clean the data and perform linear interpolation to handle missing values.
[0022] Subsequently, principal component analysis was used to reduce the dimensionality of the high-dimensional meteorological features, retaining principal components whose cumulative variance contribution rate reached a predetermined threshold, in order to eliminate redundancy and accelerate model training.
[0023] The dimensionality-reduced features are concatenated with the normalized historical carbon dioxide concentration values to form the model's total input matrix. Finally, a sliding window method is used to construct training samples, where the input step size (historical window size) is set to S = 24 hours, and the output step size (prediction window) is 1 hour, that is, using data from the past 24 hours to predict the concentration value for the next hour.
[0024] Step 2: Construct and train a multi-scale attention master prediction model The master prediction model is an improved recurrent neural network, whose structure includes the following components in sequence: 2.1 Pyramid Multi-Scale Processing Module This module consists of one original scale branch and four downsampling scale branches in parallel, for a total of five branches. The detailed construction of each branch is as follows: Original Scale Branch: This branch directly processes the original input sequence. It is constructed as a GRU layer containing 32 hidden units and configured to return the entire output sequence. The feature sequence output by this branch is denoted as... Its dimensions are (B, 24, 32).
[0025] Scale branch k=12: This branch includes, in sequence: A one-dimensional average pooling layer with a pooling window size of 2, used to downsample 24 time steps to 12 time steps.
[0026] A one-dimensional upsampling layer with an upsampling factor of 2 is used to restore 12 time steps to the original 24 time steps.
[0027] A GRU layer: This layer contains 32 hidden units and returns the complete output sequence. The feature sequence output by this branch is denoted as... Its dimensions are (B, 24, 32).
[0028] Scale branch k=8: This branch includes, in sequence: A one-dimensional average pooling layer with a pooling window size of 3 is used to downsample 24 time steps to 8 time steps.
[0029] A one-dimensional upsampling layer with an upsampling factor of 3 is used to restore 8 time steps to the original 24 time steps.
[0030] A GRU layer: This layer contains 32 hidden units and returns the complete output sequence. The feature sequence output by this branch is denoted as... Its dimensions are (B, 24, 32).
[0031] Scale branch k=6: This branch includes, in sequence: A one-dimensional average pooling layer with a pooling window size of 4 is used to downsample 24 time steps to 6 time steps.
[0032] A one-dimensional upsampling layer with an upsampling factor of 4 is used to restore 6 time steps to the original 24 time steps.
[0033] A GRU layer: This layer contains 32 hidden units and returns the complete output sequence. The feature sequence output by this branch is denoted as... Its dimensions are (B, 24, 32).
[0034] Scale branch k=4: This branch includes, in sequence: A one-dimensional average pooling layer with a pooling window size of 6 is used to downsample 24 time steps to 4 time steps.
[0035] A one-dimensional upsampling layer with an upsampling factor of 6 is used to restore 4 time steps to the original 24 time steps.
[0036] A GRU layer: This layer contains 32 hidden units and returns the complete output sequence. The feature sequence output by this branch is denoted as... Its dimensions are (B, 24, 32).
[0037] Module data processing flow: This module will take the input sequence. (Where B is the batch size, S is the time step, and F is the feature dimension) The inputs are fed in parallel into multiple processing branches with different receptive fields. A set of scaling factors is defined. For each scale: Calculate the pooling size for each scale ,in , This indicates rounding down to the nearest integer.
[0038] Perform one-dimensional average pooling on the input sequence: Upsample the pooled sequence back to its original length S: The upsampled sequence is input into a gated recurrent unit to extract sequence features at this scale: Meanwhile, the original input sequence is also processed by a separate GRU branch to obtain the original scale features. Ultimately, a multi-scale feature set is obtained. .
[0039] 2.2 Scale Fusion Attention Module This module is used for adaptively weighted fusion of feature sequences from five scales, and its specific construction is as follows: First, for the feature sequences at each scale Each vector is compressed along the time step dimension using a global average pooling layer, generating five corresponding summary vectors. Each vector has dimensions (B, 32).
[0040] Next, these five summary vectors are concatenated along the feature dimension to form a fused summary vector. Its dimensions are (B, 160).
[0041] Then, the fusion summary vector The input is fed into a fully connected layer with 5 neurons, which uses the Softmax activation function to output a five-dimensional scale attention weight vector. Each weight value corresponds to the importance of a scale.
[0042] Finally, the feature sequences at five scales respectively with the weight vector The corresponding weight values are multiplied together, and then all the weighted feature sequences are summed to obtain the fused feature map. Its dimensions are (B, 24, 32).
[0043] Module data processing flow: This module is used to adaptively weight and fuse features from different scales.
[0044] First, for each scale feature Perform global average pooling to obtain its summary vector.
[0045] Concatenate the summary vectors of all scales: .
[0046] Through a fully connected layer and Activation function, calculate attention weights at each scale: ,in This is the weight matrix. For bias vectors, and .
[0047] Features at each scale Multiply by its corresponding weight Then, sum them up to obtain the fused feature map: .
[0048] 2.3 Time Step Attention Module This module is used to evaluate the importance of each time step in the fused feature sequence, and its specific construction is as follows: Feature map after fusion First, the data is fed into a fully connected layer with one neuron that does not use an activation function, where an unnormalized attention score is computed for each of the 24 time steps. .
[0049] Then, these 24 scores are normalized using a Softmax function to obtain the attention weights for 24 time steps. The sum of the weights is 1.
[0050] Finally, these 24 weights are used to perform a weighted summation of the original 24 time-step features to generate a context vector that condenses global temporal information. Its dimensions are (B, 32).
[0051] Module data processing flow: This module is used to identify the fused sequence. The importance of each time step.
[0052] The unnormalized importance score for each time step is calculated using a fully connected layer: (in, It is a feature of the t-th time step. (This is the weight vector).
[0053] The scores at all time steps are normalized to attention weights using the Softmax function: , This represents the summation of the unnormalized importance scores of all time steps used, where S is the total number of time steps, which is 24.
[0054] Calculate the weighted context vector: .
[0055] 2.4 Deep Feature Extraction and Output This module further processes the fused features and generates the final prediction, as detailed below: Fusion Feature Map First, they are fed into a two-layer deep GRU network: The first-layer GRU contains 64 hidden units and is configured to return the entire output sequence; The second-layer GRU contains 32 hidden units and is configured to return only the output of the last time step, denoted as . Its dimensions are (B, 32).
[0056] Subsequently, the context vector output by the time-step attention module is... (Dimension (B, 32)) and the final state of a deep GRU network (B, 32) are concatenated to obtain a combined feature vector with dimensions (B, 64).
[0057] The combined feature vector is then passed through a progressive compression network, which includes, in sequence: The first fully connected layer contains 128 neurons, uses the ReLU activation function, and is followed by a dropout layer with a dropout rate of 0.3.
[0058] The second fully connected layer contains 64 neurons, uses the ReLU activation function, and is followed by a Dropout layer with a dropout rate of 0.2.
[0059] The third fully connected layer contains 32 neurons and uses the ReLU activation function.
[0060] Output layer: Contains 1 neuron, uses a linear activation function, and outputs the initial prediction value ŷ_main of the main model, with dimensions (B, 1).
[0061] Module data processing flow: Fusion features The input is fed into a deeper GRU network for deep feature transformation to obtain high-level features. The time-step attention context vector The final state of deep GRU The concatenated vectors are then processed through a progressive compression network consisting of fully connected layers and Dropout layers, ultimately outputting the preliminary predictions of the main model. .
[0062] Step 3: Build and train the residual correction model The master model may produce inherent systematic prediction biases. To address this, a residual correction model is introduced, which specifically learns the prediction residuals of the master model.
[0063] Calculate the residuals of the main model on the training set: ,in This represents the actual concentration value. These are the initial predictions made by the main model on the training set. (Note that the training set used for the XGBoost model is the same as that used for the main model; the difference lies in the labels.) The carbon dioxide concentration was replaced by the concentration residual. ) The input features (shape: B×24×F) of the main model at each step Flattened into a vector (shape: B×24*F), it serves as the input to the residual model.
[0064] An extreme gradient boosting tree is used as the residual model and trained on it to learn the residual from the input features. The mapping relationship is used to obtain the residual prediction value. The XGBoost model, by integrating multiple decision trees, can effectively capture complex, nonlinear residual patterns that are difficult for the main model to learn.
[0065] Step 4: Integrated Prediction The final predicted carbon dioxide concentration is obtained by adding the initial prediction from the master model to the correction value from the residual model. This integration method effectively corrects the systematic bias of the master model, thereby significantly improving the accuracy of the final prediction.
[0066] Application examples of this invention: 1. Data Sources and Preprocessing 1.1 Data Collection The data used in this application example comes from carbon dioxide concentration data collected by the Yunnan Provincial Meteorological Bureau at its newly built greenhouse gas observation station in Dali Bai Autonomous Prefecture. The data has been quality controlled and includes hourly observations from March 1, 2024 to June 1, 2025. The dataset contains the following fields: Target variable: Carbon dioxide concentration (unit: ppm).
[0067] Characteristic variables: temperature, relative humidity, wind speed, wind direction, air pressure, and atmospheric boundary layer height.
[0068] Time markers: year, month, day, hour.
[0069] 1.2 Data Preprocessing First, the raw data undergoes quality control to remove outliers that clearly exceed their physical meaning. Then, a small number of missing values due to instrument malfunction are filled using linear interpolation. Finally, the year, month, day, and hour are merged into a standard datetime format and set as a data index.
[0070] 1.3 Feature Engineering and Sequence Construction Principal Component Analysis (PCA): Dimensionality reduction was performed on seven meteorological features (temperature, humidity, wind speed, wind direction, and air pressure). A cumulative variance contribution rate threshold of cum_variance = 0.8 was set, effectively reducing data redundancy and noise while retaining most of the information.
[0071] Data normalization: The target variable carbon dioxide concentration value is normalized using MinMaxScaler, scaling it to the [0, 1] interval.
[0072] Constructing a supervised learning sequence: The dimensionality-reduced principal component features are concatenated with the normalized historical concentration values to form the model's input matrix. A sliding window method is used to construct samples, with an input step size S=24 and an output step size of 1. That is, each sample uses data from the past 24 hours (containing 5 principal component features and 1 concentration value label) to predict the carbon dioxide concentration in the next hour.
[0073] 2. Model Building and Training 2.1 Dataset Partitioning The sample dataset arranged in chronological order is divided into: Training set: The first 60% of the data, used for model training.
[0074] Validation set: The middle 20% of the data, used for hyperparameter tuning and early stopping.
[0075] Test set: The last 20% of the data, used for the final evaluation of model performance.
[0076] 2.2 Configuration of Multi-Scale Attention Master Prediction Model According to step 2 of this invention, the main model is constructed, and its key hyperparameters are configured as follows: Multi-scale pyramid: Set the scale factor K = {12, 8, 6, 4} to capture data fluctuation patterns at different time scales and process them in parallel with the original 24-hour scale.
[0077] GRU Units: The multi-scale branch GRU hidden unit is 32, and the deep GRU hidden unit is 64 and 32.
[0078] Attention mechanism: Both scale fusion attention and time step attention use the tanh activation function to calculate scores.
[0079] Optimizer and Training: The Adam optimizer is used, and the CyclicalLearningRate strategy is integrated (base learning rate 5e-5, maximum learning rate 1e-2), batch size=64, training epoch=150, and an early stop callback function is set (patience=20).
[0080] 2.3 Configuration of Residual Correction Model Model selection: The XGBoost regression model was used as the residual corrector.
[0081] Model parameters: set n_estimators=100, learning_rate=0.1, max_depth=5, eval_metric='mae', and enable early stopping (early_stopping_rounds=8).
[0082] Input processing: The sequence input to the main model is flattened into a vector, which is used as the input feature of XGBoost.
[0083] Training objective: To use the prediction residuals of the main model on the training set As a learning objective.
[0084] 3. Performance Evaluation and Comparative Analysis To verify the superiority of the present invention, this embodiment compares the proposed hybrid model with the following benchmark models on an independent test set: Single GRU model Single XGBoost model GRU model without multi-scale attention Multi-scale attention GRU model without residual correction The evaluation index uses the mean absolute error (MAE, unit: ppm), which is widely used in the field of environmental prediction. The lower the value, the higher the prediction accuracy.
[0085] 3.1 Comparison of Quantitative Results The MAE comparison of each model on the test set is shown in the table below: Model Name Test set MAE (ppm) Single XGBoost model 2.15 Single GRU model 1.98 GRU model without multi-scale attention 1.91 Multi-scale attention GRU model without residual correction 1.73 This invention model 1.52 3.2 Results Analysis The following conclusions can be drawn from the table above: The hybrid model proposed in this invention achieves optimal performance, with a MAE of only 1.52 ppm. Compared to the strongest single baseline model (single GRU), the prediction error is reduced by more than 29%. This fully demonstrates the effectiveness and advancement of this invention in solving the problem of carbon dioxide concentration prediction.
[0086] 4. Conclusion This embodiment fully implements all steps of the present invention on a real, long-term carbon dioxide concentration dataset. Experimental results clearly demonstrate that the present invention, through the synergistic effect of multi-scale feature extraction, dual attention focusing, and residual bias correction, can significantly improve prediction accuracy, providing a more reliable and accurate carbon dioxide concentration prediction tool.
[0087] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.
Claims
1. A carbon dioxide concentration prediction method based on multi-scale attention mechanism and residual learning, characterized in that, Includes the following steps: S1, Data Processing and Sequence Construction Historical values of carbon dioxide concentration and related historical meteorological data were collected to obtain the raw dataset. The raw data was cleaned and linearly interpolated. Principal component analysis was used to reduce the dimensionality of the high-dimensional meteorological features. Principal components with a cumulative variance contribution rate that reached a predetermined threshold were retained. The dimensionality-reduced features were concatenated with the normalized historical carbon dioxide concentration values to form the total input matrix of the model. S2, construct and train a multi-scale attention master prediction model; The main prediction module includes the following components in sequence. (1) Pyramid Multi-Scale Processing Module The pyramid multi-scale processing module is constructed by setting up the original scale branch and four downsampling scale branches in parallel. By processing multiple time scales in parallel, this method simultaneously captures intraday patterns, short-term fluctuations, and ultra-short-term abrupt changes in carbon dioxide concentration, overcoming the shortcomings of single-scale models in fully modeling complex time patterns; thus obtaining a multi-scale feature set. ; (2) Scale fusion attention module Global average pooling is performed on the five scales described in the above steps to obtain summary vectors for each of the five scales; after concatenating all summary vectors, they are passed through a fully connected layer and... Activation function, calculate attention weights at each scale: ,in This is the weight matrix. For bias vectors, and ; Features at each scale Multiply by its corresponding weight Then, sum them up to obtain the fused feature map: ; (3) Time step attention module The unnormalized importance score for each time step is calculated using a fully connected layer: , in, It is a feature of the t-th time step. This is the weight vector; The scores at all time steps are normalized to attention weights using the Softmax function: This represents the summation of the unnormalized importance scores of all time steps used, where S is the total number of time steps, which is 24. Calculate the weighted context vector: The importance of each time step in the fused feature sequence is evaluated. The context vector c is a weighted summary of the fused feature sequence in the time dimension, used to evaluate the importance of different time steps. (4) Deep feature extraction and output Fusion features The input is fed into a GRU network for deep feature transformation to obtain high-level features. ; Transfer the attention context vector at each time step The final state of deep GRU The vectors are concatenated; the concatenated vectors are then passed through a progressive compression network consisting of fully connected layers and Dropout layers, ultimately outputting the preliminary predictions of the main model. ; S3, Construct and train the residual correction model; S4, Integrated Prediction.
2. The carbon dioxide concentration prediction method based on multi-scale attention mechanism and residual learning according to claim 1, characterized in that, Step S3 specifically involves: Calculate the residuals of the main model on the training set: ,in This represents the actual concentration value. The initial predictions of the main model on the training set; Input features of the main model at each step Flattened into vectors, these vectors serve as input to the residual model. Extreme gradient boosting trees are used as the residual model. The extreme gradient boosting trees are trained to learn the residual from the input features. The mapping relationship is used to obtain the residual prediction value. .
3. The carbon dioxide concentration prediction method based on multi-scale attention mechanism and residual learning according to claim 2, characterized in that, In step S4, The final predicted carbon dioxide concentration is obtained by adding the initial prediction from the master model to the correction value from the residual model. 。 4. The carbon dioxide concentration prediction method based on multi-scale attention mechanism and residual learning according to any one of claims 1 to 3, characterized in that, In step S1, the collected data includes temperature, humidity, wind speed, and air pressure; Set the cumulative variance contribution rate threshold cum_variance = 0.8-0.
95.
5. The carbon dioxide concentration prediction method based on multi-scale attention mechanism and residual learning according to claim 1, characterized in that, In step S2, in the pyramid multi-scale processing module, Input sequence The inputs are fed in parallel into four processing branches with different receptive fields, where B is the batch size, S is the time step, and F is the feature dimension. Set a set of scaling factors Calculate the pooling size for each scale ,in Indicates rounding down; Perform one-dimensional average pooling on the input sequence: Upsample the pooled sequence back to its original length S: The upsampled sequence is input into a gated recurrent unit to extract sequence features at this scale: Meanwhile, the original input sequence, i.e., the input sequence constructed in step 1, is also processed by a separate GRU branch to obtain the original scale features. Finally, a multi-scale feature set is obtained. .