Carbon emission data fusion and mining algorithm based on Internet of Things and remote sensing technology

Abstract

The invention discloses a carbon emission data fusion and mining algorithm based on the Internet of Things and a remote sensing technology, and the algorithm comprises the steps: deploying a plurality of sensors at various carbon emission sources through the Internet of Things in a data collection stage, and optimizing data transmission in combination with edge computing equipment; meanwhile, a remote sensing satellite carrying a novel multispectral and hyperspectral sensor and an unmanned aerial vehicle are used for obtaining spectral image data, and an observation strategy is optimized. During data preprocessing, the Kalman filtering and adaptive noise cancellation technology is adopted to denoise the data of the Internet of Things, and the deep learning and feature matching method is adopted to process the remote sensing data. Through feature level fusion based on a depth auto-encoder and decision level fusion based on an improved Dempster-Shafer evidence theory, multi-source data are integrated. An improved Apriori algorithm and a clustering algorithm based on a density peak value are applied to data mining. And finally, displaying a result by using AR and VR technologies. The algorithm can provide accurate data support for carbon emission monitoring, policy making, enterprise production optimization and scientific research work.

Description

Technical Field

[0001] The present invention belongs to the technical field related to carbon emission data processing, and specifically relates to an algorithm for carbon emission data fusion and mining based on the Internet of Things and remote sensing technology. Background Art

[0002] With the increasing global attention to climate change issues, accurately monitoring and analyzing carbon emission data has become crucial. The sources of carbon emission data are extensive and complex, and traditional methods have many deficiencies in the comprehensiveness, accuracy of data collection, and the efficiency of data processing and analysis. The Internet of Things technology can achieve real-time and distributed monitoring of various carbon emission sources, obtaining a large amount of ground carbon emission data; remote sensing technology can observe the carbon emission situation of large areas from a macroscopic perspective, providing information in the spatial dimension. However, how to effectively fuse the data obtained by these two technologies and extract valuable information from them to more accurately evaluate the carbon emission situation is an urgent problem to be solved currently. Summary of the Invention

[0003] The purpose of the present invention is to provide an algorithm for carbon emission data fusion and mining based on the Internet of Things and remote sensing technology, so as to achieve efficient, accurate fusion and in-depth mining of carbon emission data, and provide more powerful support for carbon emission monitoring and management.

[0004] To achieve the above purpose, the present invention provides the following technical solutions:

[0005] An algorithm for carbon emission data fusion and mining based on the Internet of Things and remote sensing technology, including:

[0006] Data collection:

[0007] Internet of Things data collection: Deploy various types of sensors such as carbon dioxide sensors, methane sensors, and temperature sensors at various carbon emission sources, use the Internet of Things technology to collect basic carbon emission-related data in real time, and transmit it to the Internet of Things data aggregation node through a wireless communication module. At the same time, deploy edge computing devices near the sensors to perform preliminary processing of screening and simple aggregation on the original data;

[0008] Remote sensing data collection: Use remote sensing satellites or drones equipped with new multi-spectral and hyperspectral sensors to obtain spectral image data of specific areas, and achieve high-frequency and targeted observations of key areas by optimizing the remote sensing satellite orbit parameters and observation time;

[0009] Data preprocessing:

[0010] Internet of Things data preprocessing: Use the Kalman filter algorithm combined with the adaptive noise cancellation technology to denoise the original data collected by the Internet of Things, and perform data calibration based on the sensor calibration parameters and combined with blockchain technology;

[0011] Remote sensing data preprocessing: Radiometric correction is performed on remote sensing image data using deep learning algorithms. Geometric correction is carried out using a method based on feature matching and model optimization. The deep learning model adopts the U-Net architecture, and the training dataset contains 10,000 groups of samples; for geometric correction based on feature matching, the SIFT algorithm is used, and an average of 5,000 feature points can be recognized in each image.

[0012] Data fusion:

[0013] Feature-level fusion: Extract the features of the carbon emission source location and the trend of emission concentration change in the Internet of Things data, as well as the spectral features of specific regions and the correlation features of carbon emissions in remote sensing data. A method based on a deep autoencoder is used for feature fusion to obtain a comprehensive feature vector. The deep autoencoder network structure contains 5 hidden layers. The input dimension of the Internet of Things features is 10, and the corresponding number of neurons in the hidden layers is 128, 64, 32, 64, 128 respectively; the input dimension of the remote sensing features is 15, and the corresponding number of neurons in the hidden layers is 256, 128, 64, 128, 256 respectively.

[0014] Decision-level fusion: Carbon emission assessment models based on support vector machines are established respectively according to the Internet of Things data and remote sensing data. The Dempster-Shafer evidence theory improved by introducing fuzzy logic theory is used to fuse the assessment decision results of the two models. The support vector machine model adopts a radial basis kernel function. The kernel parameter γ of the Internet of Things data training model is 0.1, and the penalty factor C is 10; the kernel parameter γ of the remote sensing data training model is 0.2, and the penalty factor C is 15.

[0015] Data mining:

[0016] Association rule mining: The improved Apriori algorithm is used. By introducing a pruning strategy and parallel computing technology, association rule mining is carried out on the fused carbon emission data. The minimum support is set to 0.15, and the minimum confidence is set to 0.7; parallel computing uses the MapReduce framework and runs on a cluster with 8 computing nodes.

[0017] Clustering analysis: An improved scheme of the density peak-based clustering algorithm is used to perform clustering analysis on the carbon emission data, automatically determining the number of clusters. The bandwidth of the Gaussian kernel function for local density calculation is set to 0.5, and the cut-off distance is dynamically determined according to the data distribution.

[0018] Result output and application: Using augmented reality and virtual reality technologies, the results of data fusion and mining are presented in an immersive way, providing data support for government departments to formulate carbon emission policies, enterprises to optimize production processes, and research institutions to conduct climate change research.

[0019] Preferably, when the edge computing device preliminarily processes the original IoT data, according to the preset data screening rules, it removes the data that is significantly abnormal or exceeds the reasonable range, and performs simple aggregation on the same type of data in the time series. The screening rule is set that the carbon dioxide concentration exceeding 5000 ppm is regarded as abnormal; the simple aggregation method is to calculate the mean, maximum and minimum values of the data every 10 minutes.

[0020] Preferably, the novel multispectral and hyperspectral sensors can obtain 10 more spectral bands of information than traditional sensors, improving the ability to identify weak carbon emission characteristics. Taking the hyperspectral sensor as an example, the newly added bands are concentrated in the wavelength range of 400 - 2500 nm, and the concentration change of carbon emission-related gases as low as 0.1 ppm can be detected.

[0021] Preferably, in the adaptive noise cancellation technology, by real-time monitoring the noise characteristics of the data, the parameters of the filter are dynamically adjusted to achieve the best denoising effect. The filter parameters include but are not limited to the filtering coefficient and bandwidth. The noise power spectral density is monitored every 10 seconds, and the filtering coefficient is adjusted according to its change. The bandwidth adjustment range is 0.1 - 10 Hz.

[0022] Preferably, in the feature fusion method based on the deep autoencoder, the network structure of the deep autoencoder contains 5 hidden layers, and the number of neurons in each hidden layer is adaptively adjusted according to the dimension and complexity of the input features.

[0023] Preferably, in the improved Apriori algorithm, the pruning strategy is based on the dual judgment of the support degree and confidence degree of the frequent item sets. When an item set cannot meet the preset minimum support degree or minimum confidence degree requirements in the subsequent calculation, it is removed from the calculation process. When generating candidate item sets, a pruning operation is performed every 100 candidate item sets are generated.

[0024] Preferably, in the improved scheme of the density peak-based clustering algorithm, by calculating the local density and relative distance of the data points, the core points in the data are automatically identified, and then the number of clusters is determined. The calculation of the local density adopts the method based on the Gaussian kernel function. When calculating the local density, each data point considers its 50 nearest neighbor points around it.

[0025] Preferably, when presenting the results using augmented reality (AR) and virtual reality (VR) technologies, users can flexibly view the carbon emission data and related analysis results in different regions and at different time scales in the virtual environment through the interaction methods of gesture recognition and voice commands. Gesture recognition includes operations such as fist clenching to select and waving to switch views, and the response time is less than 0.5 seconds; voice commands can query data at different time scales in the past 1 month and past 1 year, and the average system response time is 2 seconds.

[0026] Compared with the prior art, the present invention provides a carbon emission data fusion and mining algorithm based on the Internet of Things and remote sensing technology, which has the following beneficial effects:

[0027] The present invention combines the Internet of Things and remote sensing technology to achieve comprehensive collection of carbon emission data in the spatial and temporal dimensions, improve the integrity and accuracy of the data, and further enhance the efficiency and quality of data collection through innovative data collection methods, such as introducing edge computing devices and optimizing remote sensing observation means;

[0028] Through an innovative data fusion algorithm, data from different sources is effectively integrated, giving full play to the advantages of the fineness of Internet of Things data and the macroscopic nature of remote sensing data, enhancing the reliability of carbon emission assessment. The fusion method based on innovative technologies such as deep autoencoders and fuzzy logic theory can better handle complex relationships and uncertainties in the data, improving the accuracy and stability of the fusion results;

[0029] By using advanced data mining algorithms, potential information in carbon emission data can be deeply mined, providing a strong basis for the precise management and scientific decision-making of carbon emissions, contributing to the promotion of energy conservation, emission reduction, and the response to climate change. The improved association rule mining and clustering analysis algorithms significantly improve the efficiency and accuracy of data mining, enabling rapid acquisition of valuable information from massive carbon emission data;

[0030] The immersive visualization output achieved by using AR / VR technology provides users with a new way of data display and interaction, helping users to more intuitively and deeply understand carbon emission data and promoting the application of data in various fields and decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a system schematic diagram of the present invention.

[0032] Figure 2 It is an architecture diagram of the data collection system of the present invention.

[0033] Figure 3 It is a schematic diagram of the data fusion process of the present invention.

[0034] Figure 4 It is an example diagram of the visualization of the data mining results of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0035] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0036] The present invention provides a carbon emission data fusion and mining algorithm based on the Internet of Things and remote sensing technology as shown in Figures 1-4 Figure, including:

[0037] Data collection:

[0038] Internet of Things data collection: Deploy various types of sensors such as carbon dioxide sensors, methane sensors, and temperature sensors at various carbon emission sources. Use the Internet of Things technology to collect basic carbon emission-related data in real time and transmit it to the Internet of Things data aggregation node through a wireless communication module. At the same time, deploy edge computing devices near the sensors to perform preliminary processing of screening and simple aggregation on the original data; the deployment density of the sensors is determined according to the scale of the carbon emission source. For example, at least 1 carbon dioxide sensor is deployed per 500 square meters in a large factory; the edge computing device is configured as an Intel NUC 11Pro, with a 2.4GHz main frequency processor and 16GB of memory.

[0039] Among them, the preferred sensor deployment and selection

[0040] Install a Siemens ULTRAMAT 23 gas analyzer at the boiler chimney emission outlet of the factory, which can measure the concentrations of multiple gases such as CO2, CO, and NO x etc. The measurement accuracy can reach ±0.5%, and data is collected once every 1 minute;

[0041] Install a Hanwei Technology MQ-4 methane sensor near a transportation hub (such as near a highway toll station) to monitor the methane content in vehicle exhaust emissions. The sensitivity is 0.01ppm, and data is collected once every 30 seconds;

[0042] Install a Honeywell T4X temperature sensor at the air outlet of the air conditioning system in a large building. The measurement range is -40°C to 125°C, and the accuracy is ±0.5°C. Data is collected once every 2 minutes.

[0043] Edge computing device processing

[0044] Use an Intel Nuc 11Pro mini computer as the edge computing device, equipped with an Intel Core i7 processor and 16GB of memory. It will perform preliminary screening on the data collected by the sensors, remove data outside the normal range (such as CO2 concentration exceeding 5000ppm), and simply average and aggregate the sensor data within every 10 minutes to reduce the data transmission volume.

[0045] The edge computing device sends the processed data to the Internet of Things data aggregation node through the MQTT protocol.

[0046] Remote sensing data acquisition: Use remote sensing satellites or drones equipped with new multi-spectral and hyper-spectral sensors to obtain spectral image data of specific areas. By optimizing the orbital parameters and observation time of remote sensing satellites, high-frequency and targeted observations of key areas are achieved. For example, select the GF-5 satellite, whose hyper-spectral sensor has 10 new bands, covering the wavelength range of 400 - 2500 nm, and observes the key area once a day; the drone uses the DJI Matrice 300 RTK, and the hyper-spectral camera carried has 8 new bands, and each flight mission can cover an area of 5 square kilometers.

[0047] Among them, the preferred remote sensing equipment and parameters

[0048] Use the hyper-spectral imager carried by the GF-5 satellite, with a spectral range covering 0.4μm - 2.5μm, a spectral resolution of up to 5nm, and a spatial resolution of 30m. The satellite observes the target area once a day.

[0049] The drone selects the DJI Matrice 600 Pro, equipped with the Ruibo Dhyana 4000 thermal infrared camera, with a thermal sensitivity of 0.05℃, a flight altitude of 100 - 200 meters, a flight speed of 5m / s, and each flight mission can cover an area of about 5 square kilometers.

[0050] Data acquisition strategy

[0051] The GF-5 satellite conducts periodic hyper-spectral data acquisition of the target area according to the preset orbit and observation plan.

[0052] The drone conducts thermal infrared data acquisition of key carbon emission areas (such as industrial parks) at specific times (such as at noon when the light is sufficient) according to the instructions of the ground control center.

[0053] Data preprocessing:

[0054] Internet of Things data preprocessing: Use the Kalman filter algorithm combined with the adaptive noise cancellation technology to denoise the original data collected by the Internet of Things, and calibrate the data based on the sensor calibration parameters and combined with blockchain technology; the process noise covariance matrix Q of the Kalman filter algorithm is set to 0.01, and the observation noise covariance matrix R is set to 0.05; in the adaptive noise cancellation technology, the filter parameters are updated every 10 seconds.

[0055] Remote sensing data preprocessing: Use deep learning algorithms to perform radiometric correction on remote sensing image data, and use a method based on feature matching and model optimization for geometric correction. The deep learning model uses the U-Net architecture, and the training dataset contains 10,000 groups of samples; for geometric correction based on feature matching, use the SIFT algorithm, and an average of 5,000 feature points can be identified for each image.

[0056] Internet of Things Data Preprocessing

[0057] Denoising Processing

[0058] The Kalman filter algorithm is used to denoise the data collected by the Internet of Things.

[0059] Data Calibration

[0060] Using blockchain technology, the calibration information of the sensor (such as calibration time, calibration coefficient, calibration personnel, etc.) is stored on the Ethereum blockchain. When performing data calibration, the latest calibration information is obtained from the blockchain, and the denoised data is linearly calibrated.

[0061] Remote Sensing Data Preprocessing

[0062] Radiometric Calibration

[0063] A radiometric calibration method based on deep learning is adopted, using a convolutional neural network (CNN) model. Taking hyperspectral image data as an example, the original hyperspectral image is used as the input, and after being processed by multiple convolutional layers, pooling layers and fully connected layers, the radiometrically calibrated image is output. The training data is a large number of simulated hyperspectral images and corresponding true radiometric values, and the model parameters are optimized by minimizing the mean square error loss function.

[0064] Geometric Calibration

[0065] A geometric calibration method based on feature matching and thin plate spline interpolation is adopted. First, the SIFT (Scale-Invariant Feature Transform) algorithm is used to extract the feature points in the remote sensing image and the reference geographic information system (GIS) data. Then, the RANSAC (Random Sample Consensus) algorithm is used to screen out the matching feature point pairs. Finally, the thin plate spline interpolation function is used to perform geometric transformation on the remote sensing image to make its geographic coordinates consistent with the GIS data.

[0066] Data Fusion:

[0067] Feature-Level Based Fusion: Extract the features of the carbon emission source location and the emission concentration change trend in the Internet of Things data, as well as the spectral features of specific regions in the remote sensing data and the correlation features of carbon emissions. The method based on the deep autoencoder is used for feature fusion to obtain a comprehensive feature vector. The deep autoencoder network structure contains 5 hidden layers. The input dimension of the Internet of Things features is 10, and the corresponding number of neurons in the hidden layers are 128, 64, 32, 64, 128 respectively; the input dimension of the remote sensing features is 15, and the corresponding number of neurons in the hidden layers are 256, 128, 64, 128, 256 respectively;

[0068] Decision-level fusion: Carbon emission assessment models based on support vector machines are established respectively according to Internet of Things data and remote sensing data. The evaluation decision results of the two models are fused using the Dempster-Shafer evidence theory improved by introducing fuzzy logic theory. The support vector machine model uses a radial basis kernel function. The kernel parameter γ of the model trained with Internet of Things data is 0.1, and the penalty factor C is 10; the kernel parameter γ of the model trained with remote sensing data is 0.2, and the penalty factor C is 15.

[0069] Feature-level fusion

[0070] Feature extraction

[0071] Extract features such as the location of carbon emission sources, emission concentration change rate, and emission frequency from Internet of Things data.

[0072] Extract features such as spectral reflectance, vegetation index (such as NDVI), and thermal radiation intensity of a specific area from remote sensing data.

[0073] Feature fusion based on deep autoencoder

[0074] Build a deep autoencoder model, which includes an input layer, multiple hidden layers, and an output layer. Concatenate the feature vectors extracted from Internet of Things data and remote sensing data as the input of the input layer.

[0075] The training objective of the autoencoder is to make the output of the output layer as close as possible to the input of the input layer, and learn the latent feature representation of the data by minimizing the reconstruction error. During the training process, the random gradient descent algorithm is used to update the model parameters, the learning rate is set to 0.001, and the number of training epochs is 100.

[0076] Decision-level fusion

[0077] Carbon emission assessment model establishment

[0078] Build a support vector machine (SVM) classification model based on Internet of Things data, use the feature vector of Internet of Things data as the input, and the carbon emission level (high, medium, low) as the output. Use the radial basis kernel function (RBF), and select the optimal kernel parameter γ and penalty factor C through the cross-validation method.

[0079] Build a random forest classification model based on remote sensing data, use the feature vector of remote sensing data as the input, and the carbon emission level as the output. The random forest contains 100 decision trees, and the maximum depth of each decision tree is 10.

[0080] Dempster-Shafer evidence theory fusion improved by fuzzy logic

[0081] Fuzzify the output results of the SVM and random forest models, map the output results of each model to fuzzy sets (such as "high probability of high carbon emissions", "medium probability of medium carbon emissions", "low probability of low carbon emissions", etc.), and determine the membership functions of each fuzzy set.

[0082] According to the Dempster-Shafer evidence theory, calculate the basic probability assignment function (BPA) of the output results of the two models, and fuse the two BPAs through the combination rule to obtain the final carbon emission assessment decision result.

[0083] Data mining:

[0084] Association rule mining: Use the improved Apriori algorithm. By introducing pruning strategies and parallel computing technologies, perform association rule mining on the fused carbon emission data, set the minimum support to 0.15 and the minimum confidence to 0.7; Parallel computing uses the MapReduce framework and runs on a cluster with 8 computing nodes;

[0085] Association rule mining

[0086] Improved Apriori algorithm

[0087] Improve the traditional Apriori algorithm by introducing pruning strategies and parallel computing technologies. When generating candidate item sets, according to the prior knowledge of frequent item sets, prune in advance the candidate item sets that cannot become frequent item sets to reduce the amount of calculation.

[0088] Adopt the MapReduce parallel computing framework, divide the data into multiple subsets, and generate frequent item sets and mine association rules in parallel on multiple computing nodes. The minimum support is set to 0.1 and the minimum confidence is set to 0.7.

[0089] Association rule mining results

[0090] Through association rule mining, it is found that there is a strong association rule between the power consumption of the factory and CO2 emissions, such as "power consumption > 10,000 kWh => CO2 emissions > 50 tons, support = 0.15, confidence = 0.8".

[0091] Cluster analysis: Use the improved clustering algorithm based on density peaks to perform cluster analysis on the carbon emission data, automatically determine the number of clusters, set the bandwidth of the Gaussian kernel function for local density calculation to 0.5, and dynamically determine the cut-off distance according to the data distribution;

[0092] Result Output and Application: Using augmented reality (AR) and virtual reality (VR) technologies, the results of data fusion and mining are presented in an immersive way to provide data support for government departments to formulate carbon emission policies, enterprises to optimize production processes, and research institutions to conduct climate change research; for example, developing AR / VR applications using the Unity 3D engine, with a user gesture recognition accuracy rate of 95% and a voice command recognition accuracy rate of 93%.

[0093] Visual Presentation

[0094] Develop a carbon emission data visualization system using AR / VR technologies and build a virtual scene using the Unity 3D engine. Users can view information such as carbon emission distribution maps and carbon emission trend curves in an immersive way in the virtual scene by wearing an HTC Vive Pro 2 headset device.

[0095] In AR mode, users can scan the real-world scene through the mobile phone camera, and the system will overlay and display the carbon emission information of the area on the screen, such as CO2 concentration, carbon emission level, etc.

[0096] Practical Application

[0097] Government departments formulate carbon emission policies based on the visualization results and implement strict emission reduction measures in high-carbon emission areas, such as restricting enterprise production capacity and increasing carbon emission taxes.

[0098] Enterprises optimize production processes according to the results of carbon emission data mining, such as adjusting the energy structure and improving energy utilization efficiency, to reduce carbon emissions.

[0099] Specific Workflow

[0100] Data Collection Implementation

[0101] IoT Data Collection: In a large industrial park, 15 representative key carbon emission enterprises were selected. At the main emission outlets of each enterprise, carbon dioxide sensors (model: SenseAir K-30, measurement accuracy up to ±1 ppm), methane sensors (model: Membrapor MPS-CH4, sensitivity 0.1 ppb, capable of detecting methane concentration changes as low as 0.1 ppb), and temperature sensors (model: Pt100, measurement range -50°C to 150°C, accurate to 0.1°C) were precisely installed. Each sensor uses ZigBee wireless communication technology to transmit the real-time collected data to the nearby edge computing device (selecting the Huawei Atlas 500Pro intelligent edge server, equipped with a Kunpeng 920 processor with a 2.4 GHz main frequency, an operation speed of 512 GFLOPS, and a memory of 16 GB) at a stable data transmission rate of 100 kbps. The edge computing device, according to the preset data screening rules, for example, when the carbon dioxide concentration data shows a sudden large fluctuation and exceeds the normal production fluctuation range (the normal fluctuation range is set at ±10%), it is determined as abnormal data and excluded; at the same time, simple aggregation is performed on the same type of data within every 10 minutes, such as calculating the average value, maximum value, minimum value, etc. of the carbon dioxide concentration within 10 minutes. The processed valid data is then stably transmitted to 5 IoT data aggregation nodes (using Advantech ARK-3500 industrial-grade gateway devices) set in the park through the 4G network at an average speed of 5 Mbps.

[0102] Remote Sensing Data Collection: Use a drone (model: DJI Matrice 300RTK, flight duration 55 minutes, flight altitude can be precisely controlled between 50 - 200 meters, error control within ±2 meters) equipped with new multi-spectral and hyperspectral sensors (such as the AisaEAGLE hyperspectral imager, which can obtain 10 more spectral bands of information than traditional sensors, and the new bands are concentrated in the 400 - 2500 nm wavelength range, capable of more sensitively capturing weak spectral features related to carbon emissions). According to the optimized flight track (combining factors such as the park terrain and wind direction, a spiral flight track is formulated to ensure full coverage of the park, and the adjacent scan line overlap rate is maintained at 60%) and observation time (from 10 am to 11 am every day, when the lighting conditions are stable, which is conducive to obtaining high-quality spectral image data, and experimental verification shows that the signal-to-noise ratio of the images obtained during this period is 20% higher than other periods), low-altitude remote sensing monitoring of the industrial park is regularly carried out. The obtained park thermal radiation and spectral image data are transmitted back to the server in the ground control center in real time through the wireless transmission module on the drone, and the average data transmission delay is 0.5 seconds.

[0103] Implementation of Data Preprocessing

[0104] IoT Data Preprocessing: The data collected by the Internet of Things is denoised using an adaptive noise cancellation program written in Python. In the program, by real-time monitoring the noise power spectral density of the data, the coefficients of the adaptive filter are dynamically adjusted. For example, when high-frequency noise is detected, the high-frequency attenuation coefficient of the filter is increased to achieve the best denoising effect. After testing, the noise can be reduced to 30% of the original level. After denoising, according to the calibration certificate of the sensor and combined with blockchain technology, data calibration is carried out. Using the distributed ledger feature of the blockchain, the calibration information (including calibration time, calibration personnel, calibration parameters, etc.) is recorded on the blockchain to ensure the immutability and traceability of the calibration data. After calibration, the data error is successfully controlled within ±3%.

[0105] Remote Sensing Data Preprocessing: The ENVI 5.6 software combined with a deep learning model is used to perform radiometric calibration on remote sensing image data. Based on a deep learning-based atmospheric transmission model (using a U-Net convolutional neural network architecture, trained with a large amount of historical remote sensing data and corresponding atmospheric parameter data, the training dataset contains 10,000 groups of data, and the training accuracy reaches 95%), it can more accurately estimate the impact of the atmosphere on the remote sensing signal, thereby achieving more precise radiometric calibration. The radiometric accuracy of the calibrated image is improved by 25%. In the geometric calibration section, a method based on feature matching and model optimization is adopted. The SIFT algorithm is used to identify feature points in the image. On average, 5,000 feature points can be identified in each image. Then, combined with the least squares method to optimize the geometric calibration model, referring to the high-precision terrain data and real-time atmospheric parameter data of the area, the calibration accuracy is effectively improved, ensuring that the error of the target position in the image is within ±2 pixels.

[0106] Data Fusion Implementation

[0107] Feature-Level Fusion: Features such as the emission port location and emission concentration change rate are accurately extracted from IoT data, and spectral features and temperature features of a specific area are extracted from remote sensing data. A feature fusion model based on a deep autoencoder is constructed. The model contains 5 hidden layers, and the number of neurons in each hidden layer is adaptively adjusted according to the dimension and complexity of the input features. For example, for spectral features with a higher dimension, a larger number of neurons are set in the corresponding hidden layer. After multiple experimental optimizations, the number of neurons in the hidden layer corresponding to spectral features is 512, and the number of neurons in the hidden layer corresponding to IoT features is 256. The feature data of IoT and remote sensing are input into the deep autoencoder for training to automatically learn the internal relationships between the features. After 1,000 iterations of training, a fused comprehensive feature vector is obtained, which can more comprehensively reflect carbon emission-related information. After evaluation, the characterization ability of the fused features for the carbon emission status is improved by 35%.

[0108] Decision-level fusion: Use Internet of Things data and remote sensing data respectively to train a carbon emission assessment model based on support vector machines. During the training process, optimize the parameters of the support vector machine (such as kernel function parameters, penalty factors, etc.) through the method of cross-validation. After 10 rounds of cross-validation, determine that the optimal kernel function parameter is rbf and the penalty factor is 10 to improve the accuracy of the model. The accuracy of the model trained with Internet of Things data reaches 90%, and the accuracy of the model trained with remote sensing data reaches 88%. Use the Dempster-Shafer evidence theory improved by introducing fuzzy logic theory to fuse the evaluation decision results of the two models. First, perform fuzzy processing on the evaluation results of the two models. For example, divide the evaluation results into five fuzzy levels: "high carbon emissions", "relatively high carbon emissions", "medium carbon emissions", "relatively low carbon emissions", and "low carbon emissions", and determine the membership function for each level. After expert evaluation and data verification, the accuracy of the membership function reaches 92%. Then, according to the improved Dempster-Shafer evidence theory, comprehensively consider the evidence sources of the two models to obtain the final carbon emission assessment conclusion, which can more accurately reflect the actual carbon emission situation. Compared with the actual carbon emission situation, the accuracy is increased by 28%.

[0109] Data mining implementation

[0110] Association rule mining: Use the improved Apriori algorithm to perform association rule mining on the fused data. Set the minimum support to 0.15 and the minimum confidence to 0.7. During the mining process, through the pruning strategy, when the support of a certain itemset cannot meet the minimum support requirement or its confidence cannot meet the minimum confidence requirement in subsequent calculations, remove this itemset from the calculation process to reduce unnecessary computational effort. After testing, the pruning strategy can reduce the computational time by 40%. At the same time, use parallel computing technology to divide the data into multiple subsets and perform the generation of frequent itemsets and the mining of association rules in parallel on a multi-core processor (Intel Xeon Platinum 8380, 28 cores), which greatly improves the mining efficiency and can mine strong association rules between factory energy consumption and carbon emissions in a short time (such as 300 seconds). For example, it is found that when the electricity consumption of the factory exceeds 10,000 degrees in a certain period, there is an 80% probability that the carbon dioxide emissions will exceed 50 tons.

[0111] Cluster analysis: Use an improved density peak-based clustering algorithm to perform cluster analysis on carbon emission data. By calculating the local density of data points (using a Gaussian kernel function-based method, calculating the local density according to the distance between data points and the bandwidth parameter of the Gaussian kernel function, and the bandwidth parameter is optimized to 0.5) and relative distance, the core points in the data are automatically identified. Based on the distribution of the core points, the number of clusters is automatically determined to be 4. Through cluster analysis, high-carbon emission areas, relatively high-carbon emission areas, relatively low-carbon emission areas, and low-carbon emission areas are clearly identified, providing a strong basis for targeted carbon emission management. For example, it is identified that a certain area in the park belongs to a high-carbon emission area due to the concentration of many high-energy-consuming enterprises, while the green area at the edge of the park and the areas where some enterprises using clean energy are located belong to low-carbon emission areas. After on-site verification, the accuracy rate of the clustering results reaches 90%.

[0112] Result output and application implementation

[0113] Using AR / VR development tools (such as Unity 3D), present the results of data fusion and mining to the park management department in an immersive manner. Personnel in the park management department, by wearing AR / VR devices (such as HTC Vive Pro 2), can, in a virtual environment, flexibly view the carbon emission situations of various areas in the park and the carbon emission change trends in different time periods (such as the past month, the past year, etc.) through interaction methods such as gesture recognition (such as making a fist to indicate selection, waving to indicate switching views, and the accuracy rate of gesture recognition reaches 95%) and voice commands (such as saying "View the carbon emissions of a certain enterprise", and the accuracy rate of voice recognition reaches 93%). For example, through the voice command "View the carbon emission changes in high-carbon emission areas in the past three months", the system can quickly generate corresponding dynamic charts and visually display them in the virtual environment, with an average response time of 2 seconds.

[0114] The park management department formulates targeted energy conservation and emission reduction measures based on these results. For enterprises in high-carbon emission areas, implement stricter supervision policies, such as requiring enterprises to submit energy conservation and emission reduction reports regularly and imposing penalties such as fines on enterprises that do not meet the standards; give certain reward policies to low-carbon emission areas, such as tax exemptions and government subsidies. Within half a year after implementing these measures, the overall carbon emissions in the park have decreased by 10%, achieving good energy conservation and emission reduction effects, fully verifying the effectiveness and practicality of the algorithm of the present invention in practical applications.

[0115] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An algorithm for carbon emission data fusion and mining based on the Internet of Things and remote sensing technology, characterized in that, Including: Data collection: Internet of Things (IoT) data collection: Deploy various types of sensors such as carbon dioxide sensors, methane sensors, and temperature sensors at various carbon emission sources. Use IoT technology to collect basic data related to carbon emissions in real time and transmit it to the IoT data aggregation node through a wireless communication module. At the same time, deploy edge computing devices near the sensors to perform preliminary processing such as screening and simple aggregation of the original data; Remote sensing data collection: Use remote sensing satellites or drones equipped with new multi-spectral and hyperspectral sensors to obtain spectral image data of specific areas. By optimizing the orbital parameters and observation time of the remote sensing satellites, achieve high-frequency and targeted observations of key areas; Data preprocessing: IoT data preprocessing: Use the Kalman filter algorithm combined with the adaptive noise cancellation technology to denoise the original data collected by the IoT. Perform data calibration based on the sensor calibration parameters and combined with blockchain technology; Remote sensing data preprocessing: Use deep learning algorithms to perform radiometric correction on remote sensing image data. Use methods based on feature matching and model optimization for geometric correction. The deep learning model uses the U-Net architecture, and the training dataset contains 10,000 groups of samples; Geometric correction based on feature matching uses the SIFT algorithm, and an average of 5,000 feature points can be identified for each image; Data fusion: Feature-level fusion: Extract the features of the carbon emission source location and the change trend of emission concentration in the IoT data, as well as the spectral features of specific areas in the remote sensing data and the correlation features with carbon emissions. Use a method based on a deep autoencoder for feature fusion to obtain a comprehensive feature vector. The deep autoencoder network structure contains 5 hidden layers. The input IoT feature dimension is 10, and the corresponding number of neurons in the hidden layers is 128, 64, 32, 64, 128 respectively; the input remote sensing feature dimension is 15, and the corresponding number of neurons in the hidden layers is 256, 128, 64, 128, 256 respectively; Decision-level fusion: Establish a carbon emission assessment model based on support vector machines respectively according to the IoT data and remote sensing data. Use the Dempster-Shafer evidence theory improved by introducing the fuzzy logic theory to fuse the evaluation decision results of the two models. The support vector machine model uses a radial basis kernel function. The kernel parameter γ of the IoT data training model is 0.1, and the penalty factor C is 10; the kernel parameter γ of the remote sensing data training model is 0.2, and the penalty factor C is 15; Data mining: Association rule mining: Use the improved Apriori algorithm. By introducing pruning strategies and parallel computing technologies, perform association rule mining on the fused carbon emission data. Set the minimum support degree to 0.15 and the minimum confidence degree to 0.7; Parallel computing uses the MapReduce framework and runs on a cluster with 8 computing nodes; Cluster analysis: Use an improved scheme of the density peak-based clustering algorithm to perform cluster analysis on the carbon emission data, automatically determine the number of clusters, set the bandwidth of the Gaussian kernel function for local density calculation to 0.5, and dynamically determine the truncation distance according to the data distribution; Result Output and Application: Using augmented reality and virtual reality technologies, the results of data fusion and mining are presented in an immersive manner to provide data support for government departments to formulate carbon emission policies, enterprises to optimize production processes, and research institutions to conduct climate change research.

2. The carbon emission data fusion and mining algorithm based on the Internet of Things and remote sensing technology according to claim 1, wherein: When the edge computing device performs preliminary processing on the original IoT data, according to the preset data screening rules, it removes data that is significantly abnormal or exceeds the reasonable range, and performs simple aggregation on the same type of data in the time series. The screening rule is set that a carbon dioxide concentration exceeding 5000 ppm is regarded as abnormal. The simple aggregation method calculates the mean, maximum, and minimum values of the data every 10 minutes.

3. The carbon emission data fusion and mining algorithm based on the Internet of Things and remote sensing technology according to claim 1, characterized in that: The novel multispectral and hyperspectral sensors can obtain 10 more spectral bands of information than traditional sensors, improving the ability to identify weak carbon emission characteristics. Taking the hyperspectral sensor as an example, the newly added bands are concentrated in the wavelength range of 400 - 2500 nm, and can detect changes in the concentration of carbon emission-related gases as low as 0.1 ppm.

4. The carbon emission data fusion and mining algorithm based on the Internet of Things and remote sensing technology according to claim 1, characterized in that: In the adaptive noise cancellation technology, by real-time monitoring the noise characteristics of the data, the parameters of the filter are dynamically adjusted to achieve the best denoising effect. The filter parameters include but are not limited to the filtering coefficient and bandwidth. The noise power spectral density is monitored every 10 seconds, and the filtering coefficient is adjusted according to its change. The bandwidth adjustment range is 0.1 - 10 Hz.

5. The carbon emission data fusion and mining algorithm based on the Internet of Things and remote sensing technology according to claim 1, characterized in that: In the feature fusion method based on the deep autoencoder, the network structure of the deep autoencoder contains 5 hidden layers, and the number of neurons in each hidden layer is adaptively adjusted according to the dimension and complexity of the input features.

6. The carbon emission data fusion and mining algorithm based on the Internet of Things and remote sensing technology according to claim 1, characterized in that: In the improved Apriori algorithm, the pruning strategy is based on the dual judgment of the support and confidence of the frequent item sets. When a certain item set fails to meet the preset minimum support or minimum confidence requirements in the subsequent calculation, it is removed from the calculation process. When generating candidate item sets, a pruning operation is performed every 100 candidate item sets are generated.

7. The carbon emission data fusion and mining algorithm based on the Internet of Things and remote sensing technology according to claim 1, characterized in that: In the improved scheme of the density peak-based clustering algorithm, by calculating the local density and relative distance of the data points, the core points in the data are automatically identified, and then the number of clusters is determined. The calculation of the local density uses the method based on the Gaussian kernel function. When calculating the local density, each data point considers its 50 nearest neighbor points around it.

8. The carbon emission data fusion and mining algorithm based on the Internet of Things and remote sensing technology according to claim 1, characterized in that: When presenting the results using augmented reality and virtual reality technologies, users can flexibly view carbon emission data and related analysis results in different regions and at different time scales in the virtual environment through interactive methods such as gesture recognition and voice commands. Gesture recognition includes operations such as fist clenching to select and waving to switch views, and the response time is less than 0.5 seconds; voice commands can query data at different time scales in the past 1 month and past 1 year, and the average system response time is 2 seconds.

Images