City energy consumption analysis method fusing internet of things and big data

Abstract

The application discloses a city energy consumption analysis method fusing internet of things and big data, the method collects city district energy consumption data and environmental parameters through an internet of things sensing layer, and pre-processes collected data; a big data analysis platform is used to calculate correlation coefficients between various environmental parameters and energy consumption indexes, and key factors having a significant influence on city energy consumption are screened out; real-time data collected by internet of things equipment, historical energy consumption data, city building information, population distribution data and meteorological data are fused; a city energy consumption prediction model is constructed according to the key factors and fused data, a multi-dimensional space-time sequence analysis method is used to predict energy consumption trends in different regions; the application fuses internet of things and big data technologies, realizes accurate analysis and prediction of city energy consumption, and provides a scientific basis for city energy management and energy saving and emission reduction.

Description

Technical Field

[0001] This invention belongs to the field of energy analysis technology, specifically involving a method for analyzing urban energy consumption that integrates the Internet of Things and big data. Background Technology

[0002] Traditional energy consumption analysis methods mainly rely on manual meter reading and periodic statistics, which suffer from problems such as untimely data acquisition, low analytical accuracy, and inability to perform real-time monitoring. Most existing energy analysis systems lack the ability to integrate multi-source data, making it difficult to comprehensively consider multi-dimensional influencing factors such as environmental factors, building characteristics, and population distribution. This results in low accuracy in energy consumption prediction and fails to provide effective support for urban energy management and optimization. In addition, traditional methods usually focus on macro-level statistical analysis and lack mechanisms for identifying and warning of abnormal energy consumption in urban micro-regions, making it difficult to detect energy waste and abnormal consumption problems in a timely manner.

[0003] With the development of IoT and big data technologies, it has become possible to conduct refined analysis and prediction of urban energy consumption using advanced technologies. However, there is currently a lack of an effective method to deeply integrate IoT sensing data with multi-source urban data for energy consumption analysis. Summary of the Invention

[0004] This invention provides a method for analyzing urban energy consumption that integrates the Internet of Things (IoT) and big data. By using IoT sensing layer data acquisition and multi-source data fusion technology, combined with spatiotemporal attention neural networks and multi-level early warning mechanisms, it aims to accurately monitor urban energy consumption patterns, accurately predict energy consumption trends, and promptly identify abnormal areas.

[0005] A method for analyzing urban energy consumption that integrates the Internet of Things and big data, the method comprising the following steps:

[0006] Step 1: Collect energy consumption data and environmental parameters of various urban areas through the Internet of Things (IoT) sensing layer. Preprocess the collected data, including data cleaning, outlier detection, and missing value imputation, to form a standardized urban energy consumption big data set.

[0007] Step two involves calculating the correlation coefficients between various environmental parameters and energy consumption indicators based on a big data analysis platform, and then identifying key factors that have a significant impact on urban energy consumption.

[0008] Step 3: Perform multi-source data fusion on real-time data, historical energy consumption data, urban building information, population distribution data, and meteorological data collected by IoT devices.

[0009] Step four: Based on the key factors and the fused data, construct an urban energy consumption prediction model and use a multidimensional spatiotemporal sequence analysis method to predict energy consumption trends in different regions.

[0010] Step 5: Generate an urban energy consumption analysis report, including an energy utilization efficiency assessment and identification of areas with abnormal energy consumption, and issue early warnings for abnormal energy consumption situations.

[0011] The IoT sensing layer includes a regional concentrator and terminal sensing devices. The regional concentrator collects data from the terminal devices via the MQTT protocol. The terminal sensing devices include intelligent monitoring terminals for power consumption, gas consumption, heat consumption, and water consumption, as well as environmental sensors for temperature, humidity, wind speed, and air quality.

[0012] The data collection frequency is set as follows: electricity data every 5 minutes, gas data every 30 minutes, heat and water resource data every hour, and environmental parameters every 10 minutes; the regional concentrator calculates the 5-minute average, maximum, and minimum values ​​through edge computing.

[0013] Methods for preprocessing the collected data include:

[0014] Using the sliding window method to detect outliers, for time points Data In a window size of Calculate the average value within the time window. and standard deviation :

[0015] ; ;like Then The value was identified as an outlier, among which The preset threshold is set between 2.5 and 3.5, depending on the data type.

[0016] For detected outliers and missing values ​​in the dataset, a combination of linear interpolation and exponentially weighted moving average is used for imputation. The imputation formula is as follows: ; The value to fill in, These are linear interpolation weights, with values ​​ranging from [0.4, 0.6]. This is the historical data attenuation factor, with a value range of [0.7, 0.9]. The historical time window length is taken as 24 hours.

[0017] Methods for calculating the correlation coefficients between various environmental parameters and energy consumption indicators include:

[0018] For environmental parameters Energy consumption indicators The correlation was calculated based on the Kendall rank correlation coefficient. The calculation formula is:

[0019] ,in, Indicates the number of cooperative pairs. Indicates the number of non-cooperative pairs. , The total number of samples, for Number of combinations of repeated values The number of combinations of repeated values ​​in Y;

[0020] Perform a significance test on the correlation coefficient; if The value is less than the significance level. , ,and Therefore, this environmental parameter is considered a key factor affecting energy consumption. The threshold for key factor screening is set to 0.4.

[0021] Step three includes methods for multi-source data fusion, including:

[0022] Real-time data collected from IoT devices Historical energy consumption data Urban building information Population distribution data and meteorological data Perform multi-source data fusion;

[0023] Spatiotemporal alignment and scale standardization are performed on each data source to convert them into a unified feature representation.

[0024] ; ; ; ; ;

[0025] in For data source Standardized data, and Data sources The mean and standard deviation, ;

[0026] For each data source, extract its key feature vectors;

[0027] ; ; ; ; in, This refers to the feature extraction function for the corresponding data source;

[0028] A weighted fusion method based on an attention mechanism is adopted;

[0029] in, For data source Attention weights, and satisfying ;

[0030] The contribution of each data source to energy consumption forecasting is dynamically adjusted, and the contribution calculation formula is as follows:

[0031] ;in, For data source The score, , and These are learnable parameters.

[0032] The method for constructing the urban energy consumption prediction model in step four is based on a spatiotemporal attention neural network, including:

[0033] The urban area is divided into grids, and each grid cell is represented as... ,in , Use grid coordinates; construct a spatiotemporal feature tensor It contains grid cells. In time Energy consumption data and environmental parameters;

[0034] Define the spatiotemporal attention weight matrix The calculation formula is:

[0035] ;in, and For learnable network parameters, This represents the size of the historical time window, with a value of 24 hours.

[0036] Prediction results based on attention mechanism The calculation formula is:

[0037] ;in, express Attention weight at any moment and These are model parameters;

[0038] extract Energy consumption data in the grid is used as the basis for each grid cell. The predicted energy consumption value at any given time.

[0039] The training method for the spatiotemporal attention neural network includes:

[0040] Constructing the loss function Simultaneously considering the constraints of prediction accuracy and spatiotemporal smoothness:

[0041] ,in, The mean squared error between the predicted and actual values. For spatial smoothness regularization, For time smoothness regularization, , and These are weighting coefficients, with values ​​of 0.7, 0.15, and 0.15 respectively.

[0042] The formula for calculating the spatial smoothness regularization term is:

[0043]

[0044] in, Represents grid cells The neighborhood group, Spatial correlation weights are calculated based on the distance between grid cells. ,in, The distance between two grid cells. This is the distance attenuation parameter, and its value is the side length of the grid cell;

[0045] The formula for calculating the time smoothness regularization term is:

[0046]

[0047] The Adam optimization algorithm is used to minimize the loss function L to obtain the optimal parameters of the model; the initial learning rate is set to 0.001, and it decays to 0.8 times the original value every 30 training cycles.

[0048] In step five, the method for generating the urban energy consumption analysis report includes:

[0049] Calculate the energy efficiency index of each region The formula is:

[0050] ;in, For grid Economic output, For the corresponding energy consumption, The normalization coefficient is the average energy price in the city.

[0051] Identify areas with abnormal energy consumption, if the area Energy consumption growth rate satisfy:

[0052] If it is an abnormal energy consumption area, then it is marked as such. and These are the mean and standard deviation of the city's energy consumption growth rate, respectively.

[0053] Based on the analysis of spatiotemporal energy consumption patterns and the identification of abnormal areas, energy-saving optimization suggestions are generated, including energy structure adjustment schemes for high-energy-consuming areas and time-specific energy consumption control strategies.

[0054] Calculate the seasonal fluctuation range of energy consumption in different urban areas based on historical data. The calculation formula is: ; ;in, and Other regions The mean and standard deviation of energy consumption for the same historical period. This is the early warning coefficient;

[0055] Energy Consumption Anomaly Index (EAI): ;

[0056] in, For the region Current energy consumption.

[0057] Set a three-level early warning threshold. The values ​​are 10%, 20%, and 30% respectively. A red alert was issued at that time. An orange alert was issued at that time. A yellow alert was issued at that time.

[0058] Compared with the prior art, the beneficial effects of this invention are:

[0059] This invention achieves refined collection of urban energy consumption data through the Internet of Things (IoT) sensing layer, uses a big data analysis platform to screen key influencing factors, adopts a multi-source data fusion method based on the attention mechanism to improve the value of data utilization, constructs a spatiotemporal attention neural network model to achieve high-precision energy consumption prediction, and establishes a multi-level energy consumption anomaly early warning mechanism. It significantly improves the accuracy, timeliness, and reliability of urban energy consumption analysis and provides a scientific basis for urban energy optimization management. Attached Figure Description

[0060] Figure 1 This is a flowchart of the urban energy consumption analysis method integrating the Internet of Things and big data of the present invention. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention are described clearly and completely below. Obviously, the described embodiments are only a part of the embodiments of this invention, not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0062] like Figure 1 The diagram shown is a flowchart of the urban energy consumption analysis method integrating the Internet of Things and big data according to the present invention. The method includes the following steps:

[0063] Step 1: Collect energy consumption data and environmental parameters of various urban areas through the Internet of Things (IoT) sensing layer. Preprocess the collected data, including data cleaning, outlier detection, and missing value imputation, to form a standardized urban energy consumption big data set.

[0064] The IoT sensing layer includes regional concentrators and terminal sensing devices. The regional concentrators collect data from the terminal devices via the MQTT protocol. The terminal sensing devices include intelligent monitoring terminals for power consumption, gas consumption, heat consumption, and water consumption, as well as environmental sensors for temperature, humidity, wind speed, and air quality.

[0065] The regional concentrators utilize high-performance edge computing chips, supporting concurrent connections from up to 500 terminal devices and featuring local data caching to ensure data integrity even during network outages. The MQTT protocol was chosen due to its low bandwidth consumption, reliable message delivery mechanism, and support for QoS (Quality of Service) levels, making it particularly suitable for city-scale IoT data transmission scenarios. In practical deployments, 3-5 regional concentrators can be deployed per square kilometer, depending on the city size and monitoring density, to form a comprehensive monitoring network.

[0066] Data collection frequencies are set as follows: electricity data every 5 minutes, gas data every 30 minutes, heat and water resource data every hour, and environmental parameters every 10 minutes. The regional concentrator calculates the 5-minute average, maximum, and minimum values ​​using edge computing. Electricity consumption fluctuates rapidly, so collecting data every 5 minutes captures peak and off-peak periods. Gas consumption is relatively stable, but it's necessary to monitor seasonal changes in consumption such as cooking and heating. Heat and water resource consumption exhibit a slower trend. Environmental parameters are collected every 10 minutes, effectively capturing the impact of weather changes on energy consumption. In practical applications, edge computing not only calculates statistics but also performs preliminary data compression, reducing the original data volume by approximately 60%, significantly lowering data transmission and storage costs.

[0067] Methods for preprocessing the collected data include:

[0068] Using the sliding window method to detect outliers, for time points Data In a window size of Calculate the average value within the time window. and standard deviation :

[0069] ; ;like Then The value was identified as an outlier, among which The preset threshold is set between 2.5 and 3.5, depending on the data type.

[0070] For detected outliers and missing values ​​in the dataset, a combination of linear interpolation and exponentially weighted moving average is used for imputation. The imputation formula is as follows: ; The value to fill in, These are linear interpolation weights, with values ​​ranging from [0.4, 0.6]. This is the historical data attenuation factor, with a value range of [0.7, 0.9]. The historical time window length is taken as 24 hours.

[0071] The sliding window method dynamically adjusts the window size *w* based on the data type: for electricity data, *w*=12, equivalent to a 1-hour data window; for gas data, *w*=8, equivalent to a 4-hour data window; and for heat and water resource data, *w*=12, equivalent to a 12-hour data window. The preset threshold *α* also varies depending on the data type: for electricity data, *α*=3.5, due to its greater volatility; for gas and heat data, *α*=3.0; and for water resource data, *α*=2.5, as its changes are typically more gradual. In practical applications, within a month, outliers in electricity data (approximately 1.8%), gas data (approximately 1.2%), heat data (approximately 0.9%), and water resource data (approximately 0.5%) were detected in a large urban area, indicating that this method has high detection accuracy.

[0072] Step two involves calculating the correlation coefficients between various environmental parameters and energy consumption indicators based on a big data analysis platform, and then identifying key factors that have a significant impact on urban energy consumption.

[0073] Methods for calculating the correlation coefficients between various environmental parameters and energy consumption indicators include:

[0074] For environmental parameters Energy consumption indicators The correlation was calculated based on the Kendall rank correlation coefficient. The calculation formula is:

[0075] ,in, Indicates the number of cooperative pairs. Indicates the number of non-cooperative pairs. , The total number of samples, for Number of combinations of repeated values The number of combinations of repeated values ​​in Y;

[0076] Perform a significance test on the correlation coefficient; if The value is less than the significance level. , ,and Therefore, this environmental parameter is considered a key factor affecting energy consumption. The threshold for key factor screening is set to 0.4.

[0077] The primary reason for choosing Kendall's rank correlation coefficient over Pearson's correlation coefficient is its robustness to nonlinear relationships and outliers. In urban energy data analysis, environmental parameters such as temperature and energy consumption often exhibit nonlinear relationships (e.g., a U-shaped relationship caused by cooling and heating demand), and Kendall's rank correlation coefficient can more accurately capture these relationships. In practical applications, taking one year's data from a certain city as an example, the Kendall's rank correlation coefficient between temperature and electricity consumption is 0.68, and the correlation coefficient with gas consumption is -0.72, indicating that electricity consumption increases while gas consumption decreases when temperature rises. The correlation coefficients for humidity are significantly lower, only 0.23 and 0.18, failing the significance test and therefore not considered a key factor.

[0078] In the significance test, we used the Bootstrap method to generate 10,000 resampling operations to ensure the reliability of the p-value calculation. The key factor screening threshold τth of 0.4 has been experimentally verified as the optimal balance point in the energy field. Lower thresholds introduce too much noise, while excessively high thresholds may ignore some meaningful influencing factors. In a typical energy analysis case in a northern city, the final selected key factors usually include: temperature (τ=±0.65-0.55), weekday indicators (τ=±0.50-0.55), and population activity density (τ=±0.40-0.50), while factors such as humidity, wind speed, and air pressure usually fail the screening.

[0079] Step 3: Perform multi-source data fusion on real-time data, historical energy consumption data, urban building information, population distribution data, and meteorological data collected by IoT devices.

[0080] Step three includes methods for multi-source data fusion, including:

[0081] Real-time data collected from IoT devices Historical energy consumption data Urban building information Population distribution data and meteorological data Perform multi-source data fusion;

[0082] Spatiotemporal alignment and scale standardization are performed on each data source to convert them into a unified feature representation.

[0083] ; ; ; ; ;

[0084] in For data source Standardized data, and Data sources The mean and standard deviation, .

[0085] In the process of multi-source data standardization, in addition to mean-standard deviation standardization, robust standardization methods are applied to data sources with outlier distributions. Specifically, for data with obvious long-tail distribution characteristics (such as population distribution data for certain regions and energy consumption data for special buildings), a standardization method based on median and interquartile range is adopted. Where IQR is the interquartile range.

[0086] For each data source, extract its key feature vectors;

[0087] ; ; ; ; in, For the corresponding data source, feature extraction functions are defined as follows: For IoT real-time data (DI), time series features such as peak-to-valley ratio, volatility, and trend slope are extracted; for historical energy consumption data (DH), periodic features such as intraday, intraweek, and seasonal patterns are extracted; for urban building information (DB), static features such as building density, years of use, and functional type are extracted; for population distribution data (DP), features such as population density, mobility, and activity periods are extracted; and for meteorological data (DW), features such as temperature fluctuation range and extreme weather event markers are extracted.

[0088] A weighted fusion method based on an attention mechanism is adopted;

[0089] in, For data source Attention weights, and satisfying ;

[0090] The contribution of each data source to energy consumption forecasting is dynamically adjusted, and the contribution calculation formula is as follows:

[0091] ;in, For data source The score, , and These are learnable parameters.

[0092] Step four: Based on the key factors and the fused data, construct an urban energy consumption prediction model and use a multidimensional spatiotemporal sequence analysis method to predict energy consumption trends in different regions.

[0093] The method for constructing the urban energy consumption prediction model in step four is based on a spatiotemporal attention neural network, including:

[0094] The urban area is divided into grids, and each grid cell is represented as... ,in , Use grid coordinates; construct a spatiotemporal feature tensor It contains grid cells. In time Energy consumption data and environmental parameters;

[0095] Define the spatiotemporal attention weight matrix The calculation formula is:

[0096] ;in, and For learnable network parameters, This refers to the size of the historical time window, with a value of 24 hours. Setting it to 24 hours means the model will consider the impact of energy consumption patterns over the past day on current predictions; the softmax function ensures that the sum of attention weights for all historical moments is 1, forming a probability distribution; in model analysis, it was found that the weight distribution usually exhibits obvious peak and valley characteristics: for electricity consumption, the data from the first 1-2 hours and the data from the same period in the first 24 hours (daily cycle) have significantly higher weights than other time periods; for gas and heat consumption, it tends to assign higher weights to the data from the first 4-6 hours and the data from the first 168 hours (weekly cycle).

[0097] Prediction results based on attention mechanism The calculation formula is:

[0098] ;in, express Attention weight at any moment and These are the model parameters; during the model training phase, the attention weight parameters are optimized through backpropagation. , and This allows the model to automatically identify and strengthen data sources that contribute significantly to the prediction results. In practical applications, we observed significant time-varying characteristics in the weights of data sources: during the high-temperature period in summer, meteorological data... weight It can reach above 0.35; during weekday peak hours, population distribution data weight Typically between 0.25 and 0.30; however, during stable nighttime periods, historical energy consumption data... weight It could be as high as 0.40.

[0099] extract Energy consumption data in the grid is used as the basis for each grid cell. The predicted energy consumption value at any given time.

[0100] The training method for the spatiotemporal attention neural network includes:

[0101] Constructing the loss function Simultaneously considering the constraints of prediction accuracy and spatiotemporal smoothness:

[0102] ,in, The mean squared error between the predicted and actual values. For spatial smoothness regularization, For time smoothness regularization, , and These are weighting coefficients, with values ​​of 0.7, 0.15, and 0.15 respectively.

[0103] The formula for calculating the spatial smoothness regularization term is:

[0104]

[0105] in, Represents grid cells The neighborhood group, Spatial correlation weights are calculated based on the distance between grid cells. ,in, The distance between two grid cells. This is the distance attenuation parameter, and its value is the side length of the grid cell.

[0106] The granularity of urban area grid division needs to be determined based on the city size and energy consumption distribution characteristics. In densely populated urban centers, the grid cell size can be set to 500m × 500m to capture fine differences in energy consumption; in suburban and industrial areas, the grid cell size can be appropriately increased to 1km × 1km to balance computational efficiency and analytical accuracy. The spatiotemporal feature tensor Ft associated with each grid cell ci,j typically contains 20-30 feature dimensions, covering the grid's energy consumption data (electricity, gas, heat, water resources), environmental parameters (temperature, humidity, wind speed, etc.), temporal features (hours, dates, weekday markers, etc.), and regional features (building density, population density, etc.).

[0107] The formula for calculating the time smoothness regularization term is: The Adam optimization algorithm is used to minimize the loss function L to obtain the optimal parameters of the model; the initial learning rate is set to 0.001, and it decays to 0.8 times the original value every 30 training cycles.

[0108] Step 5: Generate an urban energy consumption analysis report, including an energy utilization efficiency assessment and identification of areas with abnormal energy consumption, and issue early warnings for abnormal energy consumption situations.

[0109] In step five, the method for generating the urban energy consumption analysis report includes:

[0110] Calculate the energy efficiency index of each region The formula is:

[0111] ;in, For grid Economic output, For the corresponding energy consumption, The normalization coefficient is the average energy price in the city.

[0112] Identify areas with abnormal energy consumption, if the area Energy consumption growth rate satisfy:

[0113] If it is an abnormal energy consumption area, then it is marked as such. and These are the mean and standard deviation of the city's energy consumption growth rate, respectively.

[0114] Based on the analysis of spatiotemporal energy consumption patterns and the identification of abnormal areas, energy-saving optimization suggestions are generated, including energy structure adjustment schemes for high-energy-consuming areas and time-specific energy consumption control strategies.

[0115] Calculate the seasonal fluctuation range of energy consumption in different urban areas based on historical data. The calculation formula is: ; ;in, and Other regions The mean and standard deviation of energy consumption for the same historical period. This is the early warning coefficient;

[0116] Energy Consumption Anomaly Index (EAI): ;

[0117] in, For the region Current energy consumption.

[0118] Set a three-level early warning threshold. The values ​​are 10%, 20%, and 30% respectively. A red alert was issued at that time. An orange alert was issued at that time. A yellow alert was issued at that time.

[0119] The warning coefficient γ is typically determined based on the historical fluctuations in urban energy consumption. For large cities, it is usually set at 2.0-2.5, while for small and medium-sized cities or areas with significant energy consumption fluctuations, it can be set at 3.0-3.5. A larger γ value will broaden the range of normal fluctuations and reduce the false alarm rate, but may cause minor anomalies to go undetected in time. A smaller γ value increases system sensitivity but may increase the false alarm rate. In practical applications, the γ value can be dynamically adjusted according to different seasons and regional characteristics; for example, a smaller value (e.g., 2.0) can be used in winter and summer, while a larger value (e.g., 2.5 or 3.0) can be used in spring and autumn.

[0120] The Energy Consumption Anomaly Index (EAI) is designed to account for percentage deviations outside the normal range, allowing for direct comparison of the degree of anomalies across different energy types and regions of varying sizes. EAI is a two-way indicator, detecting both abnormally high consumption (potentially indicating energy waste or equipment malfunction) and abnormally low consumption (potentially indicating data acquisition errors or power outages). In practice, high consumption anomalies account for over 75% of all warnings, while low consumption anomalies account for approximately 25%.

[0121] Warning threshold , , The system is based on extensive historical data analysis and practical application experience. A 10% deviation (yellow alert) typically indicates a minor anomaly in the energy system, requiring attention but not immediate intervention; a 20% deviation (orange alert) indicates a significant anomaly, requiring inspection and handling within 24 hours; and a deviation exceeding 30% (red alert) signifies a serious anomaly requiring immediate response. Alert information is automatically pushed to relevant management personnel via an IoT platform, triggering different response procedures based on the alert level. A red alert simultaneously notifies the regional energy management center, emergency response team, and relevant facility administrators; an orange alert notifies the regional energy management center and relevant facility administrators; and a yellow alert only notifies relevant facility administrators to conduct routine inspections.

[0122] This method, combining a three-tiered early warning mechanism with an automated response process, significantly improves the efficiency of handling urban energy anomalies. In a pilot application in a provincial capital city, after implementing this early warning mechanism, the average response time for energy anomalies decreased from 48 hours to 12 hours, and the anomaly handling completion rate increased from 85% to over 95%, effectively avoiding resource waste and economic losses caused by prolonged energy anomalies.

[0123] The following is a detailed explanation of the application effects of this invention through a practical example. From January 2023 to December 2023, a certain city applied the method of this invention to analyze urban energy consumption. The specific implementation process is as follows:

[0124] Firstly, regarding the deployment of the IoT sensing layer, the city has installed 125 regional concentrators and 8,500 smart terminal monitoring devices, covering 80% of the buildings and public areas in the urban area. The smart terminal devices include 3,200 power monitoring terminals, 2,800 gas monitoring terminals, 1,500 heat monitoring terminals, and 1,000 water resource monitoring terminals, as well as 3,600 environmental parameter sensors. All terminal devices collect data at set frequencies and transmit it to the regional concentrators via the MQTT protocol, then upload it to the city's big data center via the 5G network, with an average daily data volume of 1.2TB.

[0125] During the data preprocessing stage, the system detected approximately 620,000 abnormal data entries (1.4% of the total data volume) and 350,000 missing data entries (0.8% of the total data volume). Using the sliding window anomaly detection method and the imputation method combining linear interpolation and exponentially weighted moving averages of this invention, over 95% of the data problems were successfully repaired. Taking the electricity data of a commercial area on January 15th as an example, the system detected a sudden increase in electricity consumption at 14:25, reaching 4.8 times the average (more than 3.5 times the average). ), was identified as an outlier, and used , Data repair was performed, correcting the outlier value of 189.6 kWh to a reasonable value of 42.3 kWh.

[0126] In the key factor screening phase, the system conducted Kendall rank correlation analysis on more than 30 environmental parameters and urban characteristics. The results showed a high correlation coefficient between temperature and electricity consumption. It is 0.71 ( ), correlation coefficient with gas consumption -0.68 ( ); Correlation coefficient between weekday indicators and electricity consumption It is 0.52 ( ); Correlation coefficient between population activity density and electricity consumption It is 0.48 ( ); Correlation coefficient between building density and heat consumption It is 0.43 ( These factors all passed the significance test and It was identified as a key influencing factor.

[0127] In the multi-source data fusion stage, the system standardized and extracted features from real-time IoT data, historical energy consumption data, urban building information (including information on the type, area, and age of 52,000 buildings), population distribution data, and meteorological data. Weights were dynamically adjusted using an attention mechanism; during the high-temperature period in summer, meteorological data received higher weights. The highest value reached 0.38; during weekday peak hours, the weight of population distribution data... The weighting reached 0.27; during the nighttime period, the weighting of historical energy consumption data... It reached 0.42.

[0128] In the predictive model building phase, the city was divided into 512 500m×500m grid cells, and a spatiotemporal feature tensor containing 25 feature dimensions was constructed. Model training used data from January to October 2023, validation used November data, and testing used December data. The final model achieved an average prediction error of 8.7% on the test set, significantly outperforming traditional time series methods (15.3%) and simple neural network models (12.1%).

[0129] During the energy consumption analysis and early warning phase, the system calculated the Energy Efficiency Index (EEI) for each grid unit, identifying 32 areas with low energy efficiency (mainly concentrated in old urban areas and some industrial zones). Through monitoring the Energy Consumption Anomaly Index (EAI), a total of 84 red alerts, 235 orange alerts, and 612 yellow alerts were issued throughout 2023. For example, on December 20th, the energy consumption of a certain commercial center was 36.2% higher than the historical average for the same period, with an EAI value of 36.2%, prompting the system to immediately issue a red alert.

[0130] After responding to the warning, the management found that multiple central air conditioning units in the commercial center were running simultaneously but there were few customers. They promptly adjusted the operating parameters, avoiding a waste of approximately 12,000 kilowatt-hours of electricity, which saved about 9,600 yuan based on local electricity prices.

[0131] After a year of system operation, the city achieved significant energy-saving results: the city's energy utilization efficiency increased by 8.7%, saving approximately 68,000 tons of standard coal annually, reducing carbon dioxide emissions by approximately 175,000 tons, and saving approximately 120 million yuan in energy costs. Meanwhile, the average response time for abnormal energy consumption events decreased from 37 hours before implementation to 9 hours, and the completion rate of abnormal event handling increased from 82% to 97%, providing the city's energy management department with a scientific basis for decision-making and achieving refined management and optimized allocation of energy use.

[0132] This case study fully demonstrates the significant effects of integrating the Internet of Things and big data in urban energy consumption analysis in practical applications. It not only improves energy efficiency and reduces energy costs, but also reduces carbon emissions, which is of great significance for promoting green and low-carbon urban development.

[0133] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for analyzing urban energy consumption that integrates the Internet of Things and big data, characterized in that: The method includes the following steps: Step 1: Collect energy consumption data and environmental parameters of various urban areas through the Internet of Things sensing layer, and preprocess the collected data, including data cleaning, outlier detection and missing value imputation, to form a standardized urban energy consumption big data set. Step 2: Calculate the correlation coefficients between various environmental parameters and energy consumption indicators based on the big data analysis platform, and screen out the key factors that have a significant impact on urban energy consumption. For environmental parameters Energy consumption indicators The correlation was calculated based on the Kendall rank correlation coefficient. The calculation formula is: ,in, Indicates the number of cooperative pairs. Indicates the number of non-cooperative pairs. , The total number of samples, for Number of combinations of repeated values The number of combinations of repeated values ​​in Y; Perform a significance test on the correlation coefficient; if The value is less than the significance level. , ,and Therefore, this environmental parameter is considered a key factor affecting energy consumption. The threshold for key factor screening is set to 0.4; Step 3: Perform multi-source data fusion on real-time data, historical energy consumption data, urban building information, population distribution data, and meteorological data collected by IoT devices; Real-time data collected from IoT devices Historical energy consumption data Urban building information Population distribution data and meteorological data Perform multi-source data fusion; Spatiotemporal alignment and scale standardization are performed on each data source to convert them into a unified feature representation. ； ； ； ； ； in For data source Standardized data, and Data sources The mean and standard deviation, ; For each data source, extract its key feature vectors; ; ; ; ; in, This refers to the feature extraction function for the corresponding data source; A weighted fusion method based on an attention mechanism is adopted; in, For data source Attention weights, and satisfying ; The contribution of each data source to energy consumption forecasting is dynamically adjusted, and the contribution calculation formula is as follows: ;in, For data source The score, , and These are learnable parameters; Step 4: Based on the key factors and the fused data, construct an urban energy consumption prediction model and use a multidimensional spatiotemporal sequence analysis method to predict the energy consumption trend in different regions. The method for constructing a city energy consumption prediction model is based on a spatiotemporal attention neural network. The urban area is divided into grids, with the granularity of the grid division determined according to the city size and energy consumption distribution characteristics. Each grid cell is represented as... ,in , Use grid coordinates; construct a spatiotemporal feature tensor It contains grid cells. In time Energy consumption data and environmental parameters; Define the spatiotemporal attention weight matrix The calculation formula is: ;in, and For learnable network parameters, This represents the size of the historical time window, with a value of 24 hours. Prediction results based on attention mechanism The calculation formula is: ;in, express Attention weight at any moment and These are model parameters; extract Energy consumption data in the grid is used as the basis for each grid cell. Predicted energy consumption at any given time; Step 5: Generate an urban energy consumption analysis report, including an energy utilization efficiency assessment and identification of areas with abnormal energy consumption, and issue early warnings for abnormal energy consumption situations.

2. The urban energy consumption analysis method integrating the Internet of Things and big data according to claim 1, characterized in that, The IoT sensing layer includes a regional concentrator and terminal sensing devices. The regional concentrator collects data from the terminal devices via the MQTT protocol. The terminal sensing devices include intelligent monitoring terminals for power consumption, gas consumption, heat consumption, and water consumption, as well as environmental sensors for temperature, humidity, wind speed, and air quality. The data collection frequency is set as follows: electricity data every 5 minutes, gas data every 30 minutes, heat and water resource data every hour, and environmental parameters every 10 minutes; the regional concentrator calculates the 5-minute average, maximum, and minimum values ​​through edge computing.

3. The urban energy consumption analysis method integrating the Internet of Things and big data according to claim 2, characterized in that, Methods for preprocessing the collected data include: Using the sliding window method to detect outliers, for time points Data In a window size of Calculate the average value within the time window. and standard deviation : ; ;like Then The value was identified as an outlier, among which The preset threshold is set between 2.5 and 3.5, depending on the data type. For detected outliers and missing values ​​in the dataset, a combination of linear interpolation and exponentially weighted moving average is used for imputation. The imputation formula is as follows: ; The value to fill in, These are linear interpolation weights, with values ​​ranging from [0.4, 0.6]. This is the historical data attenuation factor, with a value range of [0.7, 0.9]. The historical time window length is taken as 24 hours.

4. The urban energy consumption analysis method integrating the Internet of Things and big data according to claim 3, characterized in that, The training method for the spatiotemporal attention neural network includes: Constructing the loss function Simultaneously considering the constraints of prediction accuracy and spatiotemporal smoothness: ,in, The mean squared error between the predicted and actual values. For spatial smoothness regularization, For time smoothness regularization, , and These are weighting coefficients, with values ​​of 0.7, 0.15, and 0.15 respectively. The formula for calculating the spatial smoothness regularization term is: ; in, Represents grid cells The neighborhood group, Spatial correlation weights are calculated based on the distance between grid cells. ,in, The distance between two grid cells. This is the distance attenuation parameter, and its value is the side length of the grid cell; The formula for calculating the time smoothness regularization term is: ; The Adam optimization algorithm is used to minimize the loss function L to obtain the optimal parameters of the model; the initial learning rate is set to 0.001, and it decays to 0.8 times the original value every 30 training cycles.

5. The urban energy consumption analysis method integrating the Internet of Things and big data according to claim 4, characterized in that, In step five, the method for generating the urban energy consumption analysis report includes: Calculate the energy efficiency index of each region The formula is: ;in, For grid Economic output, For the corresponding energy consumption, This is the normalization coefficient.

6. The urban energy consumption analysis method integrating the Internet of Things and big data according to claim 5, characterized in that, Identify areas with abnormal energy consumption, if the area Energy consumption growth rate satisfy: If it is an abnormal energy consumption area, then it is marked as such. and These represent the mean and standard deviation of the city's energy consumption growth rate, respectively.

7. The urban energy consumption analysis method integrating the Internet of Things and big data according to claim 6, characterized in that, Calculate the seasonal fluctuation range of energy consumption in different urban areas based on historical data. The calculation formula is: ; ;in, and Other regions The mean and standard deviation of energy consumption for the same historical period. This is the early warning coefficient; Energy Consumption Anomaly Index (EAI): ; in, For the region Current energy consumption.

8. The urban energy consumption analysis method integrating the Internet of Things and big data according to claim 7, characterized in that, Set a three-level early warning threshold. The values ​​are 10%, 20%, and 30% respectively. A red alert was issued at that time. An orange alert was issued at that time. A yellow alert was issued at that time.

Images