Development data total element fusion dynamic monitoring system

Abstract

The invention discloses a development data total factor fusion dynamic monitoring system, which is characterized in that power consumption data is acquired through a power data acquisition device, and the data is processed and analyzed by combining power historical data and utilizing various analysis methods and algorithms (such as clustering analysis, time sequence analysis, statistical test and the like); and the enterprise power consumption data and the household power consumption data are distinguished. The system comprises the steps of data acquisition, time interval analysis, clustering analysis, workday and weekend electricity consumption comparative analysis, monitoring period judgment, model establishment and verification and the like, total element fusion dynamic monitoring of electric power data can be realized, and the efficiency and accuracy of electric power data processing are improved. Data cleaning, comparison, visualization and modeling analysis are performed through Pandas, NumPy, statsmodels and other libraries of Python, and finally a model is established to directly classify power data, so that powerful support is provided for power management and decision making.

Description

Technical Field

[0001] The present invention relates to the technical field of data fusion and classification, and in particular to the development of a dynamic monitoring system for data full-factor fusion. Background Art

[0002] As urbanization accelerates, complex buildings, as a new type of urban building form that integrates multiple functions such as business, office, and residence, are increasingly becoming an important part of urban development. These complexes often have a mixture of enterprises and households, forming a complex and changeable electricity consumption environment. However, in the field of power management and data analysis, how to accurately distinguish between corporate electricity consumption data and household electricity consumption data within the complex has become a technical problem that needs to be solved urgently.

[0003] At present, the classification of electricity consumption data mainly relies on manual investigation and registration. That is, relevant personnel are required to distinguish the enterprises and residents in the complex by themselves and manually record their electricity consumption information. However, this method has encountered many difficulties in actual operation. First, the mobility of enterprises and residents within the complex is relatively large, new enterprises and residents are constantly moving in, and the original enterprises and residents may move out at any time, which makes the real-time update of individual household information extremely difficult. Secondly, manual investigation and registration are not only time-consuming and labor-intensive, but also prone to errors, resulting in the inability to guarantee the accuracy and reliability of electricity consumption data.

[0004] To this end, we propose to develop a dynamic monitoring system that integrates all elements of data Summary of the invention

[0005] The purpose of the present invention is to provide a dynamic monitoring system for the integration of all elements of development data to solve the above-mentioned deficiencies in the prior art.

[0006] In order to achieve the above object, the present invention provides the following technical solution: developing a data full-factor fusion dynamic monitoring system, comprising the following steps:

[0007] Step 1: The data to be fused is collected through the power data collection device, including power consumption data, and the power history data of the local power system is collected through the interface, and then the average daily power consumption data of local household users is calculated based on the historical data;

[0008] Step 2: According to the timestamp of the data collected by the power data collection device, the collected data is divided into daytime data and nighttime data, wherein the daytime is set as interval A and the nighttime is set as interval B. The sum of the power consumption data of interval A and interval B collected by each power data collection device is calculated, and then the power consumption data threshold is set according to the average value of the daily power consumption data of local household users, and then it is found whether the total power consumption data of interval A and interval B is greater than the power consumption data threshold. If so, it is preliminarily determined that the data is enterprise power consumption data, and if not, it is preliminarily determined that the data is household power consumption data;

[0009] Step 3: Use cluster analysis to find the demarcation value between interval A and interval B, and use the difference in peak electricity consumption time between enterprises and households (the electricity consumption of enterprises in interval A is often greater than that of households in interval A, while the electricity consumption of households in interval B is greater than that of interval A), and mark the data in interval A that is greater than the demarcation value as enterprise electricity consumption data, and mark the data in interval A that is less than the demarcation value as household electricity consumption data;

[0010] Step 4: Compare and analyze the power consumption on weekdays and weekends for the data collected by each power data collection device according to the timestamp. If the power consumption trend on weekdays is stable, but the power consumption on weekends is significantly reduced, and this pattern remains stable for a period of time, it is judged as corporate power consumption data. On the contrary, if there is no difference in power consumption between weekdays and weekends, or the power consumption on weekends is greater than that on weekdays (affected by increased family activities), it is judged as household power consumption data.

[0011] Step 5: Set 7 days a week as a monitoring cycle. In each monitoring cycle, if there are five or more days with the same judgment results in steps 2, 3, and 4, the marking classification is accurate, and the accurately classified data is divided into the establishment data set and the verification data set. Otherwise, the marking classification is abnormal and enters the anomaly detection stage;

[0012] Step 6: Build a model based on the established data set, and verify the established model through the verification data set. Then, when the fused data is subsequently collected and monitored, the power data is directly classified by using the established model.

[0013] As a further description of the above technical solution: In the step 1, the data collected by the power data acquisition device is cleaned using Python's Pandas library, including removing duplicate values, filling missing values ​​by interpolation, smoothing to eliminate noise, and removing outliers.

[0014] As a further description of the above technical solution: In the step 2, the NumPy library is used to compare the sum of the power consumption data of interval A and interval B with the set power consumption data threshold, and at the same time, a chart and a bar chart are generated by the data visualization software (Matplotlib.

[0015] As a further description of the above technical solution: the specific operation of performing comparative analysis of power consumption on weekdays and weekends in step 4 is divided into the following steps:

[0016] A: Use Python's time series analysis library (statsmodels) to model and analyze weekday and weekend electricity consumption data to identify its changing trends and periodic patterns;

[0017] B: Use statistical test methods, including but not limited to T-test and ANOVA analysis of variance, to compare whether there is a significant difference in electricity consumption data between weekdays and weekends. If the test results show that there is a significant difference in electricity consumption between weekdays and weekends (such as P value is less than the significance level of 0.05), and it meets the characteristics of corporate electricity consumption data (stable electricity consumption on weekdays and reduced electricity consumption on weekends), then it is judged as corporate electricity consumption data; otherwise, it is judged as household electricity consumption data.

[0018] In the above technical solution, the development data full-factor fusion dynamic monitoring system provided by the present invention has the following beneficial effects:

[0019] The system does not require manual door-to-door surveys and classifications. Instead, it collects and analyzes electricity data to automatically identify and classify corporate and household electricity consumption data, greatly improving classification efficiency and accuracy. The system also uses the differences and patterns between businesses and households in terms of electricity consumption and electricity consumption time, and through methods such as cluster analysis and time series analysis, it achieves accurate classification of electricity consumption data, providing strong support for electricity management and decision-making. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below.

[0021] The embodiment of the present invention provides a technical solution: developing a data full-factor fusion dynamic monitoring system, characterized by comprising the following steps:

[0022] Step 1: The data to be fused is collected through the power data collection device, including power consumption data, and the power history data of the local power system is collected through the interface, and then the average daily power consumption data of local household users is calculated based on the historical data;

[0023] In addition to traditional power data collection devices, such as smart meters, advanced technologies such as IoT sensors and remote monitoring equipment can also be considered for data collection. When collecting data, the data interface with the local power system should be standardized so that historical power data can be obtained. This includes standardization of data format, transmission protocol, interface address, etc. to ensure data accuracy and consistency.

[0024] In addition, in order to calculate the average value of daily electricity consumption data of local household users, the influence of seasonal and cyclical factors should be considered. For example, in summer, electricity consumption may increase due to the use of cooling equipment such as air conditioners; while in winter, electricity consumption may increase due to the use of heating equipment; therefore, in calculating the average value, seasonal adjustment or cyclical analysis can be used to eliminate the influence of these factors;

[0025] Step 2: According to the timestamp of the data collected by the power data collection device, the collected data is divided into daytime data and nighttime data, where the daytime is set as interval A and the nighttime is set as interval B. The sum of the power consumption data of interval A and interval B collected by each power data collection device is calculated, and then the power consumption data threshold is set according to the average value of the daily power consumption data of local household users, and then the total power consumption data of interval A and interval B is found to be greater than the power consumption data threshold. If so, it is preliminarily determined that the data is enterprise power consumption data, if not, it is preliminarily determined that the data is household power consumption data, wherein the NumPy library can be used to compare the sum of the power consumption data of interval A and interval B with the set power consumption data threshold, and at the same time, the data visualization software (Matplotlib) is used to generate charts and bar charts;

[0026] It should be noted that since electricity consumption data usually has a certain volatility. Therefore, when calculating the sum, you can consider using methods such as sliding windows and weighted averages to smooth the data and reduce the impact of volatility. When setting the threshold for electricity consumption data, the threshold setting should also be dynamically adjusted according to actual conditions. For example, the threshold can be adjusted regularly based on factors such as the trend of historical data changes and seasonal changes. For example, in spring and autumn, there is no need to turn on the air conditioner, and the electricity consumption is low. The threshold can be set to 10% to 20% of the average electricity consumption. In winter and summer, the air conditioner is turned on more often and the electricity consumption is high, so the threshold can be set to 5% of the average electricity consumption.

[0027] Step 3: Use cluster analysis to find the demarcation value between interval A and interval B, and use the difference in peak electricity consumption time between enterprises and households (the electricity consumption of enterprises in interval A is often greater than that of households in interval A, while the electricity consumption of households in interval B is greater than that of interval A), and mark the data in interval A that is greater than the demarcation value as enterprise electricity consumption data, and mark the data in interval A that is less than the demarcation value as household electricity consumption data;

[0028] In addition to using cluster analysis, you can also draw a box plot to find the quartiles of the data (minimum, first quartile, median, third quartile and maximum value), as well as the outliers of the data. In the development of the dynamic monitoring system of all-factor fusion of data, you can consider the third quartile (or a slightly higher value) as the dividing value, because usually the electricity consumption of enterprises is higher than that of households, so the third quartile may be a reasonable dividing point;

[0029] Step 4: Compare and analyze the power consumption on weekdays and weekends for the data collected by each power data collection device according to the timestamp. If the power consumption trend on weekdays is stable, but the power consumption on weekends is significantly reduced, and this pattern remains stable for a period of time, it is judged as corporate power consumption data. On the contrary, if there is no difference in power consumption on weekdays and weekends, or the power consumption on weekends is greater than that on weekdays (affected by increased family activities), it is judged as household power consumption data.

[0030] In order to further improve the observation effect of power consumption fluctuations on weekdays and weekends, time series analysis methods can be used to study the changing trends of power consumption on weekdays and weekends. By drawing time series graphs and calculating moving averages and other indicators, we can more intuitively understand the fluctuations in power consumption, and use statistical test methods (such as t-tests, variance analysis, etc.) to compare whether the difference in power consumption on weekdays and weekends is significant. This helps to quantify the degree of change in power consumption and improve the accuracy of judgment;

[0031] In order to further improve the judgment accuracy of this step, not only can we compare the electricity consumption on weekends and weekdays, but we can also analyze the daily electricity consumption fluctuations, for example, whether the electricity consumption data collected by each power data collection device has periodic fluctuations, such as daily, weekly or monthly fluctuation patterns. Enterprise electricity consumption may be affected by factors such as production plans and shift arrangements, showing relatively stable periodic fluctuations; while household electricity consumption may be affected by factors such as the periodic operation of household appliances and the activity habits of family members, showing more random periodic fluctuations. For example, in household electricity consumption, there may be periodic fluctuations in electricity consumption during the day, which may be caused by the intermittent start-up of the refrigerator, or there may be a sudden decrease in electricity consumption, which may continue until the user gets off work at night. This may be caused by electrical appliances, such as washing machines or sweeping robots, being powered off after work. According to these situations, the accuracy of data classification can be significantly improved;

[0032] Step 5: Set 7 days a week as a monitoring cycle. In each monitoring cycle, if there are five or more days with the same judgment results in steps 2, 3, and 4, the marking classification is accurate, and the accurately classified data is divided into the establishment data set and the verification data set. Otherwise, the marking classification is abnormal and enters the anomaly detection stage;

[0033] Among them, in step two, step three and step four, multiple judgment indicators or methods are involved. In order to obtain more accurate classification results, these judgment results can be integrated. For example, weighted average can be used to integrate multiple judgment results. When integrating the judgment results, weights can be assigned according to the importance or accuracy of each step. For example, if the judgment result of a certain step is more reliable, it can be given a higher weight. For example, the electricity consumption of enterprises will be greater than that of households. The accuracy of this judgment result is greater, so it can be given a larger weight. The electricity consumption pattern will be affected by more factors, so it can be given a smaller weight.

[0034] Step 6: Build a model based on the established data set, and verify the established model through the verification data set. Then, when collecting and monitoring the fused data in the future, use the established model to directly classify the power data. It should be noted that when building the model, you can use Scikit-learn's RandomForestClassifier to build a random forest model, or use SVC to build a support vector machine model. When training the model, you can also use the fit method to train the model, and use the predict method to classify new data. At the same time, cross-validation is used to evaluate the performance of the model. After the model is established and verified, it is applied to the subsequent collection and monitoring of fused data. By using the established model, the power data can be directly classified, thereby realizing the rapid identification of different user types. This helps to improve the efficiency and accuracy of power data processing and provide strong support for power management and decision-making.

[0035] In one embodiment of the present invention, in step 1, the data collected by the power data acquisition device is cleaned using the Python Pandas library, including removing duplicate values, filling missing values ​​by interpolation, smoothing to eliminate noise, and removing outliers;

[0036] It should be noted that the specific operation of the comparative analysis of power consumption on weekdays and weekends in step 4 is divided into the following steps:

[0037] A: Use Python's time series analysis library (statsmodels) to model and analyze weekday and weekend electricity consumption data to identify its changing trends and periodic patterns;

[0038] B: Use statistical test methods, including but not limited to T-test and ANOVA analysis of variance, to compare whether there is a significant difference in electricity consumption data between weekdays and weekends. If the test results show that there is a significant difference in electricity consumption between weekdays and weekends (such as P value is less than the significance level of 0.05), and it meets the characteristics of corporate electricity consumption data (stable electricity consumption on weekdays and reduced electricity consumption on weekends), then it is judged as corporate electricity consumption data; otherwise, it is judged as household electricity consumption data;

[0039] Among them, the anomaly detection stage includes but is not limited to manual investigation and online investigation. There is no need to conduct door-to-door investigation. Instead, it is only necessary to conduct investigation when anomalies occur, which reduces the workload.

[0040] The above description is only by way of illustration of certain exemplary embodiments of the present invention. It is undoubted that, for those skilled in the art, the described embodiments can be modified in various ways without departing from the spirit and scope of the present invention. Therefore, the above description is illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. Develop a dynamic monitoring system that integrates all elements of data, characterized by: The steps include: Step 1: The data to be fused is collected through the power data collection device, including power consumption data, and the power history data of the local power system is collected through the interface, and then the average daily power consumption data of local household users is calculated based on the historical data; Step 2: According to the timestamp of the data collected by the power data collection device, the collected data is divided into daytime data and nighttime data, wherein the daytime is set as interval A and the nighttime is set as interval B. The sum of the power consumption data of interval A and interval B collected by each power data collection device is calculated, and then the power consumption data threshold is set according to the average value of the daily power consumption data of local household users, and then it is found whether the total power consumption data of interval A and interval B is greater than the power consumption data threshold. If so, it is preliminarily determined that the data is enterprise power consumption data, and if not, it is preliminarily determined that the data is household power consumption data; Step 3: Use cluster analysis to find the demarcation value between interval A and interval B, and use the difference in peak electricity consumption time between enterprises and households (the electricity consumption of enterprises in interval A is often greater than that of households in interval A, while the electricity consumption of households in interval B is greater than that of interval A), and mark the data in interval A that is greater than the demarcation value as enterprise electricity consumption data, and mark the data in interval A that is less than the demarcation value as household electricity consumption data; Step 4: Compare and analyze the power consumption on weekdays and weekends for the data collected by each power data collection device according to the timestamp. If the power consumption trend on weekdays is stable, but the power consumption on weekends is significantly reduced, and this pattern remains stable for a period of time, it is judged as corporate power consumption data. On the contrary, if there is no difference in power consumption between weekdays and weekends, or the power consumption on weekends is greater than that on weekdays (affected by increased family activities), it is judged as household power consumption data. Step 5: Set 7 days a week as a monitoring cycle. In each monitoring cycle, if there are five or more days with the same judgment results in steps 2, 3, and 4, the marking classification is accurate, and the accurately classified data is divided into the establishment data set and the verification data set. Otherwise, the marking classification is abnormal and enters the anomaly detection stage; Step 6: Build a model based on the established data set, and verify the established model through the verification data set. Then, when the fused data is subsequently collected and monitored, the power data is directly classified by using the established model.

2. The development data full-factor fusion dynamic monitoring system according to claim 1 is characterized in that: In the step 1, the data collected by the power data acquisition device is cleaned using the Python Pandas library, including removing duplicate values, filling missing values ​​by interpolation, smoothing to eliminate noise, and removing outliers.

3. The development data full-factor fusion dynamic monitoring system according to claim 1 is characterized in that: In the second step, the NumPy library is used to compare the sum of the power consumption data of interval A and interval B with the set power consumption data threshold, and at the same time, a chart and a bar chart are generated by the data visualization software (Matplotlib).

4. The development data full-factor fusion dynamic monitoring system according to claim 1 is characterized in that: The specific operation of the comparative analysis of power consumption on weekdays and weekends in step 4 is divided into the following steps: A: Use Python's time series analysis library (statsmodels) to model and analyze weekday and weekend electricity consumption data to identify its changing trends and periodic patterns; B: Use statistical test methods, including but not limited to T-test and ANOVA analysis of variance, to compare whether there is a significant difference in electricity consumption data between weekdays and weekends. If the test results show that there is a significant difference in electricity consumption between weekdays and weekends (such as P value is less than the significance level of 0.05), and it meets the characteristics of corporate electricity consumption data (stable electricity consumption on weekdays and reduced electricity consumption on weekends), then it is judged as corporate electricity consumption data; otherwise, it is judged as household electricity consumption data.

5. The development data full-factor fusion dynamic monitoring system according to claim 1 is characterized in that: The anomaly detection stage includes but is not limited to manual research and network research.