Energy data collection method

Abstract

The present invention discloses a smart energy data collection method. The method obtains a sample data set and processes the data using a clustering algorithm based on different product types, product manufacturer information, product IDs, and product application environments. A data collection prediction model is used to predict new data, and the prediction results are sorted according to the importance of the data items. A collection plan is determined and a meter reading execution script is generated. Based on the message rules of the meter reading execution script, a corresponding meter reading command is generated, the meter reading command is compressed, and the meter reading command is sent to a target device via an Internet of Things gateway and an adapter. The target device receives a data message return code, and the return code is parsed based on the message rules of the meter reading execution script in the data collection prediction model to generate meter reading data. Through script customization, the present invention provides different collection methods for different types of energy-consuming devices, enabling data collection from different manufacturers and devices, thereby better controlling and managing energy.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to an energy data acquisition method. Background Art

[0002] At present, various smart buildings such as industrial parks, commercial complexes, and hotels collect data from other energy-consuming equipment such as power distribution, air conditioning, lighting, water boilers, various instruments related to water, electricity, gas, and heat, and on-site temperature, humidity, pressure, and other sensors.

[0003] The meter reading technology on the market is generally divided into three categories according to water, electricity and fuel for reading data. This requires different equipment to collect data for different categories, which makes the cost too high. In addition, there are many manufacturers of various types and no unified standard, which causes certain difficulties in data collection. Summary of the Invention

[0004] The technical problem to be solved by the present invention is that none of the existing meter reading technologies can simultaneously collect data information on three categories of water, electricity and fuel, and there is no unified collection standard for different types of energy-consuming equipment from different manufacturers. The present invention provides an energy data collection method that, through script customization, provides different collection methods for different types of energy-consuming equipment, realizing data collection from different manufacturers and equipment, thereby better controlling and managing energy.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solution: an energy data collection method, the data collection method comprising the following steps:

[0006] S1. Obtain a sample data set and generate an energy data acquisition and measurement unit and a meter reading command; wherein the sample data set includes product type, product manufacturer information, product ID, and product application environment;

[0007] S2. Import the meter reading command into the source data acquisition and measurement unit to generate a meter reading execution script;

[0008] S3. Generate corresponding meter reading data message according to the meter reading message rules in the meter reading execution script and send it to the meter reading device;

[0009] S4. Receive the data message return code from the meter reading device, parse the return code according to the meter reading script template in the energy data acquisition and measurement unit, and generate meter reading data.

[0010] According to one aspect of the present application, in step S1, generating an energy data acquisition and measurement unit includes the following sub-steps:

[0011] S11, dividing the required collected data items according to the acquired sample data set, and establishing corresponding atomic tags for the required collected data items;

[0012] S12: Based on the product type and collection period, the corresponding atomic tag is retrieved to generate a meter reading script template.

[0013] According to one aspect of the present application, the process of generating the energy data acquisition and measurement unit is further as follows:

[0014] Step S1a: Use a clustering algorithm to group the sample data set, and classify similar data into one category based on different product types, manufacturer information, product IDs, and application environments;

[0015] Step S1b: For each type of data, use a feature selection algorithm to select the most representative and discriminative data items as atomic labels;

[0016] Step S1c: For each atomic tag, use an encoding algorithm to convert it into binary or other formats to facilitate subsequent data transmission and processing;

[0017] Step S1d: Based on the product type and collection period, the corresponding atomic tag is retrieved to generate a meter reading script template; the collection script template includes the collection object device, collection protocol, communication method, collection period, collection data items, and reading message rules for the collection data items; wherein the collection data items are represented by atomic tags.

[0018] According to one aspect of the present application, step S1 further includes:

[0019] Step S13: Use a graph model-based method to train a sample data set to generate a data acquisition prediction model; use the prediction model to automatically determine an acquisition plan for new data;

[0020] Step S13a: Obtain a sample data set from the database, including energy consumption data of different regions and time periods; divide the sample data set into a training set and a test set, and perform normalization processing;

[0021] Step S13b, using ST-ResNet to build a deep residual network model, using data from different time periods as input to different layers, and using graph convolution operations to model spatial relationships; the deep residual network model includes an input layer, a residual layer, and an output layer; defining a loss function and an optimizer, and using a training set to train the deep residual network model; using a test set to evaluate the performance and accuracy of the deep residual network model;

[0022] Step S13c: Acquire new data from the database, including energy consumption data for different regions and time periods; normalize the new data and use it as input to the deep residual network model; use the deep residual network model to predict the new data and obtain energy consumption trends over a period of time; determine the collection plan based on the prediction results and user needs, including the collection object, collection period, and collection data items;

[0023] Step S14: Generate corresponding meter reading commands according to the acquisition plan and meter reading script rules.

[0024] According to one aspect of the present application, step S2 further includes:

[0025] Step S21: Obtain meter reading commands and sort the data items by importance according to user needs;

[0026] Step S22: Generate a dynamic meter reading command according to the data importance ranking; and generate a meter reading execution script based on the homomorphic meter reading command.

[0027] According to one aspect of the present application, step S3 is further:

[0028] Step S31: Generate a corresponding meter reading data message according to the meter reading message rules in the meter reading execution script;

[0029] Step S32: compress the meter reading command using a data compression algorithm, obtain the compressed meter reading command, and send it.

[0030] According to one aspect of the present application, step S31 further includes:

[0031] S31a, converting the meter reading command into a numerical vector as input to the deep learning model;

[0032] S31b. Build a deep learning model based on VAE or GAN, and train the model according to different objective functions and optimization methods;

[0033] S31c, compressing the input vector using the trained deep learning model to obtain a compressed vector as the output of the deep learning model;

[0034] S31d, selecting an asymmetric encryption algorithm and generating an encryption function based on different keys and encryption methods;

[0035] S31e, encrypting the compressed vector using an encryption function to obtain an encrypted vector as output;

[0036] S31f, select Hamming code or Reed-Solomon code error correction code algorithm, and generate a coding function according to different coding rates and error correction capabilities;

[0037] S31g, using an encoding function to encode the encrypted vector to obtain an encoded vector, and adding a check bit and a parity bit as output, that is, obtaining a compressed and encrypted data message;

[0038] The meter reading data is encoded using an autoencoder model to obtain a compressed feature representation; the compressed encoding vector is transmitted; and the receiving end uses an autoencoder to decode the encoding and restore the data.

[0039] According to one aspect of the present application, step S31 further includes:

[0040] Convert the meter reading command into a numerical vector as the input of the converter model;

[0041] Build a ConvTransformer transformer model and train the model according to different objective functions and optimization methods;

[0042] Use the trained model to compress the input vector to obtain the compressed vector, which is used as the compressed data packet as output.

[0043] According to one aspect of the present application, step S4 is further:

[0044] Step S41: After receiving and decompressing the meter reading command, the meter reading device verifies the collected data;

[0045] Step S42: If the verification fails, resend the meter reading command to recollect data; if the verification succeeds, convert the collected data into a standardized communication protocol and data format on the edge node and send it to the central server; the central server receives the collected data and performs training, and uses the training results to drive the subsequent data collection process.

[0046] According to one aspect of the present application, in step S42, the central server receives the collected data and performs training, and the process of using the training results to drive the subsequent data collection process is further as follows:

[0047] Step S42a: Acquire the collected data and clean and format it; and convert the collected data into a window;

[0048] Step S42b: construct and use an LSTM-ARIMA model, input historical window data, predict energy consumption in the future, adjust the model structure and parameters, train the model, and obtain prediction results;

[0049] According to the prediction results, if the predicted energy consumption is greater than the threshold, the collection frequency is increased; if the predicted energy consumption is less than the threshold, the collection frequency is reduced; the collection frequency and accuracy are adjusted in real time;

[0050] Collect actual data and feed it back to the prediction model for further training; when the prediction error is too large, retrain the model; continuously optimize the prediction model;

[0051] The steps for building the LSTM-ARIMA combined model include:

[0052] Differentiate the original time series data to make it stable; normalize the processed data to the range of [0,1] as the input data of the LSTM model;

[0053] Use ACF and PACF plots to determine the ARIMA parameters (p, d, q), train the ARIMA model using the training dataset, and use the ARIMA model to predict future data points.

[0054] Subtract the ARIMA prediction from the actual time series data to obtain the residuals, and use the residual data to train the LSTM model. At this time, the goal of the LSTM is to learn and predict the patterns that the ARIMA model fails to capture, and use the LSTM model to predict future residuals;

[0055] For a future data point, the ARIMA prediction value is added to the residual of the LSTM prediction to obtain the final prediction value and use it as the prediction result.

[0056] The energy data collection method proposed in this invention uses scripts to define the methods for collecting data items and extracting data, automatically adapting to multiple products, manufacturers, and application scenarios. Users can also define and modify scripts according to their needs, easily implementing the reading of various data items. The related technical advantages will be described below in conjunction with specific examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 Schematic diagram of the flow of the energy data collection method of the present invention.

[0058] Figure 2 Schematic diagram of the meter reading script of the energy data collection method of the present invention.

[0059] Figure 3 This is the application scenario intention of the energy data collection method of the present invention.

[0060] Figure 4 Schematic diagram of data items of the energy data collection method of the present invention. DETAILED DESCRIPTION

[0061] The following examples may enable those skilled in the art to more fully understand the present invention, but are not intended to limit the present invention in any way.

[0062] A method for collecting energy data, comprising the following steps:

[0063] S1. Obtain a sample data set and generate an energy data acquisition and measurement unit and a meter reading command; wherein the sample data set includes product type, product manufacturer information, product ID, and product application environment;

[0064] S2. Import the meter reading command into the source data acquisition and measurement unit to generate a meter reading execution script;

[0065] S3. Generate corresponding meter reading data message according to the meter reading message rules in the meter reading execution script and send it to the meter reading device;

[0066] S4. Receive the data message return code from the meter reading device, parse the return code according to the meter reading script template in the energy data acquisition and measurement unit, and generate meter reading data.

[0067] Because different energy data collection targets may have different communication protocols, data formats, and parameter types, energy data collection is diverse and complex. Using a fixed data collection solution may not adapt to different energy data collection targets or require frequent modification and adjustment. This solution dynamically generates data collection solutions based on different energy data collection targets by acquiring a sample data set and generating energy data collection measurement units and meter reading commands. This improves the flexibility and versatility of energy data collection. Because energy data collection involves extensive communication and computation, using inefficient or incorrect data collection solutions can reduce the efficiency and accuracy of energy data collection. For example, using inappropriate reading message rules or return code parsing rules can lead to communication failures or data errors. This solution imports meter reading commands into source data collection measurement units and generates meter reading execution scripts. This solution generates appropriate reading message rules and return code parsing rules based on different meter reading commands, thereby improving the efficiency and accuracy of energy data collection. Because energy data involves the operation and management of the power system, using unsecure or unreliable data collection solutions can lead to energy data tampering or loss. For example, using unencrypted or unverified communication or storage methods can lead to energy data being stolen or damaged. This solution generates corresponding execution reading data messages based on the reading message rules in the meter reading execution script, sends them to the reading device, receives the data message return code from the reading device, and parses the return code according to the meter reading script template in the energy data acquisition and measurement unit to generate meter reading data. This can implement operations such as encryption, verification, and decryption of energy data, thereby improving the security and reliability of energy data collection.

[0068] According to one aspect of the present application, in step S1, generating an energy data acquisition and measurement unit includes the following sub-steps:

[0069] S11, dividing the required collected data items according to the acquired sample data set, and establishing corresponding atomic tags for the required collected data items;

[0070] S12: Based on the product type and collection period, the corresponding atomic tag is retrieved to generate a meter reading script template.

[0071] According to one aspect of the present application, the process of generating the energy data acquisition and measurement unit is further as follows:

[0072] Step S1a: Use a clustering algorithm to group the sample data set, and classify similar data into one category based on different product types, manufacturer information, product IDs, and application environments;

[0073] Step S1b: For each type of data, use a feature selection algorithm to select the most representative and discriminative data items as atomic labels;

[0074] Step S1c: For each atomic tag, use an encoding algorithm to convert it into binary or other formats to facilitate subsequent data transmission and processing;

[0075] Step S1d: Based on the product type and collection period, the corresponding atomic tag is retrieved to generate a meter reading script template; the collection script template includes the collection object device, collection protocol, communication method, collection period, collection data items, and reading message rules for the collection data items; wherein the collection data items are represented by atomic tags.

[0076] Because different sample datasets can contain a large number of data items, manually or using fixed rules to generate energy data collection and measurement units and meter reading script templates can be time-consuming and error-prone. This solution uses clustering and feature selection algorithms to quickly group and filter sample datasets, and uses an encoding algorithm to convert data items into atomic labels, thereby achieving automated and intelligent generation of energy data collection and measurement units and meter reading script templates. Because different sample datasets may contain some noisy or redundant data items, using manual or fixed rules to generate energy data collection and measurement units and meter reading script templates can result in inconsistent results or contain errors. This solution uses clustering and feature selection algorithms to effectively remove these noisy and redundant data items, and uses an encoding algorithm to convert data items into atomic labels, thereby achieving precise and optimized generation of energy data collection and measurement units and meter reading script templates. Because different sample datasets may vary in factors such as product type, manufacturer information, product ID, and application environment, using manual or fixed rules to generate energy data collection and measurement units and meter reading script templates can be unsuitable for different situations or require frequent modification and adjustment. This solution retrieves the corresponding atomic tags based on product type and collection cycle, and can dynamically generate corresponding energy data collection and measurement units and meter reading script templates according to different situations, thereby realizing the flexible and universal generation of energy data collection and measurement units and meter reading script templates.

[0077] According to one aspect of the present application, step S1 further includes step S13, using a graph model-based method to train a sample data set to generate a data acquisition prediction model; using the prediction model to automatically determine an acquisition plan for new data;

[0078] Step S13a: Obtain a sample data set from the database, including energy consumption data of different regions and time periods; divide the sample data set into a training set and a test set, and perform normalization processing;

[0079] Step S13b, using ST-ResNet to build a deep residual network model, using data from different time periods as input to different layers, and using graph convolution operations to model spatial relationships; the deep residual network model includes an input layer, a residual layer, and an output layer; defining a loss function and an optimizer, and using a training set to train the deep residual network model; using a test set to evaluate the performance and accuracy of the deep residual network model;

[0080] Step S13c: Acquire new data from the database, including energy consumption data for different regions and time periods; normalize the new data and use it as input to the deep residual network model; use the deep residual network model to predict the new data and obtain energy consumption trends over a period of time; determine the collection plan based on the prediction results and user needs, including the collection object, collection period, and collection data items;

[0081] Step S14: Generate corresponding meter reading commands according to the acquisition plan and meter reading script rules.

[0082] Because energy data has complex spatiotemporal characteristics and correlations, using traditional machine learning or statistical learning methods to build prediction models may not fully capture the inherent patterns and trends of the data. This solution uses ST-ResNet to construct a deep residual network model. This effectively leverages the residual structure and graph convolution operations to model temporal proximity, periodicity, and trends, and spatial proximity, correlation, and similarity, respectively. These features are then dynamically integrated, enabling efficient and accurate learning of energy data collection prediction models. Because energy data is influenced by multiple factors, such as weather, holidays, and events, using overly simple or overly complex prediction models can lead to overfitting or underfitting, reducing the model's performance on new data. This solution uses ST-ResNet to construct a deep residual network model. This effectively leverages the residual structure and graph convolution operations to enhance the model's expressiveness and generalization capabilities. It also incorporates external factors, such as weather and the day of the week, to improve the model's adaptability and robustness, thereby optimizing and improving the energy data collection prediction model. Because energy data collection plans involve selecting multiple parameters, such as the collection object, collection period, and data items, manually or using fixed rules to generate collection plans can be time-consuming and labor-intensive, prone to errors, and inconsistent with user needs. This solution uses a deep residual network model to predict new data. Based on the prediction results and user needs, it automatically determines the collection plan and generates the corresponding meter reading commands based on the meter reading script rules, thus achieving automated and intelligent generation of energy data collection plans.

[0083] According to one aspect of the present application, step S2 further includes:

[0084] Step S21: Obtain meter reading commands and sort the data items by importance according to user needs;

[0085] Step S22: Generate a dynamic meter reading command according to the data importance ranking, and generate a meter reading execution script based on the homomorphic meter reading command.

[0086] Data items in meter reading commands are ranked by importance based on the needs of different users in different scenarios, optimizing energy data collection for each scenario. Different users may have varying levels of interest and needs for different energy data. For example, some users may be more interested in real-time parameters like voltage and current, while others may be more concerned with cumulative parameters like power and energy. Using a fixed or random data item ranking method to generate meter reading commands can result in the delay or omission of important or urgent data items, impacting the effectiveness and quality of energy data collection.

[0087] Specific methods include allowing users to select parameters to read based on their needs and preferences and specify their priorities. For example, a user can choose to read only voltage and current, prioritizing voltage over current. Furthermore, the system can recommend an appropriate data item sorting method based on the user's historical data and behavior, as well as feedback and evaluations from other users. For example, the system can recommend an energy-saving and environmentally friendly data item sorting method based on the user's electricity usage and usage time, such as prioritizing power and energy, and provide corresponding energy-saving recommendations. Finally, the system can automatically learn and adjust the data item sorting method based on the user's real-time data and feedback, as well as the data and feedback from other users, using machine learning or deep learning methods. For example, the system can automatically adjust the data item sorting method based on abnormal power usage or fault conditions. Entropy-based methods can also be used to calculate the information content of each data item and then rank the data items according to their importance. A greater amount of information indicates a more important data item, and vice versa.

[0088] According to one aspect of the present application, step S3 is further:

[0089] Step S31: Generate a corresponding meter reading data message according to the meter reading message rules in the meter reading execution script;

[0090] Step S32: compress the meter reading command using a data compression algorithm, obtain the compressed meter reading command, and send it.

[0091] Because different meter reading execution scripts may have different reading message rules, using a fixed message format or manually crafted messages may result in non-compliant or erroneous messages. This solution automatically generates corresponding execution reading data messages based on the reading message rules in the meter reading execution script, ensuring message accuracy and consistency. Since execution reading data messages may contain large amounts of data, using raw data formats or inefficient communication methods can increase data transmission time and costs. This solution compresses meter reading commands using a data compression algorithm, generates the compressed command, and sends it, reducing data transmission size and time, thereby saving communication resources and costs. Because different meter reading devices may have different communication protocols and data formats, using a unified execution reading data message may result in some devices being unable to recognize or parse it. This solution generates corresponding execution reading data messages based on the reading message rules in the meter reading execution script and encrypts them using a homomorphic encryption algorithm, achieving adaptability and confidentiality for different devices.

[0092] According to one aspect of the present application, step S31 further includes:

[0093] S31a. Convert meter reading commands (e.g., reading the electricity meter's power level) into a format that can be understood and processed by the deep learning model: a numerical vector. A numerical vector is a one-dimensional or multi-dimensional array of numbers that can represent different features or attributes. For example, a meter reading command can be represented by an 8-bit binary number, such as 00000001 for reading the electricity meter, 00000010 for reading the water meter, and so on.

[0094] S31b. Build a deep learning model based on a VAE or GAN and train the model according to different objective functions and optimization methods. Use these two deep learning models to compress and generate meter reading commands. A VAE, or variational autoencoder, compresses high-dimensional data (such as images) into low-dimensional latent variables (such as numerical vectors) and reconstructs the original data from these latent variables. A GAN, or generative adversarial network, generates real-world data (such as images) from random noise. Both models require parameter training based on different objective functions (such as minimizing reconstruction error or maximizing generation quality) and optimization methods (such as gradient descent or stochastic gradient descent).

[0095] S31c. Use the trained deep learning model to compress the input vector, obtaining a compressed vector as the output of the deep learning model. Use the trained VAE or GAN model to compress the input vector (i.e., meter reading command) to obtain a shorter or more sparse vector as the output of the deep learning model. This can reduce the bandwidth or space required to transmit or store data, improving efficiency and saving resources.

[0096] S31d. Select an asymmetric encryption algorithm and generate an encryption function based on the key and encryption method. Use the asymmetric encryption algorithm to encrypt the compressed vector to ensure data security and reliability. Asymmetric encryption uses different public and private keys for encryption and decryption. The public key is publicly available, while the private key is kept secret by the holder. Data encrypted with the public key can only be decrypted with the private key, and vice versa. Common asymmetric encryption algorithms include RSA, DSA, and ECC. Different encryption functions can be generated based on the key and encryption method.

[0097] S31e. Encrypt the compressed vector using an encryption function to obtain an encrypted vector as output; then encrypt the compressed vector using the generated encryption function to obtain a vector that is more difficult to crack or tamper with as output. This prevents data from being stolen or modified by malicious attackers during transmission or storage.

[0098] S31f. Select either the Hamming code or the Reed-Solomon code error correction algorithm and generate a coding function based on the coding rate and error correction capability. Encode the encrypted vector using either the Hamming code or the Reed-Solomon code to increase the data's fault tolerance and robustness. Error correction algorithms add redundant information to data, allowing for error detection and correction. Examples of error correction algorithms include Hamming code and Reed-Solomon code. Different coding functions can be generated based on the coding rate (the ratio of redundant information to the total information) and error correction capability (the number of errors that can be corrected).

[0099] S31g. Encode the encrypted vector using an encoding function to obtain an encoded vector, and add a check bit and a parity bit as output, thereby obtaining a compressed and encrypted data message. Encode the encrypted vector using the generated encoding function to obtain a vector that is more resistant to noise or interference, and add a check bit and a parity bit as output, thereby obtaining a compressed and encrypted data message. Check bits and parity bits are used to detect whether data is correctly transmitted or stored. A check bit is one or more bits calculated based on certain bits in the data, and a parity bit is a bit determined by whether the number of 1s in the data is odd or even. This allows for timely detection and action to be taken when data errors occur.

[0100] The meter reading data is encoded using an autoencoder model to obtain a compressed feature representation; the compressed encoding vector is transmitted; and the receiving end uses an autoencoder to decode the encoding and restore the data.

[0101] According to one aspect of the present application, step S31 further includes:

[0102] Convert the meter reading command into a numerical vector as the input of the converter model;

[0103] Build a ConvTransformer transformer model and train the model according to different objective functions and optimization methods;

[0104] Use the trained model to compress the input vector to obtain the compressed vector, which is used as the compressed data packet as output.

[0105] Since meter reading commands may contain different data items and parameters, using fixed-length or sparse coding to represent them may result in information loss or redundancy. This solution can ensure the integrity and density of information by converting meter reading commands into numerical vectors. Since meter reading commands may contain a large amount of data, using the original data format or inefficient compression algorithm may increase the time and cost of data transmission. This solution can reduce the size and time of data transmission by compressing the input vector using the ConvTransformer transformer model, thereby saving communication resources and costs. Since meter reading commands may need to be dynamically generated according to different scenarios and requirements, if they are generated using manual writing or fixed rules, the message may not meet the requirements or errors may occur. This solution can adapt to and meet different scenarios and requirements by using the ConvTransformer transformer model to generate input vectors.

[0106] According to one aspect of the present application, step S4 is further:

[0107] Step S41: After receiving and decompressing the meter reading command, the meter reading device verifies the collected data;

[0108] Step S42: If the verification fails, resend the meter reading command to recollect data; if the verification succeeds, convert the collected data into a standardized communication protocol and data format on the edge node and send it to the central server; the central server receives the collected data and performs training, and uses the training results to drive the subsequent data collection process.

[0109] According to one aspect of the present application, in step S42, the central server receives the collected data and performs training, and the process of using the training results to drive the subsequent data collection process is further as follows:

[0110] Step S42a: Acquire the collected data and clean and format it; and convert the collected data into a window;

[0111] Step S42b: construct and use an LSTM-ARIMA model, input historical window data, predict energy consumption in the future, adjust the model structure and parameters, train the model, and obtain prediction results;

[0112] According to the prediction results, if the predicted energy consumption is greater than the threshold, the collection frequency is increased; if the predicted energy consumption is less than the threshold, the collection frequency is reduced; the collection frequency and accuracy are adjusted in real time;

[0113] Collect actual data and feed it back to the prediction model for further training; when the prediction error is too large, retrain the model; continuously optimize the prediction model;

[0114] The steps for building the LSTM-ARIMA combined model include:

[0115] Differentiate the original time series data to make it stable; normalize the processed data to the range of [0,1] as the input data of the LSTM model;

[0116] Use ACF and PACF plots to determine the ARIMA parameters (p, d, q), train the ARIMA model using the training dataset, and use the ARIMA model to predict future data points.

[0117] Subtract the ARIMA prediction from the actual time series data to obtain the residuals, and use the residual data to train the LSTM model. At this time, the goal of the LSTM is to learn and predict the patterns that the ARIMA model fails to capture, and use the LSTM model to predict future residuals;

[0118] For a future data point, the ARIMA prediction value is added to the residual of the LSTM prediction to obtain the final prediction value and use it as the prediction result.

[0119] Because energy consumption is a time series data with complex temporal and spatial dependencies, using a single statistical method or neural network for prediction can result in inaccurate or unstable predictions. This solution uses a combined LSTM-ARIMA model for prediction, leveraging the ARIMA model to capture trends and seasonality in the time series and the LSTM model to capture long-term dependencies and nonlinear relationships, thereby improving prediction accuracy and stability. Because energy consumption is a time series data with dynamic changes and uncertainty, using a fixed collection frequency and accuracy can result in excessive or insufficient data, wasting resources or losing information. By adjusting the collection frequency and accuracy based on prediction results, this solution enables dynamic and flexible control of collected data, conserving resources and ensuring information integrity. Because energy consumption is a time series data with constant changes and updates, using a fixed prediction model can result in inconsistent or biased predictions. This solution collects actual collected data and feeds it back into the prediction model for further training, enabling continuous and adaptive optimization of the prediction model and improving prediction effectiveness and quality.

[0120] Through this combination, the LSTM-ARIMA model can effectively capture both linear and nonlinear patterns in time series data, thereby improving forecast accuracy. However, building and tuning this combined model requires more time and computing resources, so in practical applications, a trade-off needs to be made based on specific circumstances.

[0121] The present invention may also adopt: a multivariate time series analysis method to model the collected data, such as using a vector autoregression (VAR) or a vector error correction model (VECM).

[0122] Use methods such as Kalman Filter or Particle Filter to perform state estimation and prediction on multivariate time series models.

[0123] Use methods such as Bayesian Optimization or Reinforcement Learning to automatically adjust and optimize model structure and parameters.

[0124] You can use an RNN-based approach to build and use an LSTM-ARIMA model. By inputting historical window data, you can predict energy consumption for the next 15 minutes, adjust the model structure and parameters, and train the model. Specifically, you can use PredRNN2 to build a recurrent neural network that takes historical window data as input and uses multiple layers of LSTM units and convolution operations to model spatiotemporal dependencies. This improves the long-term memory and generalization capabilities of the prediction model.

[0125] The preferred embodiments of the present invention are described in detail above. However, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all fall within the scope of protection of the present invention.

Claims

1. A method for collecting energy data, characterized in that: The data collection method comprises the following steps: S1. Obtain a sample data set and generate an energy data acquisition and measurement unit and a meter reading command; wherein the sample data set includes product type, product manufacturer information, product ID, and product application environment; S2. Import the meter reading command into the energy data acquisition and measurement unit to generate a meter reading execution script; S3. Generate corresponding meter reading data message according to the meter reading message rules in the meter reading execution script and send it to the meter reading device; S4. Receive the data message return code from the meter reading device, parse the return code according to the meter reading script template in the energy data acquisition and measurement unit, and generate meter reading data; Step S1 includes the following sub-steps: S11, dividing the required collected data items according to the acquired sample data set, and establishing corresponding atomic tags for the required collected data items; S12. Retrieve the corresponding atomic tag based on the product type and collection period, and generate a meter reading script template; S13. Use a graph-based model to train a sample dataset and generate a data collection prediction model; use the prediction model to automatically determine a collection plan for new data; S14. Generate corresponding meter reading commands according to the collection plan and meter reading script rules.

2. The energy data collection method according to claim 1, wherein: The process of generating an energy data acquisition measurement unit is further as follows: Step S1a: Use a clustering algorithm to group the sample data set, and classify similar data into one category based on different product types, manufacturer information, product IDs, and application environments; Step S1b: For each type of data, use a feature selection algorithm to select the most representative and discriminative data items as atomic labels; Step S1c: For each atomic tag, use an encoding algorithm to convert it into binary or other formats to facilitate subsequent data transmission and processing; Step S1d: Based on the product type and collection period, the corresponding atomic tag is retrieved to generate a meter reading script template; the meter reading script template includes the collection object device, collection protocol, communication method, collection period, collection data items, and reading message rules for the collection data items; wherein the collection data items are represented by atomic tags.

3. The energy data collection method according to claim 1, wherein: The step S13 further includes: Step S13a: Obtain a sample data set from the database, including energy consumption data of different regions and time periods; divide the sample data set into a training set and a test set, and perform normalization processing; Step S13b, using ST-ResNet to build a deep residual network model, using data from different time periods as input to different layers, and using graph convolution operations to model spatial relationships; the deep residual network model includes an input layer, a residual layer, and an output layer; defining a loss function and an optimizer, and using a training set to train the deep residual network model; using a test set to evaluate the performance and accuracy of the deep residual network model; Step S13c: Obtain new data from the database, including energy consumption data for different regions and time periods; normalize the new data and use it as input to the deep residual network model; use the deep residual network model to predict the new data and obtain energy consumption trends in the future; determine the collection plan based on the prediction results and user needs, including the collection object, collection cycle, and collection data items.

4. The energy data collection method according to claim 3, wherein: The step S2 further includes: Step S21: Obtain meter reading commands and sort the data items by importance according to user needs; Step S22: Generate a dynamic meter reading command according to the data importance ranking; and generate a meter reading execution script based on the dynamic meter reading command.

5. The energy data collection method according to claim 1, wherein: The step S3 is further as follows: Step S31: Generate a corresponding meter reading data message according to the meter reading message rules in the meter reading execution script; Step S32: compress the meter reading command using a data compression algorithm, obtain the compressed meter reading command, and send it.

6. The energy data collection method according to claim 5, characterized in that: The step S31 further includes: S31a, converting the meter reading command into a numerical vector as input to the deep learning model; S31b. Build a deep learning model based on VAE or GAN, and train the model according to different objective functions and optimization methods; S31c, compressing the input vector using the trained deep learning model to obtain a compressed vector as the output of the deep learning model; S31d, selecting an asymmetric encryption algorithm and generating an encryption function based on different keys and encryption methods; S31e, encrypting the compressed vector using an encryption function to obtain an encrypted vector as output; S31f, select Hamming code or Reed-Solomon code error correction code algorithm, and generate a coding function according to different coding rates and error correction capabilities; S31g, using an encoding function to encode the encrypted vector to obtain an encoded vector, and adding a check bit and a parity bit as output, that is, obtaining a compressed and encrypted data message; The meter reading data is encoded using an autoencoder model to obtain a compressed encoding vector; the compressed encoding vector is transmitted; and the receiving end uses an autoencoder to decode the encoding and restore the data.

7. The energy data collection method according to claim 5, wherein: The step S31 further includes: Convert the meter reading command into a numerical vector as the input of the converter model; Build a ConvTransformer transformer model and train the model according to different objective functions and optimization methods; Use the trained model to compress the input vector to obtain the compressed vector, which is used as the compressed data packet as output.

8. The energy data collection method according to claim 1, wherein: The step S4 is further as follows: Step S41: After receiving and decompressing the meter reading command, the meter reading device verifies the collected data; Step S42: If the verification is wrong, resend the meter reading command to re-collect data; If the verification is successful, the collected data will be converted into a standardized communication protocol and data format on the edge node and sent to the central server; The central server receives the collected data and performs training, and uses the training results to drive the subsequent data collection process.

9. The energy data collection method according to claim 8, characterized in that: In step S42, the central server receives the collected data and performs training, and the process of using the training results to drive the subsequent data collection process is further as follows: Step S42a: Acquire the collected data and clean and format it; and convert the collected data into a window; Step S42b: construct and use an LSTM-ARIMA model, input historical window data, predict energy consumption in the future, adjust the model structure and parameters, train the model, and obtain prediction results; According to the prediction results, if the predicted energy consumption is greater than the threshold, the collection frequency is increased; if the predicted energy consumption is less than the threshold, the collection frequency is reduced; the collection frequency and accuracy are adjusted in real time; Collect actual data and feed it back to the prediction model for further training; when the prediction error is too large, retrain the model; Continuously optimize the prediction model; The steps for building the LSTM-ARIMA combined model include: Differentiate the original time series data to make it stable; normalize the processed data to the range of [0,1] as the input data of the LSTM model; Use ACF and PACF plots to determine the ARIMA parameters (p, d, q), train the ARIMA model using the training dataset, and use the ARIMA model to predict future data points. Subtract the ARIMA prediction from the actual time series data to obtain the residuals, and use the residual data to train the LSTM model. At this time, the goal of the LSTM is to learn and predict the patterns that the ARIMA model fails to capture, and use the LSTM model to predict future residuals; For a future data point, the ARIMA prediction value is added to the residual of the LSTM prediction to obtain the final prediction value and use it as the prediction result.

Images