Data storage method and system for smart electric meter

Abstract

The present invention relates to a data storage method and system for a smart meter, the method comprising: acquiring and processing the electricity consumption data collected by each smart meter in real time to obtain the processed electricity consumption data; dividing the processed electricity consumption data into blocks to obtain a corresponding plurality of data blocks; processing each of the data blocks to generate a plurality of copies, and storing them in different edge nodes to determine the number of copies; performing real-time calculation and aggregation on each of the stored data blocks through the edge nodes to obtain the corresponding calculation results; dynamically adjusting the data distribution of the aggregated data to ensure load balancing; when any edge node fails, performing data recovery through a copy mechanism, reallocating the affected data blocks to other healthy nodes, and determining the corresponding fault recovery time. The present invention can improve the efficiency and reliability of data storage in smart meters.

Description

Technical Field

[0001] The present invention relates to the technical field of electric meter, and in particular to a data storage method, system, electronic device and non-transitory computer-readable storage medium for a smart electric meter. Background Art

[0002] Nowadays, the data storage method of smart meters mainly adopts a centralized storage mode. Specifically, the smart meter transmits the real-time collected electricity consumption data to a centralized server through a wireless or wired network, and the server stores, processes and analyzes the data. This method has the advantages of convenient centralized management and strong data processing capabilities, and is widely used in smart grids. In addition, in order to cope with problems such as large data volume and unstable transmission, some systems also use intermediate cache devices to temporarily store data to ensure data integrity and transmission stability.

[0003] However, the existing centralized storage methods have some shortcomings. First, centralized servers are prone to single point failures. Once a server has problems, the data storage and processing capabilities of the entire system will be seriously affected. Second, with the increase in the number of smart meters, the amount of data has also exploded, and the scalability and data processing capabilities of centralized storage face huge challenges. In addition, delays and packet loss may occur during data transmission, affecting the real-time and accuracy of the data. Summary of the invention

[0004] In view of the technical problems existing in the prior art, the present invention provides a data storage method, system, electronic device and non-transitory computer-readable storage medium for a smart meter, which can improve the data storage efficiency and reliability of the smart meter.

[0005] The technical solution of the present invention to solve the above technical problems is as follows:

[0006] The present invention provides a data storage method for a smart meter, the method comprising:

[0007] The electricity consumption data collected by each smart meter is acquired and processed in real time to obtain the processed electricity consumption data;

[0008] Dividing the processed power consumption data into blocks to obtain corresponding multiple data blocks;

[0009] Processing each of the data blocks to generate multiple copies, and storing them in different edge nodes, and determining the number of copies;

[0010] Perform real-time calculation and aggregation on each of the stored data blocks through the edge nodes to obtain corresponding calculation results;

[0011] Dynamically adjust the data distribution of aggregated data to ensure load balancing;

[0012] When any edge node fails, data recovery is performed through the replica mechanism, the affected data blocks are reallocated to other healthy nodes, and the corresponding failure recovery time is determined.

[0013] Optionally, the processing of each data block generates multiple copies, and stores the copies in different edge nodes, and determining the number of copies includes:

[0014] Obtaining a target storage duration for indicating a required storage duration for the plurality of data blocks;

[0015] Obtaining an average failure interval between failures of a plurality of edge nodes;

[0016] Obtaining a total number of the plurality of edge nodes;

[0017] Obtaining the failure rate and availability of each edge node;

[0018] The number of replicas is determined according to the target storage duration, the mean time between failures, the total number of the edge nodes, and the failure rate and availability of each edge node.

[0019] Optionally, the number of copies is expressed as:

[0020] ;

[0021] Where R is the number of replicas, T is the target data storage duration, M is the mean time between failures of edge nodes, N is the total number of edge nodes, F is the failure rate of each edge node, and U is the availability of each edge node. Indicates the round-up symbol. Indicates the floor symbol.

[0022] Optionally, performing real-time calculation and aggregation on each of the stored data blocks through the edge node to obtain corresponding calculation results includes:

[0023] Constructing a processing function of the edge node for each of the data blocks at a preset time;

[0024] Get the total number of data blocks;

[0025] Get the weight factor;

[0026] Determine the start time and end time of processing two adjacent data blocks;

[0027] The calculation result is determined according to the total number of the data blocks, the weight factor, and the start time and end time of processing two adjacent data blocks.

[0028] Optionally, the calculation result is expressed as: ;

[0029] in, is the calculation result, is the weight factor, It is the edge node for data blocks The processing function at time t, K is the total number of data blocks, are the start and end time of data block k-1 and data block k respectively.

[0030] Optionally, the dynamically adjusting the data distribution of the aggregated data includes:

[0031] Constructing an optimization target formula for load balancing according to the current load, processing capacity, load distribution factor, number of subtasks to be processed, load of the subtasks to be processed, and processing capacity of the subtasks to be processed of each edge node;

[0032] According to the optimization target formula of the load balancing, the data distribution of the aggregated data is dynamically adjusted.

[0033] Optionally, the optimization objective formula of the load balancing is expressed as: ;

[0034] Where N is the total number of edge nodes, is the current load of the ith edge node, is the processing capacity of the ith edge node, is the load distribution factor of the ith edge node, Q is the number of subtasks processed by the ith edge node, Process the load of the jth subtask for the i-th edge node, The processing capacity of the i-th edge node to process the j-th subtask.

[0035] Optionally, determining the corresponding fault recovery time includes:

[0036] Obtain network bandwidth for data recovery;

[0037] Get the total amount of data of the faulty node;

[0038] The data recovery time, the processing capacity of the edge node, the load factor of the edge node, the data volume, and the bandwidth of the edge node of each replica are obtained.

[0039] Optionally, the fault recovery time is expressed as:

[0040] ;

[0041] Among them, S is the total amount of data on the failed node, E is the network bandwidth, and R is the number of replicas. is the data recovery time of the ith replica, is the processing capacity of the edge node where the i-th replica is located, is the load factor of the ith edge node, is the data size of the ith replica, is the bandwidth of the edge node where the i-th replica is located.

[0042] The present invention also provides a data storage system for a smart meter, the system comprising:

[0043] The data acquisition module is used to acquire and process the electricity consumption data collected by each smart meter in real time to obtain the processed electricity consumption data;

[0044] A data block module, used for performing data block division on the processed power consumption data to obtain corresponding multiple data blocks;

[0045] A replica generation module, used to generate multiple replicas for each of the data blocks, store them in different edge nodes, and determine the number of replicas;

[0046] A data aggregation module is used to perform real-time calculation and aggregation on each of the stored data blocks through edge nodes to obtain corresponding calculation results;

[0047] Load balancing module, used to dynamically adjust the data distribution of aggregated data to ensure load balancing;

[0048] The fault recovery module is used to recover data through the replica mechanism when any edge node fails, reallocate the affected data blocks to other healthy nodes, and determine the corresponding fault recovery time.

[0049] In addition, to achieve the above objectives, the present invention also proposes an electronic device, comprising: a memory for storing a computer software program; a processor for reading and executing the computer software program, thereby implementing a data storage method for a smart meter as described above.

[0050] In addition, to achieve the above-mentioned purpose, the present invention also proposes a non-transitory computer-readable storage medium, in which a computer software program is stored. When the computer software program is executed by a processor, a data storage method of a smart meter as described above is implemented.

[0051] The beneficial effects of the present invention are:

[0052] (1) The present invention significantly improves data reliability by generating data copies and performing redundant storage on multiple edge nodes. Once a node fails, the system can quickly recover data from other nodes to avoid data loss.

[0053] (2) The present invention uses a distributed storage method to disperse and store data on multiple edge nodes, avoiding the storage bottleneck of a single centralized server. As the number of smart meters increases, the system can expand storage and processing capabilities by adding edge nodes to meet the needs of large-scale data.

[0054] (3) The edge nodes of the present invention perform preliminary data processing and calculation, which can reduce the burden on centralized servers and reduce data transmission delays. The edge computing and data aggregation steps can process and analyze data in real time, improving the efficiency and real-time performance of data processing.

[0055] (4) The present invention uses a dynamic load balancing mechanism to adjust data distribution according to the load conditions of each edge node, ensuring system load balancing and improving overall resource utilization and processing performance. The application of the load balancing optimization target formula enables the system to be dynamically adjusted during operation to improve efficiency and reliability.

[0056] (5) The replica management and fault recovery mechanism of the present invention ensures that the system can quickly recover data when an edge node fails, reducing the impact of the failure on the system. The introduction of the fault recovery time estimation formula enables the system to more accurately evaluate the fault recovery time and improve the system's fault tolerance.

[0057] (6) Since the data is initially processed and stored at the edge node, the present invention reduces the number of data transmissions and bandwidth consumption, reduces the risk of delays and data packet loss during transmission, and thus improves the integrity and accuracy of the data.

[0058] (7) The combination of distributed storage and edge computing in the present invention enables the system to flexibly cope with application scenarios of different scales and complexities. Whether it is a small-scale home user or a large-scale industrial user, the system can provide efficient and reliable data storage and processing services.

[0059] In summary, the present invention has significant improvements and enhancements in data storage reliability, system scalability, data processing real-time, load balancing, fault tolerance, transmission risk and system flexibility, which can effectively solve the shortcomings of existing centralized storage methods and adapt to the development needs of smart grids. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 A scenario diagram of a data storage method for a smart meter provided by the present invention;

[0061] Figure 2 A flow chart of a data storage method for a smart meter provided by the present invention;

[0062] Figure 3A schematic diagram of the structure of a data storage system for a smart meter provided by the present invention;

[0063] Figure 4 A schematic diagram of the hardware structure of a possible electronic device provided by the present invention;

[0064] Figure 5 A schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0066] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0067] In the description of the present invention, the term "for example" is used to mean "used as an example, illustration or explanation". Any embodiment described as "for example" in the present invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present invention.

[0068] See also Figure 1 , Figure 1 A scenario diagram of a data storage method for a smart meter provided by the present invention. Figure 1As shown, the terminal and the server are connected via a network, such as a wired or wireless network connection. The terminal may include but is not limited to portable terminals such as mobile phones and tablets installed with various network platform applications, as well as fixed terminals such as computers, query machines, and advertising machines. The server provides users with various business services, including service push servers, user recommendation servers, etc.

[0069] It should be noted that Figure 1 The scenario diagram of a data storage method for a smart meter shown is only an example. The terminal, server, and application scenario described in the embodiment of the present invention are intended to more clearly illustrate the technical solution of the embodiment of the present invention, and do not generate any limitation on the technical solution provided by the embodiment of the present invention. Ordinary technicians in this field can know that with the evolution of the system and the emergence of new business scenarios, the technical solution provided by the embodiment of the present invention is also applicable to similar technical problems.

[0070] Among them, the terminal can be used for:

[0071] The electricity consumption data collected by each smart meter is acquired and processed in real time to obtain the processed electricity consumption data;

[0072] Dividing the processed power consumption data into blocks to obtain corresponding multiple data blocks;

[0073] Processing each of the data blocks to generate multiple copies, and storing them in different edge nodes, and determining the number of copies;

[0074] Perform real-time calculation and aggregation on each of the stored data blocks through the edge nodes to obtain corresponding calculation results;

[0075] Dynamically adjust the data distribution of aggregated data to ensure load balancing;

[0076] When any edge node fails, data recovery is performed through the replica mechanism, the affected data blocks are reallocated to other healthy nodes, and the corresponding failure recovery time is determined.

[0077] See also Figure 2 , provides a flow chart of a data storage method for a smart meter of the present invention, comprising the following steps:

[0078] Step 201: Acquire and process the electricity consumption data collected by each smart meter in real time to obtain processed electricity consumption data.

[0079] In some embodiments, the processed electricity usage data may be represented as:

[0080] ;

[0081] in, is the processed electricity consumption data, which is the set of electricity consumption data processed by the i-th meter in the time period t. This set contains a series of power parameters collected at different times;

[0082] is the i-th meter at time Power; is the voltage; is the electric current; Timestamp refers to the time point when data is recorded, usually expressed as date and time, such as 2024-07-27, 10:45:00. Timestamp is used to mark the specific moment of each data collection.

[0083] On each smart meter, the user's electricity consumption data is collected in real time and processed preliminarily. Preliminary processing includes data denoising: removing noise from the collected data through filtering and other technologies to ensure data accuracy. Abnormal data identification and filtering: detecting and removing abnormal values ​​to prevent abnormal data from affecting subsequent calculations and analysis.

[0084] The data after preliminary processing is in the form of , contains multiple records, each record is represented by a power ,Voltage , Current and timestamp composition.

[0085] Through the above methods, the present invention can ensure that the data collected by each smart meter is accurate and useful, providing a reliable foundation for subsequent distributed storage, edge computing and data analysis.

[0086] Step 202: Block the processed electricity usage data to obtain a plurality of corresponding data blocks.

[0087] In some embodiments, a data block may be represented as:

[0088] ;

[0089] in, It is a data block, which contains a series of power parameter records collected within a specific time range;

[0090] k is the block number, which indicates the number of the data block. All data of the ith meter are divided into multiple data blocks, each of which has a unique number k, which starts from 0 and increases in sequence;

[0091] m is the amount of data contained in each data block, indicating the number of records in each data block. Contains m records.

[0092] When k=0, the data block Contains records 1 to m.

[0093] When k=1, the data block It contains records from m+1 to 2m.

[0094] And so on, the kth data block It includes To records.

[0095] In this way, all data is evenly divided into multiple data blocks. ,Each data block is of equal size, which facilitates uniform storage and processing on multiple edge nodes.,Each data block contains a record of power parameters within a specific time range, ensuring the continuity and integrity of the data.

[0096] Step 203: generate multiple copies for each of the data blocks, store them in different edge nodes, and determine the number of copies.

[0097] In some embodiments, step 203 may include:

[0098] Obtaining a target storage duration for indicating a required storage duration for the plurality of data blocks;

[0099] Obtaining an average failure interval between failures of a plurality of edge nodes;

[0100] Obtaining a total number of the plurality of edge nodes;

[0101] Obtaining the failure rate and availability of each edge node;

[0102] The number of replicas is determined according to the target storage duration, the mean time between failures, the total number of the edge nodes, and the failure rate and availability of each edge node.

[0103] In some embodiments, the number of replicas is expressed as:

[0104] ;

[0105] Where R is the number of replicas, T is the target data storage duration, M is the mean time between failures of edge nodes, N is the total number of edge nodes, F is the failure rate of each edge node, and U is the availability of each edge node. Indicates the round-up symbol. Indicates the floor symbol.

[0106] In the specific implementation, R is the number of replicas, that is, the number of copies of each data block that needs to be stored. By generating multiple replicas, the system can still ensure the availability and integrity of data when some nodes fail.

[0107] T is the target data storage duration, which indicates the time period for which the data needs to be stored. The unit can be hours, days, months, etc. This parameter determines the persistence and stability of the data that needs to be stored;

[0108] M is the mean time between failures of edge nodes, which indicates the average time interval between failures of nodes, and can be in hours, days, etc. A longer time between failures means that nodes are less likely to fail;

[0109] N is the total number of edge nodes, which indicates the number of all edge nodes in the system. More edge nodes can improve the scalability of the system and the flexibility of distributed storage;

[0110] F is the failure probability of each edge node, which indicates the possibility of failure of the edge node in a certain period of time. A higher failure probability means that the node is more likely to fail;

[0111] U is the availability of each edge node, which indicates the probability that the edge node can operate normally within a specific time period. High availability means that the node is less likely to fail.

[0112] Indicates rounding up, which means taking the smallest integer not less than the number. For example, ;

[0113] Indicates rounding down, which means taking the largest integer not greater than the number. For example, .

[0114] In the specific implementation, It represents the ratio of the target data storage duration T to the average failure interval M of the edge nodes. This part calculates the expected number of node failures during the data storage period. By rounding up, it ensures that even if the node failure interval M does not divide T, there will be enough replicas to cope with the possible number of failures.

[0115] It is a combination of the total number of nodes in the system, N, the failure probability of each node, F, and the node availability, U. By taking the square root and rounding down, we estimate how many replicas the system needs to ensure data reliability given the failure probability and availability. The constant term +1 ensures that no matter how it is calculated, the system will generate at least one extra replica to provide a minimum level of fault tolerance.

[0116] When the target data storage time T is long, or the mean time between failures M of the edge nodes is short, This part will become larger, indicating that more replicas are needed to ensure data reliability. When the total number of nodes N in the system is large, the failure probability F of each node is high, or the node availability U is low, This part will become larger, indicating that more copies are needed to cope with node failures. By comprehensively considering the results of these two parts and adding 1, the number of copies R that need to be generated is finally determined to ensure the reliability and fault tolerance of data in the distributed storage system. This allows the system to ensure data integrity and availability when a node fails or becomes unavailable, improving the robustness and reliability of the entire smart meter data storage solution.

[0117] Step 204: Perform real-time calculation and aggregation on each of the stored data blocks through the edge node to obtain corresponding calculation results.

[0118] In some embodiments, step 204 may include:

[0119] Constructing a processing function of the edge node for each of the data blocks at a preset time;

[0120] Get the total number of data blocks;

[0121] Get the weight factor;

[0122] Determine the start time and end time of processing two adjacent data blocks;

[0123] The calculation result is determined according to the total number of the data blocks, the weight factor, and the start time and end time of processing two adjacent data blocks.

[0124] In some embodiments, the calculation result can be expressed as:

[0125] ;

[0126] in, is the calculation result, is the weight factor, It is the edge node for data blocks The processing function at time t, K is the total number of data blocks, are the start and end time of data block k-1 and data block k respectively.

[0127] In the specific implementation, is the calculation result, which represents the comprehensive processing result of the ith meter in the time period t. This result is obtained by weighted integration of the processing results of each data block;

[0128] is the weight factor, which indicates the weight of the kth data block in the comprehensive calculation. The weight factor is used to adjust the influence of each data block on the final calculation result. Different data blocks may be given different weights according to their importance or other criteria.

[0129] It is the edge node for data blocks The processing function at time t, which indicates that within the time period t, the edge node processes the kth data block. The processing operations performed may be data calculation, analysis or other processing;

[0130] K is the total number of data blocks, which indicates the total number of blocks into which the data of the i-th meter is divided. The entire data is divided into K data blocks, and each data block contains a number of records within a specific time range;

[0131] They are the start and end time of data block k-1 and data block k, respectively, defining the range of each data block on the timeline.

[0132] For each data block In the corresponding time period [ ], calculate the processing function The integral represents the cumulative effect of edge nodes processing data blocks during this time period.

[0133] By weight factor Weight the processing results of each data block. Weight factor Adjustments can be made based on the importance of different data blocks or other criteria to ensure that the impact of different data blocks on the final calculation results is in line with expectations. Add up each weighted integral result to get the final comprehensive processing result. .

[0134] This is done by The processing results within its time period are integrated and weighted summed to obtain the comprehensive processing result of the i-th meter in the entire time period t This method can comprehensively consider the processing of data blocks in different time periods and adjust the contribution of each data block to the final result according to the weight factor.

[0135] This makes the calculation results more flexible, and the importance of different data blocks can be adjusted according to actual needs. For example, if a data block in a certain time period is particularly important for system analysis, it can be given a higher weight to increase its influence in the comprehensive results.

[0136] Processing Function It can be defined according to the specific application scenario, which may involve operations such as data calculation, filtering, and aggregation. By integrating the processing function, the processing effect on the data block within a specific time period can be fully reflected.

[0137] Processing data in blocks and performing weighted integration helps improve the system's computing efficiency and processing capabilities. Block processing can be performed in parallel to increase processing speed; weighted integration ensures the accuracy and flexibility of the results.

[0138] Through the above methods, the present invention can comprehensively process the data within the time period t, comprehensively consider the importance and processing effects of different data blocks, and thus provide a reliable and flexible comprehensive processing result. This method can effectively solve the problems of real-time, accuracy and flexibility of data processing in a distributed storage and edge computing environment.

[0139] Step 205: Dynamically adjust the data distribution of the aggregated data to ensure load balancing.

[0140] In some embodiments, step 205 may include:

[0141] Constructing an optimization target formula for load balancing according to the current load, processing capacity, load distribution factor, number of subtasks to be processed, load of the subtasks to be processed, and processing capacity of the subtasks to be processed of each edge node;

[0142] According to the optimization target formula of the load balancing, the data distribution of the aggregated data is dynamically adjusted.

[0143] In some embodiments, the optimization objective formula of load balancing is expressed as:

[0144] ;

[0145] Where N is the total number of edge nodes, is the current load of the ith edge node, is the processing capacity of the ith edge node, is the load distribution factor of the ith edge node, Q is the number of subtasks processed by the ith edge node, Process the load of the jth subtask for the i-th edge node, The processing capacity of the i-th edge node to process the j-th subtask.

[0146] In the specific implementation, N is the total number of edge nodes, which indicates the number of all edge nodes involved in load distribution and processing in the system;

[0147] is the current load of the ith edge node, indicating the total load that the node is currently processing. The load can be measured by indicators such as the number of tasks, task complexity, and data processing volume;

[0148] is the processing capacity of the ith edge node, indicating the maximum load that the node can handle. The processing capacity can be determined by factors such as hardware performance, network bandwidth, and computing resources;

[0149] is the load distribution factor of the ith edge node, which indicates the weight of the node when allocating subtasks. The load distribution factor is used to adjust the priority and resource allocation of the node when processing subtasks;

[0150] Q is the number of subtasks processed by the i-th edge node, indicating the total number of subtasks that the node needs to process. Each node may be assigned several subtasks;

[0151] The load of the i-th edge node processing the j-th subtask represents the load caused by the subtask on node i;

[0152] is the processing capability of the jth subtask of the i-th node, indicating the capability of node i to process the j-th subtask.

[0153] Indicates the current load of the i-th node With its processing power The ratio between the load on the node and the processing power of the node. This ratio measures the load on the node relative to its processing power. The smaller the ratio, the lighter the load on the node relative to its processing power.

[0154] Represents the load of all subtasks processed by the i-th node and processing power The weighted sum of the ratios between Adjusts the weight of each node when assigning subtasks. The smaller the ratio, the lighter the load on the node when processing subtasks relative to its processing capacity.

[0155] The ratio of all nodes is summed and squared. The sum operation takes into account the load of all nodes, while the square operation emphasizes the importance of load balancing to avoid situations where some nodes are overloaded while others are underloaded. The goal of optimization is to minimize the square of this sum, thereby achieving a balanced distribution of system load. By minimizing this value, we can ensure that the load of each node is distributed as evenly as possible within its processing capacity, improving the overall efficiency and performance of the system.

[0156] By minimizing the ratio of load to processing power, load balancing can be achieved to ensure that the load of each node is within a reasonable range of its processing power. This can avoid the situation where some nodes are overloaded and cause performance degradation or failure.

[0157] By introducing the load distribution factor , you can flexibly adjust the priority of different nodes when processing subtasks. Weight factor It can be set according to the specific situation of the node to ensure that the load distribution is more reasonable and efficient.

[0158] The present invention comprehensively considers the current load, processing capacity, subtask load and processing capacity of the node, as well as the load distribution factor, so as to achieve the global optimal load distribution strategy. In this way, resources can be reasonably allocated between different nodes, improving the overall performance and reliability of the system. The system can achieve efficient load balancing and resource optimization in a distributed environment, ensuring the efficiency and reliability of smart meter data storage and processing.

[0159] Step 206: When any edge node fails, data recovery is performed through a replica mechanism, the affected data blocks are reallocated to other healthy nodes, and the corresponding failure recovery time is determined.

[0160] In some embodiments, step 206 may include:

[0161] Obtain network bandwidth for data recovery;

[0162] Get the total amount of data of the faulty node;

[0163] The data recovery time, the processing capacity of the edge node, the load factor of the edge node, the data volume, and the bandwidth of the edge node of each replica are obtained.

[0164] In some embodiments, the failure recovery time is expressed as:

[0165] ;

[0166] Among them, S is the total amount of data on the failed node, E is the network bandwidth, and R is the number of replicas. is the data recovery time of the ith replica, is the processing capacity of the edge node where the i-th replica is located, is the load factor of the ith edge node, is the data size of the ith replica, is the bandwidth of the edge node where the i-th replica is located.

[0167] In the specific implementation, S is the total amount of data on the failed node, which means the amount of all data stored on the failed node that needs to be recovered. The unit can be bytes, kilobytes, megabytes, etc.

[0168] E is the network bandwidth, which indicates the network bandwidth used for data recovery. The unit can be bits per second (bps), kilobits per second (kbps), megabits per second (Mbps), etc.

[0169] R is the number of replicas, which indicates the total number of replicas generated for a data block in the system. More replicas can improve the reliability and speed of data recovery;

[0170] is the data recovery time of the ith replica, which indicates the time required to restore the ith replica, in seconds, minutes, hours, etc.;

[0171] is the processing capacity of the node where the i-th replica is located, indicating the maximum capacity of the node in data processing. The unit can be operations per second (OPS), bytes per second (BPS), etc.

[0172] is the load coefficient of the i-th node, which indicates the load of the node during data recovery. The load coefficient is used to measure the current load pressure of the node. The larger the value, the heavier the load.

[0173] is the data volume of the ith replica, indicating the specific data volume of the replica on the ith node, in bytes, kilobytes, megabytes, etc.

[0174] is the bandwidth of the node where the i-th replica is located, indicating the network bandwidth used to recover data. The unit can be bits per second (bps), kilobits per second (kbps), megabits per second (Mbps), etc.

[0175] This part represents the time required to transmit data from the failed node. S represents the total amount of data on the failed node, and E represents the network bandwidth used for recovery. The ratio of the two is the time required for data transmission.

[0176] Indicates the total time required to restore all replicas. middle represents the data recovery time of the i-th replica, represents the processing capacity of the node where the i-th replica is located, and the ratio of the two represents the time required for the i-th replica to recover.

[0177] middle represents the load factor of the i-th node, represents the amount of data in the ith replica, Represents the bandwidth of the node where the ith replica is located. The product of the load factor and the amount of data divided by the bandwidth represents the time adjustment required to restore the ith replica under the current load.

[0178] When a node failure occurs, the data recovery process needs to transfer data to other nodes through the network for recovery. Transmission time It depends on the total amount of data S on the failed node and the network bandwidth E used for recovery.

[0179] For each replica, data recovery time and load adjustment time Comprehensive consideration ensures that the time required for recovery is accurately estimated under different node load conditions and bandwidth conditions. By calculating the sum of the recovery time of all replicas, the formula comprehensively considers the impact of the number of replicas z in the system on data recovery time. More replicas can improve the reliability and efficiency of data recovery.

[0180] Load Factor The introduction of ensures that the recovery time calculation is more accurate when the node load is different. Nodes with heavier loads may take longer to recover data, and the load factor in the formula adjusts for this effect.

[0181] Through the above methods, the present invention can accurately calculate the total time required for data recovery when a faulty node occurs. The formula comprehensively considers the total amount of data, network bandwidth, number of copies, node processing capacity, load conditions and bandwidth conditions of the faulty node, thereby providing a comprehensive and accurate data recovery time estimation method. This method can effectively solve the real-time and accuracy problems of data recovery in a distributed storage and edge computing environment, and improve the reliability and robustness of the system.

[0182] See also Figure 3 , Figure 3 A structural schematic diagram of a data storage system for a smart meter provided by the present invention.

[0183] like Figure 3 As shown, a data storage system for a smart meter proposed in an embodiment of the present invention includes:

[0184] The data acquisition module 301 is used to acquire and process the electricity consumption data collected by each smart meter in real time to obtain the processed electricity consumption data;

[0185] A data block module 302 is used to block the processed power consumption data to obtain corresponding multiple data blocks;

[0186] A replica generation module 303 is used to generate multiple replicas for each of the data blocks, store them in different edge nodes, and determine the number of replicas;

[0187] The data aggregation module 304 is used to perform real-time calculation and aggregation on each of the stored data blocks through the edge nodes to obtain corresponding calculation results;

[0188] Load balancing module 305, used to dynamically adjust the data distribution of aggregated data to ensure load balancing;

[0189] The fault recovery module 306 is used to recover data through a replica mechanism when any edge node fails, reallocate affected data blocks to other healthy nodes, and determine the corresponding fault recovery time.

[0190] See also Figure 4 , Figure 4 Schematic diagram of an electronic device provided by an embodiment of the present invention. Figure 4 As shown, an embodiment of the present invention provides an electronic device 400, including a memory 410, a processor 420, and a computer program 411 stored in the memory 410 and executable on the processor 420. When the processor 420 executes the computer program 411, the following steps are implemented:

[0191] The electricity consumption data collected by each smart meter is acquired and processed in real time to obtain the processed electricity consumption data;

[0192] Dividing the processed power consumption data into blocks to obtain corresponding multiple data blocks;

[0193] Processing each of the data blocks to generate multiple copies, and storing them in different edge nodes, and determining the number of copies;

[0194] Perform real-time calculation and aggregation on each of the stored data blocks through the edge nodes to obtain corresponding calculation results;

[0195] Dynamically adjust the data distribution of aggregated data to ensure load balancing;

[0196] When any edge node fails, data recovery is performed through the replica mechanism, the affected data blocks are reallocated to other healthy nodes, and the corresponding failure recovery time is determined.

[0197] See also Figure 5 , Figure 5 A schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present invention. Figure 5As shown, this embodiment provides a computer-readable storage medium 500, on which a computer program 411 is stored. When the computer program 411 is executed by a processor, the following steps are implemented:

[0198] The electricity consumption data collected by each smart meter is acquired and processed in real time to obtain the processed electricity consumption data;

[0199] Dividing the processed power consumption data into blocks to obtain corresponding multiple data blocks;

[0200] Processing each of the data blocks to generate multiple copies, and storing them in different edge nodes, and determining the number of copies;

[0201] Perform real-time calculation and aggregation on each of the stored data blocks through the edge nodes to obtain corresponding calculation results;

[0202] Dynamically adjust the data distribution of aggregated data to ensure load balancing;

[0203] When any edge node fails, data recovery is performed through the replica mechanism, the affected data blocks are reallocated to other healthy nodes, and the corresponding failure recovery time is determined.

[0204] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0205] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0206] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or multiple boxes.

[0207] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction system, which is implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0208] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0209] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0210] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A data storage method for a smart meter, characterized in that: The method comprises: The electricity consumption data collected by each smart meter is acquired and processed in real time to obtain the processed electricity consumption data; Dividing the processed power consumption data into blocks to obtain corresponding multiple data blocks; Generate multiple copies for each data block and store them in different edge nodes, and determine the number of copies, including: obtaining a target storage time for representing the storage time required for the multiple data blocks; obtaining an average failure interval between failures of the multiple edge nodes; obtaining the total number of the multiple edge nodes; obtaining the failure rate and availability of each edge node; determining the number of copies according to the target storage time, the average failure interval, the total number of the multiple edge nodes, and the failure rate and availability of each edge node; the number of copies is expressed as: ; Where R is the number of replicas, T is the target data storage duration, M is the mean time between failures of edge nodes, N is the total number of edge nodes, F is the failure rate of each edge node, and U is the availability of each edge node. Indicates the round-up symbol. Indicates the floor symbol; Perform real-time calculation and aggregation on each of the stored data blocks through the edge nodes to obtain corresponding calculation results; Dynamically adjust the data distribution of aggregated data to ensure load balancing; When any edge node fails, data recovery is performed through the replica mechanism, the affected data blocks are reallocated to other healthy nodes, and the corresponding failure recovery time is determined.

2. The data storage method of the smart meter according to claim 1, characterized in that: The edge node performs real-time calculation and aggregation on each of the stored data blocks to obtain corresponding calculation results, including: Constructing a processing function of the edge node for each of the data blocks at a preset time; Get the total number of data blocks; Get the weight factor; Determine the start time and end time of processing two adjacent data blocks; The calculation result is determined according to the total number of the data blocks, the weight factor, and the start time and end time of processing two adjacent data blocks.

3. The data storage method of the smart meter according to claim 2, characterized in that: The calculation result is expressed as: ; in, is the calculation result, is the weight factor, It is the edge node for data blocks The processing function at time t, K is the total number of data blocks, are the start and end times of data block k-1 and data block k respectively.

4. The data storage method of the smart meter according to claim 3, characterized in that: The dynamically adjusting the data distribution of the aggregated data includes: Constructing an optimization target formula for load balancing according to the current load, processing capacity, load distribution factor, number of subtasks to be processed, load of the subtasks to be processed, and processing capacity of the subtasks to be processed of each edge node; According to the optimization target formula of the load balancing, the data distribution of the aggregated data is dynamically adjusted.

5. The data storage method of the smart meter according to claim 4, characterized in that: The optimization objective formula of the load balancing is expressed as: ; Where N is the total number of edge nodes, is the current load of the ith edge node, is the processing capacity of the ith edge node, is the load distribution factor of the ith edge node, Q is the number of subtasks processed by the ith edge node, Process the load of the jth subtask for the i-th edge node, The processing capacity of the i-th edge node to process the j-th subtask.

6. The data storage method of the smart meter according to claim 5, characterized in that: The determining the corresponding fault recovery time includes: Obtain network bandwidth for data recovery; Get the total amount of data of the faulty node; The data recovery time, the processing capacity of the edge node, the load factor of the edge node, the data volume, and the bandwidth of the edge node of each replica are obtained.

7. The data storage method of a smart meter according to any one of claims 1 to 6, characterized in that: The fault recovery time is expressed as: ; Among them, S is the total amount of data on the failed node, E is the network bandwidth, and R is the number of replicas. is the data recovery time of the ith replica, is the processing capacity of the edge node where the i-th replica is located, is the load factor of the ith edge node, is the amount of data in the ith replica, is the bandwidth of the edge node where the i-th replica is located.

8. A data storage system for a smart meter, characterized in that: The system comprises: The data acquisition module is used to acquire and process the electricity consumption data collected by each smart meter in real time to obtain the processed electricity consumption data; A data block module, used for performing data block division on the processed power consumption data to obtain corresponding multiple data blocks; The replica generation module is used to generate multiple replicas for each data block, store them in different edge nodes, determine the number of replicas, and is also used to: obtain a target storage duration for representing the required storage duration for multiple data blocks; obtain an average failure interval between failures of multiple edge nodes; obtain the total number of multiple edge nodes; obtain the failure rate and availability of each edge node; determine the number of replicas based on the target storage duration, the average failure interval, the total number of multiple edge nodes, and the failure rate and availability of each edge node; the number of replicas is expressed as: ; Where R is the number of replicas, T is the target data storage duration, M is the mean time between failures of edge nodes, N is the total number of edge nodes, F is the failure rate of each edge node, and U is the availability of each edge node. Indicates the round-up symbol. Indicates the floor symbol; A data aggregation module is used to perform real-time calculation and aggregation on each of the stored data blocks through edge nodes to obtain corresponding calculation results; Load balancing module, used to dynamically adjust the data distribution of aggregated data to ensure load balancing; The fault recovery module is used to recover data through the replica mechanism when any edge node fails, reallocate the affected data blocks to other healthy nodes, and determine the corresponding fault recovery time.

Images