A Power Data Caching Optimization System Based on Distributed Terminals

Abstract

A power data caching optimization system based on distributed terminals, which relates to the field of data caching technology; obtaining power nodes and deploying corresponding caching nodes, setting corresponding power parameters for different power nodes, and generating caching parameters for each caching node, obtaining the distributed caching nodes and backup caching nodes of each power node, obtaining distributed data sub-packets and backup data sub-packets and caching them respectively, obtaining the data integrity and caching hit rate of each caching node, re-caching power data packets according to the data integrity, and expanding nodes for caching nodes according to the caching hit rate; it is beneficial to provide effective reference indicators for distributed data caching according to the data collection volume and equipment load conditions in different time periods, and can significantly improve the caching efficiency of different caching nodes for power data packets.

Description

Technical Field

[0001] The present invention relates to the technical field of data caching, and specifically to a power data caching optimization system based on distributed terminals. Background Art

[0002] Distributed data caching and optimization of power data are crucial components in the power system. It aims to improve data access speed, relieve the pressure on the database, and optimize the overall system performance. By storing data distributively on multiple caching nodes in the network, it can achieve efficient access and processing of data. In the field of data caching, such a system can significantly enhance the real-time performance, reliability, and stability of the power system;

[0003] In the prior art, most of the caching nodes for distributed data caching lack flexibility. They ignore that the caching capabilities of caching nodes are different in different time periods and fail to implement a flexible caching mechanism for data caching according to this feature. Moreover, the partitioning of cached data also lacks flexibility and fails to form a corresponding partitioning mechanism based on the selection of caching nodes. In view of the deficiencies of the prior art, the present invention provides a power data caching optimization system based on distributed terminals. Summary of the Invention

[0004] The purpose of the present invention is to provide a power data caching optimization system based on distributed terminals.

[0005] The purpose of the present invention can be achieved through the following technical solutions: A power data caching optimization system based on distributed terminals includes the following modules:

[0006] A node deployment module, which is used to obtain power nodes in the power system and deploy corresponding caching nodes according to the power nodes by using virtual machine technology;

[0007] A data acquisition module, which is used to obtain power data packets of each power node and set corresponding power parameters for different power nodes according to the size of the data packets;

[0008] A node selection module, which is used to obtain the caching information of each caching node and generate caching parameters of each caching node according to the caching information combined with the corresponding power parameters;

[0009] A data caching module, which is used to obtain the set of caching nodes of each power node, obtain several distributed caching nodes and backup caching nodes from the set of caching nodes according to the caching parameters, perform distributed partitioning and backup partitioning on the power data packets to obtain distributed data sub-packets and backup data sub-packets, and cache the distributed data sub-packets and backup data sub-packets respectively;

[0010] The node optimization module is used to obtain the data integrity and cache hit rate of each cache node respectively, re-cache the power data packets according to the data integrity, and expand the cache nodes according to the cache hit rate.

[0011] Further, the process of obtaining the power nodes in the power system and deploying the corresponding cache nodes according to the power nodes by using virtual machine technology includes:

[0012] The power node refers to a physical device that undertakes specific functions and generates relevant data in the power system, including five node types: power generation node, power transmission node, power transformation node, power distribution node, and power consumption node;

[0013] Collect the location information and connection information of each power node respectively, use virtual machine technology to construct a corresponding cache node for each power node, deploy the cache nodes according to the location information, and connect the cache nodes according to the connection information;

[0014] Obtain the deployment distances of different cache nodes according to the deployment locations of the cache nodes, and the node type of the cache node is the same as that of its power node.

[0015] Further, the process of obtaining the power data packets of each power node and setting the corresponding power parameters for different power nodes according to the packet sizes of the power data packets includes:

[0016] Set the collection period, and regard all the power data of each power node within the same collection period as its power data packet within the collection period. The power data refers to the relevant data generated by the power node during its operation;

[0017] According to the packet size S of the power data packets of each power node of the same node type within the same collection period i , i = 1, 2,..., n, where n is the number of power nodes of this node type, obtain the power parameter D of a single power node i ;

[0018]

[0019] Among them, when , its corresponding power parameter is positive, when , its corresponding power parameter is negative, when , its corresponding power parameter is 0, and continuously use the latest power parameter as the power parameter of each power node within the current collection period.

[0020] Further, the process of obtaining the cache information of each cache node and generating the cache parameters of each cache node according to the cache information combined with the corresponding power parameters includes:

[0021] The cache information refers to the load conditions of the virtual machines corresponding to the cache nodes, including the CPU utilization rate C a , the memory usage rate C b , the network bandwidth C c , the disk read volume C d , the disk write volume C e , and corresponding weight values are set for different cache information, which are Q a , Q b , Q c , Q d , Q e ;

[0022] Combined with the power parameters D of the power nodes corresponding to each cache node i , cache parameters H of each cache node in the corresponding collection period are generated;

[0023]

[0024] Taking the cache parameters when the power parameter is 0 as the standard cache parameters, comparing the cache parameters of each cache node in the current collection period with the standard cache parameters to obtain the states of each cache node, including the unavailable state and the available state.

[0025] Further, the process of obtaining the cache node set of each power node and obtaining several distributed cache nodes and backup cache nodes from the cache node set according to the cache parameters includes:

[0026] Taking the cache nodes corresponding to a single power node as the cache center, obtaining other cache nodes of the same node type as the cache center and the deployment distances between them and the cache center, and incorporating all cache nodes with a deployment distance less than or equal to the preset fixed distance into the cache node set of this power node;

[0027] Obtaining the cache parameters of each cache node in the cache node set in the current collection period, arranging the cache nodes in the available state among them in ascending order, and obtaining 2k cache nodes from them;

[0028] Randomly dividing the 2k cache nodes into two groups with the same number, one group being the distributed cache nodes and the other group being the backup cache nodes, and binding the distributed cache nodes and the backup cache nodes in the same collection period to their power nodes.

[0029] Further, the process of performing distribution partitioning and backup partitioning on the power data packet to obtain distributed data sub - packets and backup data sub - packets and caching the distributed data sub - packets and the backup data sub - packets respectively includes:

[0030] Obtain the proportional relationship of the cache parameters of each distributed cache node belonging to the same power node within the same collection period. The distribution division refers to dividing the power data packets of this power node according to the proportional relationship to obtain several distributed data sub-packets, and uploading each distributed data sub-packet to the corresponding distributed cache node in the proportional relationship for caching;

[0031] Obtain the proportional relationship of the cache parameters of each backup cache node belonging to the same power node within the same collection period. The backup division refers to dividing the power data packets of this power node according to the proportional relationship to obtain several backup data sub-packets, and uploading each backup data sub-packet to the corresponding backup cache node in the proportional relationship for caching.

[0032] Further, the process of obtaining the data integrity of each cache node and re-caching the power data packets according to the data integrity includes:

[0033] Before caching the distributed data sub-packet in the distributed cache node, obtain the initial hash value of the distributed data sub-packet. After caching is completed, regularly obtain the subsequent hash value of the distributed data sub-packet, and compare the subsequent hash value with the initial hash value to obtain the data integrity of the distributed data sub-packet, including the complete state and the incomplete state;

[0034] For the distributed data sub-packet in the incomplete state, delete the distributed data sub-packets of the power data packet corresponding to this distributed data sub-packet in each distributed cache node, and combine the backup data sub-packets of this power data packet in each backup cache node to obtain the original power data packet;

[0035] Obtain the distributed cache nodes of the power node corresponding to this power data packet in the current collection period, re-divide this power data packet to obtain new distributed data sub-packets, and upload each new distributed data sub-packet to the corresponding distributed cache node for re-caching.

[0036] Further, the process of obtaining the cache hit rate of each cache node and expanding the cache node according to the cache hit rate includes:

[0037] After the power data packet caching is completed, the user accesses the power data of its power node through the cache center. The cache hit rate refers to the proportion of the number of times each cache node successfully reads its power data sub-packet to the total number of accesses;

[0038] Set a hit rate threshold, compare the cache hit rate of each cache node with the hit rate threshold respectively to obtain the nodes to be expanded. For the nodes to be expanded, add a new cache node on the basis of its original cache node, and update each cache node set.

[0039] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0040] 1. By deploying corresponding cache nodes according to power nodes respectively, the present invention provides a basis for realizing distributed data caching. Flexible power parameters are set for power nodes according to the sizes of power data packets in different collection cycles. At the same time, targeted cache parameters are generated for them in combination with the cache information of cache nodes in the corresponding collection cycles, which is conducive to providing effective reference indicators for distributed data caching according to the data collection volume and device load conditions in different time periods;

[0041] 2. By constructing a cache node set for each power node and simultaneously obtaining a group of distributed cache nodes and a group of backup cache nodes according to the cache parameters, and dividing and caching power data packets respectively according to the proportional relationship of the cache parameters of each cache node, it is conducive to dividing targeted power data sub - packets according to the cache capabilities of different cache nodes, which can significantly improve the cache efficiency of different cache nodes for power data packets. By obtaining the data integrity and cache hit rate of cache nodes and optimizing the cache nodes accordingly, it is conducive to reducing the cache pressure of a single cache node while ensuring the integrity of data caching and improving the effectiveness of data caching. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is the schematic diagram of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0043] As Figure 1 shown, a power data cache optimization system based on a distributed terminal includes the following modules:

[0044] A node deployment module, configured to obtain power nodes in a power system and deploy corresponding cache nodes according to the power nodes by using virtual machine technology;

[0045] A data collection module, configured to obtain power data packets of each power node and set corresponding power parameters for different power nodes according to the sizes of the data packets;

[0046] A node selection module, configured to obtain the cache information of each cache node and generate cache parameters for each cache node in combination with the corresponding power parameters according to the cache information;

[0047] A data cache module, configured to obtain the cache node set of each power node, obtain several distributed cache nodes and backup cache nodes in the cache node set according to the cache parameters, perform distributed division and backup division on the power data packets to obtain distributed data sub - packets and backup data sub - packets, and cache the distributed data sub - packets and backup data sub - packets respectively;

[0048] A node optimization module, which is used to obtain the data integrity and cache hit rate of each cache node respectively, re-cache the power data packets according to the data integrity, and expand the cache nodes according to the cache hit rate.

[0049] It should be further noted that in the specific implementation process, the process of obtaining the power nodes in the power system and deploying the corresponding cache nodes according to the power nodes by using virtual machine technology includes:

[0050] The power system is divided into five power links: power generation, power transmission, power transformation, power distribution, and power consumption. The power nodes refer to the physical devices that undertake specific functions and generate relevant data in each power link, including five node types: power generation nodes, power transmission nodes, power transformation nodes, power distribution nodes, and power consumption nodes;

[0051] Collect the location information and connection information of each power node respectively. The location information refers to the geographical location of the physical device corresponding to each power node, and the connection information refers to the connection relationship between the physical devices corresponding to each power node;

[0052] Use virtual machine technology to build a corresponding cache node for each power node. The cache node is used to cache data. Deploy the constructed cache nodes according to the location information of the power nodes, so that the deployment locations of the cache nodes correspond to the geographical locations of their power nodes respectively, and connect the corresponding cache nodes according to the connection information of the power nodes;

[0053] The geographical distance between different power nodes can be obtained according to the geographical location of the power nodes, and the deployment distance between different cache nodes can also be obtained according to the deployment location of the cache nodes. The node type of the cache node is the same as that of its power node.

[0054] It should be further noted that in the specific implementation process, the process of obtaining the power data packets of each power node and setting the corresponding power parameters for different power nodes according to the packet size of the power data packets includes:

[0055] Set the collection period, and collect the power data of each power node in real time. The power data refers to the relevant data generated by the power node during its operation. All the power data collected within the same collection period is used as the power data packet of its power node in this collection period;

[0056] Include the power data of each power node of the same node type within the same collection period into the same power data packet;

[0057] Obtain the packet size of the power data packets of each power node of the same node type within the same collection period, denoted as S i, where \(i = 1, 2, \ldots, n\), and \(n\) is the number of power nodes of this node type. Based on this, the power parameters of a single power node are obtained and denoted as \(D\). i ;

[0058]

[0059] Among them, when , its corresponding power parameter is positive. When , its corresponding power parameter is negative. When , its corresponding power parameter is 0.

[0060] Using the same method, the power parameters of power nodes of different node types in this collection period are obtained, and the latest power parameters are continuously used as the power parameters of each power node in the current collection period to achieve continuous update of the power parameters.

[0061] It should be further noted that in the specific implementation process, the process of obtaining the cache information of each cache node and generating the cache parameters of each cache node according to the cache information combined with the corresponding power parameters includes:

[0062] The cache nodes are built inside the virtual machines, and each cache node corresponds to a virtual machine. The cache information refers to the load conditions of the virtual machines corresponding to the cache nodes, including CPU utilization rate, memory usage rate, network bandwidth, disk read volume, and disk write volume.

[0063] Taking any cache node as an example, mark the CPU utilization rate, memory usage rate, network bandwidth, disk read volume, and disk write volume of this cache node as \(C\) a , \(C\) b , \(C\) c , \(C\) d , \(C\) e , and set corresponding weight values for different cache information, that is, \(Q\) a , \(Q\) b , \(Q\) c , \(Q\) d , \(Q\) e ;

[0064] Combined with the power parameter \(D\) i of the power node corresponding to this cache node, generate the cache parameter of this cache node, denoted as \(H\).

[0065]

[0066] Using the same method, obtain the cache parameters of each cache node in the same collection period, and continuously use the latest cache parameters as the cache parameters of each cache node in the current collection period to achieve continuous update of the cache parameters.

[0067] Take D i When it is 0, the cache parameter is used as the standard cache parameter, denoted as H0. Compare the cache parameters within the current collection period with the standard cache parameter respectively. If H > H0, mark its cache node as unavailable; if H ≤ H0, mark its cache node as available.

[0068] It should be further noted that in the specific implementation process, the process of obtaining the cache node sets of each power node and obtaining several distributed cache nodes and backup cache nodes according to the cache parameters includes:

[0069] Taking any power node as an example, use the cache node corresponding to this power node as the cache center, obtain the deployment distances between each cache node of the same node type as others and the cache center, and include all cache nodes with a distance less than or equal to the preset fixed distance into the cache node set of this power node;

[0070] Obtain the cache parameters of each cache node within the cache node set in the current collection period. Arrange the cache nodes in the available state in ascending order according to the cache parameters, and obtain 2k cache nodes from them. The 2k cache nodes obtained need to meet two conditions. One is that they are in the available state, and the other is that they are in the case of the smallest cache parameter;

[0071] Randomly divide the 2k cache nodes into two groups, with the same number of cache nodes in both groups, both being k. Mark one group as distributed cache nodes and the other group as backup cache nodes, and bind the distributed cache nodes and backup cache nodes obtained within the same collection period to their power nodes.

[0072] It should be further noted that in the specific implementation process, the process of performing distribution division and backup division on power data packets to obtain distributed data sub - packets and backup data sub - packets and caching the distributed data sub - packets and backup data sub - packets respectively includes:

[0073] Taking the distributed cache nodes of any power node as an example, obtain the proportional relationship between the cache parameters of each distributed cache node. The distribution division refers to dividing the power data packet of this power node according to the proportional relationship to obtain several distributed data sub - packets;

[0074] Among them, the number of the distributed data sub - packets is k, and the data packet sizes of each distributed data sub - packet correspond to the proportional relationship. Upload each distributed data sub - packet to the distributed cache node corresponding to it in the proportional relationship for caching;

[0075] Adopt the same method to obtain the proportional relationship between the cache parameters of each backup cache node. The backup division refers to dividing the power data packets of the power node according to the proportional relationship to obtain several backup data sub-packets, and uploading each backup data sub-packet to the backup cache node corresponding to it in the proportional relationship for caching.

[0076] It should be further noted that in the specific implementation process, the process of obtaining the data integrity of each cache node and re-caching the power data packets according to the data integrity includes:

[0077] Before the distributed cache node caches the distributed data sub-packet, obtain the initial hash value of the distributed data sub-packet. After the caching is completed, regularly obtain the subsequent hash value of the distributed data sub-packet using the same hash algorithm, and compare the obtained subsequent hash value with the initial hash value;

[0078] The data integrity includes a complete state and an incomplete state. If the two are the same, it is determined that the distributed data sub-packet is in a complete state, and no other operations are performed on it. If the two are different, it is determined that the distributed data sub-packet is in an incomplete state;

[0079] For the distributed data sub-packet in an incomplete state, delete the distributed data sub-packets of the power data packet corresponding to the distributed data sub-packet in each distributed cache node, and combine the backup data sub-packets of the power data packet in each backup cache node to obtain the original power data packet;

[0080] Obtain the distributed cache node of the power node corresponding to the power data packet in the current collection period, re-divide the power data packet to obtain new distributed data sub-packets, and upload each new distributed data sub-packet to the corresponding distributed cache node for re-caching.

[0081] It should be further noted that in the specific implementation process, the process of obtaining the cache hit rate of each cache node and expanding the cache node according to the cache hit rate includes:

[0082] After the power data packet is cached, the user can access the power data of its power node through the cache center. The cache hit rate refers to the ratio of the number of times each cache node successfully reads its power data sub-packet to the total number of accesses;

[0083] Set a hit rate threshold, compare the cache hit rate of each cache node with the hit rate threshold, mark the cache nodes with a cache hit rate greater than or equal to the hit rate threshold as nodes to be expanded. For the nodes to be expanded, add a new cache node on the basis of its original cache node, and update each cache node set.

[0084] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical method of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A power data caching optimization system based on distributed terminals, characterized in that, It includes the following modules: A node deployment module, which is used to obtain power nodes in the power system and deploy corresponding cache nodes according to the power nodes by using virtual machine technology; A data acquisition module, which is used to obtain power data packets of each power node and set corresponding power parameters for different power nodes according to the packet size of the power data packets; A node selection module, which is used to obtain cache information of each cache node and generate cache parameters of each cache node according to the cache information in combination with the corresponding power parameters; A data caching module, which is used to obtain a cache node set of each power node, obtain several distributed cache nodes and backup cache nodes from the cache node set according to the cache parameters, perform distributed division and backup division on the power data packets to obtain distributed data sub-packets and backup data sub-packets, and cache the distributed data sub-packets and backup data sub-packets respectively; A node optimization module, which is used to obtain the data integrity and cache hit rate of each cache node respectively, re-cache the power data packets according to the data integrity, and expand the cache nodes according to the cache hit rate; The process of obtaining cache information and generating cache parameters of each cache node includes: The cache information refers to the load conditions of the virtual machines corresponding to the cache nodes, including CPU utilization C a , memory usage C b , network bandwidth C c , disk read volume C d , disk write volume C e . Corresponding weight values are set for different cache information, which are Q a , Q b , Q c , Q d , Q e ; Combining the power parameter D of the power node corresponding to each cache node i , generate the cache parameter H of each cache node within the corresponding collection period; Taking the cache parameters when the power parameter is 0 as the standard cache parameters, comparing the cache parameters of each cache node in the current collection period with the standard cache parameters to obtain the status of each cache node, including unavailable status and available status; The process of obtaining a cache node set and several distributed cache nodes and backup cache nodes includes: Taking the cache node corresponding to a single power node as the cache center, obtaining other cache nodes of the same node type as the cache center and the deployment distance between them and the cache center, and including all cache nodes with a deployment distance less than or equal to the preset fixed distance in the cache node set of this power node; Obtaining the cache parameters of each cache node in the cache node set in the current collection period, arranging the cache nodes in the available state in ascending order according to the cache parameters, and obtaining 2k cache nodes from them; Randomly dividing the 2k cache nodes into two groups with the same number, one group is the distributed cache nodes and the other group is the backup cache nodes, and binding the distributed cache nodes and backup cache nodes in the same collection period to their power nodes.

2. The power data caching optimization system based on distributed terminals according to claim 1, wherein The process of obtaining power nodes and deploying corresponding cache nodes includes: The power node refers to a physical device that undertakes a specific function and generates relevant data in the power system, including five node types: power generation node, power transmission node, power transformation node, power distribution node, and power consumption node; Collecting the location information and connection information of each power node respectively, using virtual machine technology to construct a corresponding cache node for each power node, deploying the cache node according to the location information, and connecting the cache node according to the connection information; Obtaining the deployment distance of different cache nodes according to the deployment location of the cache node, and the node type of the cache node is the same as that of its power node.

3. A power data caching optimization system based on a distributed terminal according to claim 2, characterized in that, The process of obtaining power data packets and setting corresponding power parameters for different power nodes includes: Set the acquisition period, and use all the power data of each power node within the same acquisition period as its power data packet within this acquisition period. The power data refers to the relevant data generated by the power node during its operation; According to the data packet size S of the power data packets of each power node of the same node type within the same collection period i , where i = 1, 2, ……, n, and n is the number of power nodes of this node type, obtain the power parameter D of a single power node i ; Among them, when occurs, the corresponding electrical parameter is positive. When occurs, the corresponding electrical parameter is negative. When occurs, the corresponding electrical parameter is 0. Continuously use the latest electrical parameter as the electrical parameter of each electrical node within the current acquisition period.

4. A power data cache optimization system based on a distributed terminal according to claim 3, characterized in that The process of obtaining the distributed data sub-packet and the backup data sub-packet and caching them respectively includes: Obtain the proportional relationship of the cache parameters of each distributed cache node belonging to the same power node within the same acquisition period. The distributed division means dividing the power data packet of this power node according to the proportional relationship to obtain several distributed data sub-packets, and uploading each distributed data sub-packet to the corresponding distributed cache node for caching according to their proportional relationship; Obtain the proportional relationship of the cache parameters of each backup cache node belonging to the same power node within the same acquisition period. The backup division means dividing the power data packet of this power node according to the proportional relationship to obtain several backup data sub-packets, and uploading each backup data sub-packet to the corresponding backup cache node for caching according to their proportional relationship.

5. The power data cache optimization system based on a distributed terminal according to claim 4, characterized in that, Obtain the data integrity. The process of re-caching the power data packet includes: Before caching the distributed data sub-packet in the distributed cache node, obtain the initial hash value of this distributed data sub-packet. After caching is completed, regularly obtain the subsequent hash value of the distributed data sub-packet, and compare the subsequent hash value with the initial hash value to obtain the data integrity of the distributed data sub-packet, including the complete state and the incomplete state; For the distributed data sub-packet in the incomplete state, delete the distributed data sub-packets of the power data packet corresponding to this distributed data sub-packet in each distributed cache node, and combine the backup data sub-packets of this power data packet in each backup cache node to obtain the original power data packet; Obtain the distributed cache node of the power node corresponding to this power data packet in the current acquisition period, re-divide this power data packet to obtain new distributed data sub-packets, and upload each new distributed data sub-packet to the corresponding distributed cache node for re-caching.

6. The power data caching optimization system based on a distributed terminal according to claim 5, characterized in that, Obtain the cache hit rate. The process of expanding the cache node includes: After the power data packet is cached, the user accesses the power data of its power node through the cache center. The cache hit rate refers to the ratio of the number of times each cache node successfully reads its power data sub-packet to the total number of accesses; Set the hit rate threshold, compare the cache hit rate of each cache node with the hit rate threshold respectively to obtain the nodes to be expanded. For the nodes to be expanded, add a new cache node on the basis of its original cache node, and update each cache node set.

Images