Power data cache optimization system based on distributed terminals

Abstract

The invention discloses an electric power data cache optimization system based on a distributed terminal, and relates to the technical field of data cache. The method comprises the following steps: acquiring power nodes, deploying corresponding cache nodes, setting corresponding power parameters for different power nodes, generating cache parameters of each cache node, acquiring distributed cache nodes and backup cache nodes of each power node, acquiring distributed data sub-packets and backup data sub-packets, and respectively caching the distributed data sub-packets and the backup data sub-packets, acquiring data integrity and cache hit rate of each cache node, re-caching the power data packet according to the data integrity, and performing node expansion on the cache nodes according to the cache hit rate; effective reference indexes can be provided for distributed data caching according to data acquisition quantities and equipment load conditions in different time periods, and the caching efficiency of different caching nodes on power data packets can be obviously improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data cache, and in particular to a distributed terminal-based power data cache optimization system. Background Art

[0002] Distributed data caching and optimization of power data is a vital component of the power system. It aims to improve data access speed, reduce database pressure, and optimize overall system performance. By distributing data storage on multiple cache nodes in the network, it can achieve efficient data access and processing. In the field of data caching, this system can significantly improve the real-time, reliability, and stability of the power system. In the prior art, most cache nodes used for distributed data caching lack flexibility, ignoring the fact that the cache capabilities of cache nodes in different time periods are also different. A flexible caching mechanism for data caching cannot be implemented based on this feature. In addition, the division of cached data also lacks flexibility, and a corresponding division mechanism cannot be formed according to the selection of cache nodes. In view of the shortcomings of the prior art, the present invention provides a power data caching optimization system based on distributed terminals. Summary of the invention

[0003] The object of the present invention is to provide a distributed terminal-based power data cache optimization system.

[0004] The purpose of the present invention can be achieved through the following technical solution: A distributed terminal-based power data cache optimization system, comprising the following modules: A node deployment module is used to obtain power nodes in the power system and deploy corresponding cache nodes according to the power nodes using virtual machine technology; A data acquisition module is used to obtain power data packets of each power node and set corresponding power parameters for different power nodes according to the data packet size of the power data packet; A node selection module is used to obtain cache information of each cache node and generate cache parameters of each cache node according to the cache information combined with corresponding power parameters; A data cache module is used to obtain a cache node set of each power node, obtain a number of distributed cache nodes and backup cache nodes in the cache node set according to the cache parameters, perform distribution division and backup division on the power data packet to obtain distributed data sub-packets and backup data sub-packets, and cache the distributed data sub-packets and backup data sub-packets respectively; The node optimization module is used to obtain the data integrity and cache hit rate of each cache node respectively, re-cache the power data packet according to the data integrity, and expand the cache node according to the cache hit rate.

[0005] Furthermore, the process of obtaining power nodes in the power system and deploying corresponding cache nodes according to the power nodes using virtual machine technology includes: The power node refers to a physical device that performs specific functions and generates relevant data in the power system, including five types of nodes: power generation node, transmission node, substation node, distribution node, and power consumption node; The location information and connection information of each power node are collected respectively, and a corresponding cache node is built for each power node using virtual machine technology. The cache node is deployed according to the location information, and the cache node is connected according to the connection information. The deployment distances of different cache nodes are obtained according to the deployment positions of the cache nodes, and the node types of the cache nodes are consistent with the node types of their power nodes.

[0006] Furthermore, the process of obtaining power data packets of each power node and setting corresponding power parameters for different power nodes according to the data packet size of the power data packets includes: Setting a collection cycle, taking all power data of each power node in the same collection cycle as its power data packet in the collection cycle, wherein the power data refers to the relevant data generated by the power node during its working process; According to the data packet size S of the power data packets of each power node of the same node type in the same collection cycle i , i=1, 2, ..., n, n is the number of power nodes of this node type, and the power parameter D of a single power node is obtained. i ;

[0007] Among them, when When , the corresponding power parameter is positive. When , the corresponding power parameter is negative. When , the corresponding power parameter is 0, and the latest power parameter is continuously used as the power parameter of each power node in the current collection cycle.

[0008] Furthermore, the process of obtaining cache information of each cache node and generating cache parameters of each cache node according to the cache information and corresponding power parameters includes: The cache information refers to the load of the virtual machine corresponding to the cache node, including CPU utilization C a 、Memory usage C b 、Network bandwidth C c , Disk read volume C d 、Disk write volume C e , set corresponding weight values ​​for different cache information, respectively Q a , Qb , Q c , Q d , Q e ; Combined with the power parameters D of the power nodes corresponding to each cache node i , generate cache parameters H of each cache node in the corresponding collection period;

[0009] The cache parameters when the power parameter is 0 are used as standard cache parameters, and the cache parameters of each cache node in the current acquisition cycle are compared with the standard cache parameters to obtain the status of each cache node, including unavailable status and available status.

[0010] Furthermore, the process of obtaining a cache node set of each power node and obtaining a plurality of distributed cache nodes and backup cache nodes from the cache node set according to the cache parameters includes: Take the cache node corresponding to a single power node as the cache center, obtain other cache nodes of the same node type as the cache center and the deployment distance between them and the cache center, and include all cache nodes whose distance is less than or equal to a preset fixed distance into the cache node set of the power node; Obtain cache parameters of each cache node in the cache node set in the current collection cycle, arrange the available cache nodes in the cache node set in ascending order according to the cache parameters, and obtain 2k cache nodes therefrom; The 2k cache nodes are randomly divided into two groups of equal number, one group is the distributed cache nodes, and the other group is the backup cache nodes. The distributed cache nodes and backup cache nodes in the same collection cycle are bound to their power nodes.

[0011] Furthermore, the process of performing distribution division and backup division on the power data packet to obtain a distribution data sub-packet and a backup data sub-packet, and caching the distribution data sub-packet and the backup data sub-packet respectively includes: Obtaining a proportional relationship of cache parameters of each distributed cache node belonging to the same power node in the same acquisition cycle, wherein the distribution division refers to dividing the power data packet of the power node according to the proportional relationship to obtain a plurality of distributed data sub-packets, and uploading each distributed data sub-packet to a distributed cache node corresponding to the proportional relationship for caching; The proportional relationship of cache parameters of each backup cache node belonging to the same power node within the same acquisition cycle is obtained, and the backup division refers to dividing the power data packet of the power node according to the proportional relationship to obtain a number of backup data sub-packets, and uploading each backup data sub-packet to the backup cache node corresponding to it in the proportional relationship for caching.

[0012] Furthermore, the process of obtaining the data integrity of each cache node and re-caching the power data packet according to the data integrity includes: Before the distributed cache node caches the distributed data sub-packet, an initial hash value of the distributed data sub-packet is obtained, and after the cache is completed, subsequent hash values ​​of the distributed data sub-packet are periodically obtained, and the subsequent hash values ​​are compared with the initial hash value to obtain the data integrity of the distributed data sub-packet, including a complete state and an incomplete state; For the incomplete distributed data sub-packet, the distributed data sub-packet corresponding to the distributed data sub-packet in each distributed cache node is deleted, and the backup data sub-packets of the power data packet in each backup cache node are combined to obtain the original power data packet; Obtain the distribution cache node of the power node corresponding to the power data packet in the current acquisition cycle, distribute and divide the power data packet again to obtain new distribution data sub-packets, and upload each new distribution data sub-packet to the corresponding distribution cache node for re-caching.

[0013] Furthermore, the cache hit rate of each cache node is obtained, and the process of expanding the cache node according to the cache hit rate includes: After the power data packet is cached, the user accesses the power data of its power node through the cache center. The cache hit rate refers to the ratio of the number of times each cache node successfully reads its power data sub-packet to the total number of accesses; A hit rate threshold is set, and the cache hit rate of each cache node is compared with the hit rate threshold to obtain the node to be expanded. For the node to be expanded, a new cache node is added based on its original cache node, and each cache node set is updated.

[0014] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention provides a basis for realizing distributed data caching by deploying corresponding cache nodes according to power nodes respectively, setting flexible power parameters for power nodes according to the size of power data packets in different acquisition cycles, and generating targeted cache parameters for them in combination with the cache information of cache nodes in the corresponding acquisition cycle, which is conducive to providing effective reference indicators for distributed data caching according to the data collection volume and equipment load conditions in different time periods; 2. The present invention constructs a cache node set for each power node, and simultaneously obtains a group of distributed cache nodes and a group of backup cache nodes according to the cache parameters, divides the power data packet according to the proportional relationship of the cache parameters of each cache node and caches them separately, which is beneficial to divide targeted power data sub-packets for different cache nodes according to their cache capabilities, and can significantly improve the cache efficiency of different cache nodes for power data packets. By obtaining the data integrity and cache hit rate of the cache node and optimizing the cache node accordingly, it is beneficial to reduce the cache pressure of a single cache node while ensuring the integrity of the data cache, thereby improving the effectiveness of the data cache. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram of the present invention. DETAILED DESCRIPTION

[0016] like Figure 1 As shown, a distributed terminal-based power data cache optimization system includes the following modules: A node deployment module is used to obtain power nodes in the power system and deploy corresponding cache nodes according to the power nodes using virtual machine technology; A data acquisition module is used to obtain power data packets of each power node and set corresponding power parameters for different power nodes according to the data packet size of the power data packet; A node selection module is used to obtain cache information of each cache node and generate cache parameters of each cache node according to the cache information combined with corresponding power parameters; A data cache module is used to obtain a cache node set of each power node, obtain a number of distributed cache nodes and backup cache nodes in the cache node set according to the cache parameters, perform distribution division and backup division on the power data packet to obtain distributed data sub-packets and backup data sub-packets, and cache the distributed data sub-packets and backup data sub-packets respectively; The node optimization module is used to obtain the data integrity and cache hit rate of each cache node respectively, re-cache the power data packet according to the data integrity, and expand the cache node according to the cache hit rate.

[0017] It should be further explained that, in the specific implementation process, the process of obtaining the power nodes in the power system and deploying the corresponding cache nodes according to the power nodes using the virtual machine technology includes: The power system is divided into five power links: power generation, transmission, transformation, distribution, and consumption. The power node refers to the physical equipment that performs specific functions in each power link and generates relevant data, including five types of nodes: power generation node, transmission node, transformation node, distribution node, and consumption node; The location information and connection information of each power node are collected respectively, wherein the location information refers to the geographical location of the physical device corresponding to each power node, and the connection information refers to the connection relationship between the physical devices corresponding to each power node; A corresponding cache node is constructed for each power node using virtual machine technology, and the cache node is used to cache data. The constructed cache nodes are deployed according to the location information of the power nodes, so that the deployment location of each cache node corresponds to the geographical location of its power node, and the corresponding cache nodes are connected according to the connection information of the power nodes; The geographical distance between different power nodes can be obtained according to the geographical location of the power nodes, and the deployment distance between different cache nodes can also be obtained according to the deployment location of the cache nodes, and the node type of the cache nodes is consistent with the node type of its power nodes.

[0018] It should be further explained that, in a specific implementation process, the process of obtaining power data packets of each power node and setting corresponding power parameters for different power nodes according to the data packet size of the power data packet includes: Set a collection cycle to collect power data of each power node in real time. The power data refers to the relevant data generated by the power node during its working process. All power data collected in the same collection cycle are regarded as the power data packet of the power node in the collection cycle. The power data of each power node of the same node type in the same collection cycle are included in the same power data packet; Get the data packet size of the power data packet of each power node of the same node type in the same acquisition cycle, denoted as S i , where i=1, 2, ..., n, where n is the number of power nodes of this node type, based on which the power parameters of a single power node are obtained, denoted as D i ;

[0019] Among them, when When , the corresponding power parameter is positive. When , the corresponding power parameter is negative. When , the corresponding power parameter is 0; The same method is adopted to obtain the power parameters of power nodes of different node types within the collection cycle, and the latest power parameters are continuously used as the power parameters of each power node in the current collection cycle to achieve continuous updating of the power parameters.

[0020] It should be further explained that, in a specific implementation process, the process of obtaining cache information of each cache node and generating cache parameters of each cache node according to the cache information combined with corresponding power parameters includes: The cache node is built inside the virtual machine, each cache node corresponds to a virtual machine, and the cache information refers to the load of the virtual machine corresponding to the cache node, including CPU utilization, memory utilization, network bandwidth, disk read volume, and disk write volume; Take any cache node as an example, and mark the CPU utilization, memory utilization, network bandwidth, disk read volume, and disk write volume of the cache node as C a , C b , C c , C d , C e , set corresponding weight values ​​for different cache information, namely Q a , Q b , Q c , Q d , Q e ; Combined with the power parameter D of the power node corresponding to the cache node i , generate the cache parameters of the cache node, denoted as H;

[0021] The same method is adopted to obtain the cache parameters of each cache node in the same collection cycle, and the latest cache parameters are continuously used as the cache parameters of each cache node in the current collection cycle to realize the continuous update of the cache parameters; D i The cache parameter when H is 0 is used as the standard cache parameter, recorded as H0. The cache parameters in the current acquisition cycle are compared with the standard cache parameters. If H>H0, the cache node is marked as unavailable. If H≤H0, the cache node is marked as available.

[0022] It should be further explained that, in a specific implementation process, the process of obtaining a cache node set of each power node and obtaining a plurality of distributed cache nodes and backup cache nodes in the cache node set according to the cache parameters includes: Take any power node as an example, take the cache node corresponding to the power node as the cache center, obtain the deployment distance between each cache node of the same node type and the cache center, and include all cache nodes whose distance is less than or equal to the preset fixed distance into the cache node set of the power node; Obtain cache parameters of each cache node in the cache node set in the current collection cycle, sort the available cache nodes in the cache node set in ascending order according to the cache parameters, and obtain 2k cache nodes from them. The obtained 2k cache nodes must meet two conditions: first, they are available, and second, they have the minimum cache parameters. The 2k cache nodes are randomly divided into two groups. The number of cache nodes in the two groups is the same, both k. One group is marked as distributed cache nodes, and the other group is marked as backup cache nodes. The distributed cache nodes and backup cache nodes obtained in the same acquisition cycle are bound to their power nodes.

[0023] It should be further explained that, in a specific implementation process, the process of performing distribution division and backup division on the power data packet to obtain the distribution data sub-packet and the backup data sub-packet, and caching the distribution data sub-packet and the backup data sub-packet respectively includes: Taking the distributed cache node of any power node as an example, the proportional relationship between the cache parameters of each distributed cache node is obtained, and the distribution division refers to dividing the power data packet of the power node according to the proportional relationship to obtain a plurality of distributed data sub-packets; The number of the distributed data sub-packets is k, the data packet size of each distributed data sub-packet corresponds to the proportional relationship, and each distributed data sub-packet is uploaded to a distributed cache node corresponding to the proportional relationship for caching; The same method is adopted to obtain the proportional relationship between the cache parameters of each backup cache node. The backup division refers to dividing the power data packet of the power node according to the proportional relationship to obtain a number of backup data sub-packets, and uploading each backup data sub-packet to the backup cache node corresponding to it in the proportional relationship for caching.

[0024] It should be further explained that, in the specific implementation process, the process of obtaining the data integrity of each cache node and re-caching the power data packet according to the data integrity includes: Before the distributed cache node caches the distributed data sub-packet, an initial hash value of the distributed data sub-packet is obtained. After the cache is completed, the same hash algorithm is used to periodically obtain subsequent hash values ​​of the distributed data sub-packet, and the obtained subsequent hash values ​​are compared with the initial hash value. The data integrity includes a complete state and an incomplete state. If the two are the same, the distributed data sub-package is judged to be in a complete state and no other operations are performed on it. If the two are different, the distributed data sub-package is judged to be in an incomplete state. For the incomplete distributed data sub-packet, the distributed data sub-packet corresponding to the distributed data sub-packet in each distributed cache node is deleted, and the backup data sub-packets of the power data packet in each backup cache node are combined to obtain the original power data packet; Obtain the distribution cache node of the power node corresponding to the power data packet in the current acquisition cycle, distribute and divide the power data packet again to obtain new distribution data sub-packets, and upload each new distribution data sub-packet to the corresponding distribution cache node for re-caching.

[0025] It should be further explained that, in the specific implementation process, the process of obtaining the cache hit rate of each cache node and expanding the cache node according to the cache hit rate includes: After the power data packet is cached, the user can access the power data of its power node through the cache center. The cache hit rate refers to the ratio of the number of times each cache node successfully reads its power data sub-packet to the total number of accesses; Set a hit rate threshold, compare the cache hit rate of each cache node with the hit rate threshold, mark the cache nodes whose cache hit rate is greater than or equal to the hit rate threshold as nodes to be expanded, and for the nodes to be expanded, add a new cache node based on its original cache node, and update each cache node set.

[0026] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A distributed terminal-based power data cache optimization system, characterized in that: Includes the following modules: A node deployment module is used to obtain power nodes in the power system and deploy corresponding cache nodes according to the power nodes using virtual machine technology; A data acquisition module is used to obtain power data packets of each power node and set corresponding power parameters for different power nodes according to the data packet size of the power data packet; A node selection module is used to obtain cache information of each cache node and generate cache parameters of each cache node according to the cache information combined with corresponding power parameters; A data cache module is used to obtain a cache node set of each power node, obtain a number of distributed cache nodes and backup cache nodes in the cache node set according to the cache parameters, perform distribution division and backup division on the power data packet to obtain distributed data sub-packets and backup data sub-packets, and cache the distributed data sub-packets and backup data sub-packets respectively; The node optimization module is used to obtain the data integrity and cache hit rate of each cache node respectively, re-cache the power data packet according to the data integrity, and expand the cache node according to the cache hit rate.

2. The distributed terminal-based power data cache optimization system according to claim 1, characterized in that: The process of acquiring power nodes and deploying corresponding cache nodes includes: The power node refers to a physical device that performs specific functions and generates relevant data in the power system, including five types of nodes: power generation node, transmission node, substation node, distribution node, and power consumption node; The location information and connection information of each power node are collected respectively, and a corresponding cache node is built for each power node using virtual machine technology. The cache node is deployed according to the location information, and the cache node is connected according to the connection information. The deployment distances of different cache nodes are obtained according to the deployment positions of the cache nodes, and the node types of the cache nodes are consistent with the node types of their power nodes.

3. The distributed terminal-based power data cache optimization system according to claim 2, characterized in that: The process of obtaining power data packets and setting corresponding power parameters for different power nodes includes: Setting a collection cycle, taking all power data of each power node in the same collection cycle as its power data packet in the collection cycle, wherein the power data refers to the relevant data generated by the power node during its working process; According to the data packet size S of the power data packets of each power node of the same node type in the same collection cycle i , i=1, 2, ..., n, n is the number of power nodes of this node type, and the power parameter D of a single power node is obtained. i ; Among them, when When , the corresponding power parameter is positive. When , the corresponding power parameter is negative. When , the corresponding power parameter is 0, and the latest power parameter is continuously used as the power parameter of each power node in the current collection cycle.

4. The distributed terminal-based power data cache optimization system according to claim 3, characterized in that: The process of obtaining cache information and generating cache parameters for each cache node includes: The cache information refers to the load of the virtual machine corresponding to the cache node, including CPU utilization C a 、Memory usage C b 、Network bandwidth C c , Disk read volume C d 、Disk write volume C e , set corresponding weight values ​​for different cache information, respectively Q a , Q b , Q c , Q d , Q e ; Combined with the power parameters D of the power nodes corresponding to each cache node i , generate cache parameters H of each cache node in the corresponding collection period; The cache parameters when the power parameter is 0 are used as standard cache parameters, and the cache parameters of each cache node in the current acquisition cycle are compared with the standard cache parameters to obtain the status of each cache node, including unavailable status and available status.

5. The distributed terminal-based power data cache optimization system according to claim 4, characterized in that: The process of obtaining a cache node set and a plurality of distributed cache nodes and backup cache nodes includes: Take the cache node corresponding to a single power node as the cache center, obtain other cache nodes of the same node type as the cache center and the deployment distance between them and the cache center, and include all cache nodes whose distance is less than or equal to a preset fixed distance into the cache node set of the power node; Obtain cache parameters of each cache node in the cache node set in the current collection cycle, arrange the available cache nodes in the cache node set in ascending order according to the cache parameters, and obtain 2k cache nodes therefrom; The 2k cache nodes are randomly divided into two groups of equal number, one group is the distributed cache nodes, and the other group is the backup cache nodes. The distributed cache nodes and backup cache nodes in the same collection cycle are bound to their power nodes.

6. The distributed terminal-based power data cache optimization system according to claim 5, characterized in that: The process of obtaining the distribution data sub-package and the backup data sub-package and caching them respectively includes: Obtaining a proportional relationship of cache parameters of each distributed cache node belonging to the same power node in the same acquisition cycle, wherein the distribution division refers to dividing the power data packet of the power node according to the proportional relationship to obtain a plurality of distributed data sub-packets, and uploading each distributed data sub-packet to a distributed cache node corresponding to the proportional relationship for caching; The proportional relationship of cache parameters of each backup cache node belonging to the same power node within the same acquisition cycle is obtained, and the backup division refers to dividing the power data packet of the power node according to the proportional relationship to obtain a number of backup data sub-packets, and uploading each backup data sub-packet to the backup cache node corresponding to it in the proportional relationship for caching.

7. The distributed terminal-based power data cache optimization system according to claim 6, characterized in that: The process of obtaining data integrity and re-caching the power data packet includes: Before the distributed cache node caches the distributed data sub-packet, an initial hash value of the distributed data sub-packet is obtained, and after the cache is completed, subsequent hash values ​​of the distributed data sub-packet are periodically obtained, and the subsequent hash values ​​are compared with the initial hash value to obtain the data integrity of the distributed data sub-packet, including a complete state and an incomplete state; For the incomplete distributed data sub-packet, the distributed data sub-packet corresponding to the distributed data sub-packet in each distributed cache node is deleted, and the backup data sub-packets of the power data packet in each backup cache node are combined to obtain the original power data packet; Obtain the distribution cache node of the power node corresponding to the power data packet in the current acquisition cycle, distribute and divide the power data packet again to obtain new distribution data sub-packets, and upload each new distribution data sub-packet to the corresponding distribution cache node for re-caching.

8. The distributed terminal-based power data cache optimization system according to claim 7, characterized in that: The process of obtaining the cache hit rate and expanding the cache nodes includes: After the power data packet is cached, the user accesses the power data of its power node through the cache center. The cache hit rate refers to the ratio of the number of times each cache node successfully reads its power data sub-packet to the total number of accesses; A hit rate threshold is set, and the cache hit rate of each cache node is compared with the hit rate threshold to obtain the node to be expanded. For the node to be expanded, a new cache node is added based on its original cache node, and each cache node set is updated.

Images